What is Databricks?
“Databricks is a tool that makes it easy to realise a data platform on which analysis can be performed on large amounts of data. The tool can be used for various issues, such as complex tasks related to machine learning, but also, for example, for tracking KPIs. The great thing about the Databricks platform is that it is suitable for all types of data professionals within an organisation: data engineers, data scientists and data analysts can each work together to process and analyse data from their perspectives. In that way, there is no need to purchase separate tooling.”
Can you give an example of a case where Valcon has applied Databricks?
“We recently used Databricks at an online retailer and e-commerce platform. This client had recently acquired another organisation. The acquired organisation had an immense data warehouse, which was managed locally. This data warehouse needed to be migrated to the public cloud.
Because the cloud strategy team within the organisation had determined that AWS would become the target platform for data storage, the choice of AWS was a given fact. Then, a selection was made for analytical tooling. We compared several alternatives. From this set of alternatives, Databricks emerged as the best option.
The client’s data analysts were accustomed to working with Spark SQL. Databricks is also based on Spark. Therefore, we expected that Databricks could easily be adopted by the client. In addition, the performance of Databricks was excellent, and the analysts were enthusiastic about the user interface. These were important arguments for choosing Databricks as an analytical layer on top of the data warehouse.”
You cite an example of a customer in the retail industry. Is Databricks being used at Valcon within other industries as well?
“We use Databricks for customers within different industries. The use of Databricks is independent of the industry. What our clients have in common is that they work with large amounts of data and want all their employees to be able to work with data from the same tooling.”
What are development opportunities for Databricks?
“The balance between cost and performance is a crucial part of an analytics platform. Doing analytics requires a lot of computing power, and this can be costly. To ensure that the platform remains cost-efficient while being scalable in use, Databricks recently introduced a serverless option. With this new option, Databricks takes over the management of its clients’ infrastructure, which greatly increases flexibility. Valcon has assisted the first clients in transitioning to this technology. We look forward to working with Databricks to further develop this option for our clients.”
You lead the Databricks community within Valcon. What does this community entail?
“The community within Valcon consists of a core team of enthusiastic data engineers and data scientists who can work well with Databricks. With this core team, we train other colleagues in the use of Databricks. For example, we provide training during the Data Academy – a two-month programme for starters. We also offer more advanced workshops for colleagues who have been working at Valcon for some time and have experience with Databricks. We see Databricks being used by increasingly more clients, and so it is important to keep developing ourselves in this.”
Want to learn more? If you want to learn more about Databricks within Valcon, please email [email protected] and we’ll be in touch right away.
Databricks is one of Valcon’s trusted partners that help us deliver the best-in-class solutions for our clients. See our other partners here.
