First, we had data warehouses. Now – as the sheer volume of data has proliferated – we have data lakes. The main difference is that data lakes can deal with structured and unstructured data. And the cloud proved to be a turning point for data lakes – it reduced the complexity of Hadoop management and gave a very cost-effective solution to data storage. The question is, will data lakes prevail? Or is something else on the horizon?

Data mesh seems to be the buzzword of the moment and everyone wants one. But not many organisations understand what it is – or what it does. In a nutshell, data mesh is the way in which data domains can use data products from other domains and how data from multiple domains can be combined to get a more rounded view. But in a lot of implementations, we have seen a number of common factors when data mesh projects go wrong.

Now you’ve made the decision to go for cloud, the next step is, do you go for open-source or a vendor licensed model? There are pros and cons to both. But here we suggest looking at three key factors: weighing up the cost; judging how flexible you need to be; and how much support you require, before you make a call. Whether you go for vendor licensed or open-source is really dependent on your organisation and its specific needs. Doing the groundwork is important before you decide which approach to follow

As data has proliferated, the data warehouse has given way to the data lake. Data lakes have proved to be less structurally restrictive, but are able to unleash almighty processing power. The problem is that data lakes can very easily turn into data swamps, where nothing is documented, no one can find what they are looking for and security is shaky. Read how a Databricks lakehouse could be the answer and how Valcon has worked with a global construction company to implement a Databricks solution

Everyone wants to jump onto the AI bandwagon. But there’s a lot to be said for being able to walk before you try and run. If you don’t have the right building blocks in place, AI just won’t work. So putting in some time to get this right is vital. The starting point is the data foundation, the quality and integrity of your data. After this it’s the data management – the right structures and frameworks. And last but not least, the right team. By putting this foundation in place, you’ll be well on your way to AI success.

Data pipelines are the backbone of modern data-driven enterprises and facilitate the seamless flow of data from source systems to its destination. Data catalogues are instrumental in providing a 360 view of the entire data eco system. Their traditional use has been in data discovery and governance but latest generation data catalogues mean they include data pipeline monitoring capabilities and help with identifying and resolving any issues.  

A client, one of the UK’s largest transporters of fuel and lubricants, was not short of data but was having difficulty in extracting meaningful insight which would help the company manage supply and demand and implement a dynamic pricing strategy. To address this issue and enable the company to turn this data into management information, Valcon built a state-of-the-art data pipeline using Microsoft Fabric.

Our client, a globally renowned damage recovery contractor, was struggling with a manual time-consuming invoicing system – the production of invoices was a protracted, three-week process. The company worked with Valcon to implement a Microsoft Fabric real time AI invoice processing system, which transformed their financial operations, improved cash flow and even boosted margins.

Meeting regulatory deadlines with a data warehouse and data model

Our client – a large financial institution – was struggling with meeting regulatory standards and needed to get its data house in order swiftly. Believing data warehousing and a new comprehensive data model were the answer, the bank partnered with Valcon to guide and implement a future proof solution to create consistency and encourage enterprise-wide adoption of the new data approach. The upshot has been regulatory adherence, a single source of the truth in data terms and improved performance and scalability.

Is it only for big issues or can you use it for any old report?

Data storytelling is a highly useful skill for any data professional to have. Where line graphs, pie charts and bar charts can give you a visual representation of data, data storytelling can help to bring any data to life and give it a clear narrative. It’s not just for the big issues or big stories – data storytelling can help bring any corporate report to life and make it actionable.

Time to act on effective reporting?

Sustainability objectives are no longer just a nice to have for large and listed organisations – they are a regulatory imperative. And as a result, sustainability reporting – just as financial reporting before it – has become a corporate priority. Knowing where in the business you make an environmental and social impact, and identifying opportunities to make a more positive contribution to the environment and to society is only possible through data insight and a sustainability reporting function. Read our top tips on how to set up sustainability reporting.

Delivering for a large international retailer

In the drive to report on sustainability and climate goals, our client, a large global retailer needed to up its data game. Calling on Valcon to lead its sustainability programme, the focus was on data collection processes, automation of data gathering, data transformation and visualisation to provide the insights it needed for climate reporting, meeting regulatory targets and helping the firm to do its bit to save the planet and make a meaningful contribution to society.

Keeping ICU beds in check by spreading the COVID-19 patient load

The COVID pandemic was an extremely tense situation and put many global healthcare providers under severe strain and pressure. Equitable patient distribution was vital if severely impacted COVID patients – and other hospital patients – were to get the best care. Valcon and Delta-N worked together to provide our client with the right data architecture and analysis capabilities to ensure hospitals were not overwhelmed with patients, hospital occupancy was maximised and our client could get the right insight it needed to make appropriate policy changes

Is your organisation data-driven?

Being data driven is something a lot of organisations aspire to. They understand the benefits that data analysis and deep insight brings to the corporate decision-making process. But few are hitting the mark, according to the Harvard Business Review. It might be that there is a lot going on and a lot of money is being spent, but efforts are uncoordinated. Or it might be that stakeholders aren’t interpreting the insight in the right way. To properly realise the benefits of data analysis, you need to build a successful data analytics organisation. This blog shows you six steps towards building an effective DAO.

Enabling business communications

Although the concept of data modelling started to evolve in the sixties, it tended to take a back seat in the priorities of organisations. This was because as computer processing power increased, the weight of data being collected didn’t really impact performance – so organisations felt they didn’t need data models. The result was chaos, with different departments using, storing, interpreting and categorising data in very different ways. So data models are having a moment again – and it’s about time.

Data management – the role it plays

Data management might sound a little obvious. But it is undoubtedly the most important piece in the world of data jigsaw. Without it, data would be free flowing, structure-less, meaningless. Data management gives data purpose and categorisation. It ensures it is of sound quality. It is about establishing the right governance, the rules and parameters that harness data and ensure it’s used in the right way. And when done properly, data management is the door opener to lots of other opportunities for the organisation.

The control tower of your organisation

If a control tower oversees the safe and efficient flow of air traffic, then data governance oversees the smooth and secure management of data assets. Done properly, it will help your organisation run more effectively and steal a march on competitors. But do you know where you are on a data governance maturity cycle? You should do. If you don’t, you might be embarking on initiatives that you aren’t ready for. A data maturity assessment is key if your data governance initiative is to fly high.

The five essential characteristics

Too few skilled people to get work done under tight deadlines is a common complaint on data projects. This is where a hybrid approach can be the perfect solution. It means you have a team based close to the data project who collaborate with colleagues who work in a nearshore location – and so you offset the scarcity of local data resources.

Speaking one language across a big organisation

These days organisations are swimming in data. But it’s often in separate departments, countries and systems and few organisations have one ‘version of the truth’ in data terms. Our brewing client was the same – they aspired to be the best-connected brewery in the world and they wanted to achieve it through excellent data management. Here is how the data team – comprising the brewer’s own people and Valcon – established a common business language.

Using a data catalog to find your way through the data jungle

It is estimated that 90% of the world’s data was generated in the last two years alone. And that the total amount of data created, captured, copied and consumed on a global basis is estimated to reach a breathtaking 180 zettabytes by 2025 (Statista). It is no wonder organisations find themselves in the midst of a data jungle with little idea of how to navigate it. Data catalogs make a great map. Here are some top tips for creating your data catalog.

How an automotive company used MDM to become a data driven supplier of spare parts

Supply chain issues have hit the automotive parts industry hard in the last few years since the pandemic. Our client felt their approach to data management was exacerbating this issue and also hindering their operational efficiency. They wanted to become a data driven organisation, so with Valcon, embarked on a master data management (MDM) programme. Read here to find out more about the project and about the benefits it brought.

The importance of data pools in improving product data

For food retailers, building an accurate picture of the product they put on their shelves is vital. They need to know the ingredients that went into that sandwich and where they came from – missing vital allergen information can have fatal consequences. So data alignment is absolutely crucial. It’s not an easy thing to accomplish – it requires accurate and complete data, all suppliers and retailers using the same data formats, and it requires close collaboration between these stakeholders. None of this is easy. But it becomes much easier when you bring a data pool into the picture.

Success factors for effective asset information management

For large companies with complex asset infrastructure, it is often very difficult to keep track of asset data such as design, specification, maintenance history, inspection results and outage information. But having it all nailed down in one place will help you to rein in costs and optimise your asset network. Asset information management (AIM) is a great place to start. Read about how to successfully approach an AIM programme here.

Clear Skies Ahead: Soaring to Customer 360 Excellence

Next generation Customer 360 reveals the true story of the customer. There might be a good reason why they haven’t paid their invoice – and if you have the full picture, you can approach that situation in the right way. But Customer 360 implementations can often go south. And the reason is often a lapse in data management and poor data governance. Get your approach right and the sky is the limit with your AI driven Customer 360 initiative.

Food producer uses data platform to spark organisational change

Our client – a global food production company – had grown significantly through acquisition and had over 30 production outlets in multiple countries. In data terms this caused significant problems as there was a lack of cohesion and it was difficult to get a strategic overview of the company’s operations. Valcon helped the firm develop its strategy on master data management and data analytics and implemented it in support of their corporate ‘one company’ strategy.

Data leakage – are you ready to plug the gap? 

Data leaks can cause untold damage – and need to be avoided at all costs. But if you don’t know what data you have, where it is stored or who is responsible for it, then you are vulnerable and at significant threat from a leak. The right data management and data governance approach will put you on the right track and help you do everything you can to avoid a data leak.

Helping a global port become a data led organisation

A large international port handling high levels of shipping traffic was facing the challenge of having to deal with large volumes of data across multiple systems. It also wanted to manage its energy consumption more effectively. And it knew the answer to these problems lay in effective data management. Valcon and Profisee worked together to implement a master data management (MDM) system for the port bringing benefits like an uplift in efficiency and insight into its energy consumption.

The future of artificial intelligence

The growth in the artificial intelligence (AI) market is set to skyrocket, with estimates of it being worth nearly $2trillion by 2030. AI already underpins a lot of things that we do – facial recognition on smart phones, Alexa and Siri, recommendations in e-commerce platforms. And large language applications like ChatGPT and BART have made a lot of headlines this year. There are the downsides. It’s not a perfect technology. There are worries of it getting into the hands of ‘bad actors.’ And it will displace a lot of jobs in the coming years. But the opportunities we have to improve life with AI are significant.

The rise of Machine Learning and how to make it a success

Machine learning has been around for much longer than you’d think. Back in the 1950s, IBM’s Arthur Samuels used an early form of machine learning while working on a program that played chequers. And in the last 70 years, machine learning has progressed a lot. We’re not quite at the stage where AI could apply itself to as many diverse problems as the human brain, but a majority of experts think we’ll be there some time within the next 100 years. As thousands of organisations eye up machine learning in terms of how it can improve their businesses, what are the key factors to bear in mind for a successful ML implementation?

The progression of chatbots to chatbot 3.0

ChatGPT has taken up acres of media coverage since its launch last November and conjecture about the changes it will bring and the professions it will wipe out. You’d be forgiven for thinking ChatGPT was the first of its kind. But there was a long history of chatbots that came before it. We are at the start of a golden age of chatbots and generative AI but we also have a long way to go.

Coming soon

Monitoring railway assets with computer vision

A European railway network provider needed to transform its manual track inspection process to make it more efficient, cost effective and crucially, more accurate. Valcon, an expert in computer vision – a data driven software and machine learning solution – was brought in to develop an automated railway track inspection system. As well as driving down costs, the new system has improved the accuracy of inspections, improved safety and reduced the need for track closures.

Data in airports – harnessing data for security lanes optimisation

Following a turbulent few years in the travel sector, an international airport was struggling with surging post pandemic travel flows, whilst managing with significantly reduced staff numbers. One key problem was the airport’s security process – major queues were developing and it was becoming a significant bottleneck in the airport’s daily functioning. The airport thought that data might be the solution to security woes. Valcon was hired to build a data blueprint of the security process, to help identify problematic areas and come up with solutions.

The answer to solving financial crime lies in data

Financial crime is the scourge of the global economy, costing over $2trillion annually. Financial criminals have traditionally been one step ahead of the banks, who have struggled with legacy systems, the complexity of the global financial system and adhering to regulatory pressures. The answer to fighting financial crime – and being compliant – lies in data. Data science, data engineering and effective data management have a vital role to play.

Pricing predictions through machine learning and predictive analytics for leading automotive repair organisation

Many organisations follow an unstructured approach to pricing, with humans making decisions based on poor data or worse, on opinion. Valcon has a pricing approach that creates hundreds of customer segments based on historical data and important pricing influencers, like the weather. 

Pricing advice is supplied to decision makers via a pricing app or engine. With this information, Valcon customers can dynamically price their products and services and can benefit from improved margins, quicker decisions and being able to balance short term customer wins vs long term customer retention.  

Manufacturers, fuel distributors and retailers have successfully implemented our approach.