Virtual Colour Systems
Data Engineer - Business Intelligence - C# & Blazor
Data Analytics - Data Mining - Data Transformation
Founded in 1997 as an IT consultancy to help support my doctoral research in colour science. The showcase application coded in Blazor, C#, and HTML5 is accessible thought the Virtual Colour Atlas link.
The ability to work in both software and data engineering environments provides a unique hybrid skillset allowing for the design and development of end-to-end systems that employ disparate technologies.
Glasgow Caledonian University was transferring data from an in-house student IT system to a packaged solution that involved extensive SQL and SSIS skills for the core data. The student images were held in a non-standard format that the original provider stated could not converted, however using coding skills the images were byte analysed and code was developed that extracted the raw data (image, colour depth, width, height) and re-constructed the images as jpg.
Performance tuning: applications, databases, cubes and ETL jobs.
Scottish Widows in-house web application was modified to run 11 times faster via improved coding and UI.
An Azure process (database -> DataFactory -> cubes -> PowerBI) at ScotRail that took 3 hours and 45 minutes was redeveloped to complete in less than 15 minutes.
A Standard Aberdeen SSIS job was recoded in C# from 52 minutes down to sub 2 minutes.
The databases load at Cigna that took nearly 2 hours had the SQL re-worked and subsequently loaded in 8 minutes.
Working with hundreds of millions of rows
SQL Server, Oracle and Azure SQL Server
SSIS, Azure DataFactory v2 and IBM Turbo Integerator
SSAS & MDX, Azure Analysis Services (DAX) and TM1
SSRS and PowerBI