Skill set (10)
Daniel in a few words
Product value-driven Data Platform Engineer with 4.5 years of experience specializing in data platform engi-
neering, big data solutions and cloud technologies. Proven track record of building and maintaining large-scale
data processing systems, optimizing ETL pipelines, and driving process improvements. Adept at working with
cutting-edge technologies such as streaming ingestion frameworks, Hudi, Kafka, Kubernetes and AWS. Seeking the ability to deliver elegant data solutions to drive business growth.
Experience
- Funding CircleData Platform EngineerTECHDecember 2020 - Today (4 years)•Led a project to improve monitoring and alerting capabilities across AWS infrastructure using Cloud Watch, Grafana, JSONNET, and Prometheus.•Developed a decentralized streaming ingestion tool with Apache Hudi, reducing time to ingestion and promoting a data-mesh, self-serve architecture.•Contributed to Apache Hudi open-source community by fixing bugs, creating documentation, and provid ing learning opportunities for others in the community.•Pioneered the "Embedded Data Engineer" role, leading an end-to-end self-serve data pipeline for 3rd party vendor reconciliation.•Maintained active presence in the wider team by running a book/blog club to promote best practices and cross-team collaboration.
- Thales UKData EngineerSeptember 2018 - December 2020 (2 years and 3 months)•Maintained and developed backend processing scripts in Python and C# for large-scale data systems.•Managed Microsoft Azure environment and ETL pipeline, including Azure Function Apps and Batch Processing system.•Oversaw Clickhouse DB on Linux, focusing on scalability, redundancy, and eficient table design.•Developed front-end dashboarding design and SQL queries for Grafana, streamlining dashboard and ETL pipeline creation.•Engaged in commercial discussions to sell the platform as a service to London Underground.
Recommendations
Education
- BEng Mechanical Engineering (Hons.)Coventry UniversityBEng Mechanical Engineering (Hons.)