1. My Data Story So Far
In January 2020, I start a new role as a Data Engineer. This is a very big step for me as I will need to learn a lot about programming in Python, the Agile approach to developing software and building my knowledge and understanding of Data Science (the wider department I will sit in).
In Data Science Journey, I want to take you through my experience in learning this new role. I want to document the challenges I face, the tips, tricks and knowledge that I discover, and the path of a beginner Data Engineer.
My Background
Bath Cathedral - Where I graduated from the University of Bath
Between 2013 and 2017, I studied Economics at the University of Bath. Upon graduating, I obtained a graduate position as a Data Analyst for Capital One in Nottingham.
My role as Data Analyst heavily relied on using SQL to store and retrieve data, in order produce reports and controls for the various operations departments within the company. A few of the areas I worked with closely were:
Quality Assurance
Payment Protection Insurance
Customer Experience
Vulnerable Customer Team
Over the last 2 years working as a Data Analyst, I gradually picked up learning Unix, Git and Github to help with managing version history of our teams' reports and controls. Often it would be hard to understand the evolution of the code for a report, so when I discovered Git and Github I was enamored with how clever the tool / language was for version control.
Gradually as I became more experienced with my day job, it was clear to me there were limitations to only knowing SQL. It is a language specifically designed for querying databases. If I wanted to start automating tasks, using variables, recursion, connecting to other systems like Excel, the OS (filesystem), and so on, then I needed to learn Python.
And so for the last year, I started to read tutorials on Python. Whenever a task came up at work, I would take longer to complete it so that I could learn how to do it automated in Python, instead of doing a bit of SQL along with some manual processes and manipulation by mself. One course that I eventually stumbled upon that really helped my Python ability, particularly with respect to learning pandas, was the Applied Data Science with Python from the University of Michigan on Coursera.
The New Role
The new role is a Data Engineering position, that sits within the wider department of Data Science and AI. Having read around a few of the definitions of what a Data Engineer is online, I think the one most apt is:
The data engineer is someone who develops, constructs, tests and maintains architectures, such as databases and large-scale processing systems.
The team that I sit in will be responsible for building out a software platform to enable analysts in the company to perform valuations analysis of our customers. So when a customer wants to apply for a credit card, or perhaps obtain a credit limit increase, the relevant analysts will use the software platform to perform and monitor the valuations analysis required.
Data Science Journey
And so the Data Science Journey begins. My intention is to make this an ongoing series of blogs all under the banner of "Data Science Journey". Let's see how it goes!