How To Become a Data Engineer | Beginners Guide To Data Engineering

Opensourced

In this episode I speak with Ethan Lyon. He is a marketer turned data engineer who created his own path into this career field. Ethan is an enthusiastic self educator who figured out a way to create his own valuable marketing tools with some basic data engineering knowledge.

Ethan starts off by talking about one of the tools he built for his team at Sere Interactive. In this role, Ethan was basically functioning as a product manager. Ethan was managing engineers and competing priorities for all the stakeholders in this project.

Ethan then talks about his first engineering product that he built on his own. After reading about a grad student studying machine learning that automated his online dating, Ethan decided to see if he could build his own online dating machine.

Ethan learned how to use cloud computing and data engineering to build his own data pipeline from one of the popular online dating apps to his own virtual machines running real time analysis of dating profiles.

Ethan then goes on to explain how this experience gave his the confidence to apply his newfound knowledge to his own at Sere Interactive. At Sere, Ethan first wanted to build a tool to automate keyword analysis for PPC campaigns he was running for clients. Basically, he built a tools that would scrub keywords his team was using against 3rd party databases for paid and organic search terms.

Ethan's team was able to implement the tool he built into their workflow to add another layer of analysis on top of the manual keyword analysis his team was doing. Beyond this type of immediate use for his Data Engineering skills, Ethan has many projects and technologies he is interested in pursuing in the future.

Finally, Ethan gives his perspective on the most valuable skills for a data engineer. Basically, Ethan says that you must learn python and SQL. After that Google Cloud platform is probably a good bet and maybe java depending on the tech stack and company you are trying to work with. Ethan also mentions that some data science teams use R for their analysis so it can be beneficial to know that.

If you are someone who is interested in learning more about data engineering then this is the interview for you. Finally, if you are interested in checking out Ethan's extensive write up he did on his path into data engineering you can check out his article "A Non Coder's Journey To Becoming A Data Engineer"