The modern data stack is quickly becoming the dominant, trending tech stack in the data engineering field, Murray articulated. "Great data product managers will have an understanding of how to build a reliable and scalable data product, but also apply product thinking to drive the vision, roadmap, and adoption," Murray affirmed. If one is growing data engineering skills but finds they are more compelled by talking to end users, articulating the problems to be solved, and distilling the vision and roadmap for the team, then a product management role may be a future prospect.ĭata teams are beginning to invest in this skillset as we move to treat " data as a product," ranging from critical dashboards and decision-support tools to applications of machine learning that are critical to business operations or customer experience. "These types of roles typically give a broader, but shallower, understanding of each business use case, but may be an easier jump from a software engineering role into data."Īnother path I'm seeing more often for data engineers is the data product manager role, said Murray. "Alternatively, one might specialize in a specific capability of the data platform, such as reliability engineering, business intelligence, experimentation, or feature engineering." Murray specified. In this role, you should aim to gain an understanding of the end-to-end problem-from source to the analytical use case-as it'll make you an asset to the team and the business." Murray further explained, "Here, you can specialize in a specific domain of data that is central to the business operations, such as customer data or product / behavioral data. I've found that data platform teams, which are now quite common in data teams of various sizes, are great places for data engineers to cut their teeth. "While the underlying infrastructure may change and automation will shift time and attention to the right or left, human data engineers will continue to play a crucial role in extracting value from data, whether architecting scalable and reliable data systems or as specialist engineers within a chosen domain of data." "It is emitted, it is transformed for a use, and then it is archived," Murray pointed out. However, what hasn’t changed is the data lifecycle. It’s clear that the process of building and maintaining data pipelines will become much easier, as will the ability for data consumers to access and manipulate data. "Little in this space has escaped reinvention," Murray remarked. If you don’t like change, data engineering is not for you. In either case, it’s not that you were there, but what impact did you make? That could be in their primary occupation or by contributing to open-source projects. "When I evaluate data engineering candidates for a role, I’m looking for their track record of making an impact and hitting the ground running," Murray mentioned. "This would radically simplify the self-service analytics process and further democratize data, but it will be difficult to solve beyond basic "metric fetching," given the complexity of data pipelines for more advanced analytics," commented Monte Carlo CTO Shane Murray. Generative AI has also kick-started a gold rush with dozens of very early start-up companies racing to develop an AI that can query the data warehouse and return an intelligent answer to the ad hoc questions data consumers ask in their natural language. I’ve seen Andrej Karpathy joke on Twitter, “The hottest new programming language is English.” One of the biggest impacts has been the wider adoption of “prompt engineering,” essentially the skill of prompting AI to assist in coding-related tasks.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |