Design and implement data integration, acquisition, cleansing, harmonization, and transformation processes to create curated high-quality datasets for data science, data discovery for the usage of vector stores/embeddings.
Develop and maintain scalable data processing pipelines and systems to support Generative AI applications
Monitor and optimize the performance and manage the costs of data processing pipelines and systems
Monitor technology trends and advancements in Generative AI and incorporate them to continuously innovate
Collaborate with Solution Architects, Data Scientists, Software Engineers and DevOps Engineers, Product Owners, researchers and business stakeholders on the cross-functional team and across teams to understand business needs, derive technical requirements and ensure data availability, quality and responsible data use adhering to security, privacy and compliance requirements
Continuously improve data acquisition, preparation, transformation, and publishing processes to meet business needs
