Azure Databricks: Code development environment for data processing and AI workflows.

Calls Azure OpenAI LLM model APIs for intelligent field mapping of sales data.

Uses LangChain framework for prompt engineering and chaining LLM calls.

MLflow: Experiment tracking, model evaluation, and registry.

Databricks Notebooks: Interactive development environment.

AI Outputs: Integration with Azure AI services.

Client's Azure OpenAI deployment:

• LLM Model: Deployed and hosted in Azure OpenAI.
• Model Serving: API endpoints for LLM.
• LangChain: Agentic platform/framework used within Databricks to interact with LLM APIs.
• Responsible AI: Ethical AI practices (if applicable).

Primary Master Data: Product, Customer, Currency

Primary Transaction Data: Sales Order

Secondary Master Data: Customer UOM, UI Channel Map, UI Product Map, Field Map

Secondary Transaction Data: Sales and Inventory

Azure Data Factory: Data ingestion and orchestration service.

Bronze (Landing): Raw data landing zone.

Unity Catalog: Centralized metadata and governance.

Data Storage Layers:

• Bronze: Raw data
• Silver: Cleansed and conformed data
• Gold: Curated business-level data

Azure Databricks: Code development environment for data processing and AI workflows.

Calls Azure OpenAI LLM model APIs for intelligent field mapping of sales data.

Uses LangChain framework for prompt engineering and chaining LLM calls.

MLflow: Experiment tracking, model evaluation, and registry.

Databricks Notebooks: Interactive development environment.

AI Outputs: Integration with Azure AI services.

Client's Azure OpenAI deployment:

• LLM Model: Deployed and hosted in Azure OpenAI.
• Model Serving: API endpoints for LLM.
• LangChain: Agentic platform/framework used within Databricks to interact with LLM APIs.
• Responsible AI: Ethical AI practices (if applicable).

Data Storage (Sim): Final data storage for simulation or consumption by end-user applications.

Governance Components: Monitoring & Orchestration, Data Quality & Observability, Data Lineage & Catalogue, Data Access Control, Dev-Data-ML Ops, Security