Skip to content

Commit 12bf688

Browse files
committed
Added README for Save Voice Insights Azure Function
1 parent 9667326 commit 12bf688

File tree

2 files changed

+66
-1
lines changed

2 files changed

+66
-1
lines changed
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
# JobAssistAI Save Voice Insights Azure Function
2+
3+
## Overview
4+
5+
This Azure Function allows job coaches to store important insights they encounter during work through voice memo. These insights can be any random but useful information that might help future job coaches, especially when the original job coach is absent. In the UI, the job coach can upload a voice memo, which is saved in **Azure Blob storage**. This function is triggered as **blob trigger** when the audio file is uploaded. The audio file is then transcribed using **Azure AI Speech Services** and **Azure OpenAI's Whisper model**. The transcribed text is stored in a file and uploaded to blob storage for future references and the transcribed text is categorized as `category` and `details` using the Azure Function `jobassistai-http-trigger-openai` which returns category and details in JSON format. This is used as a payload to trigger the Azure Function `jobassistai-save-insights`, which generates a vector representation of the details using **Azure OpenAI's `text-embedding-ada-002` model** and stores the insights with id, category, details and details vector in **Azure Cosmos DB**.
6+
7+
## Features
8+
9+
- **Blob Triggered**: Invoked when audio file is uploaded to Azure Blob storage.
10+
- **Azure OpenAI's Whisper model**: Transcribes the audio file to text.
11+
- **Azure Blob Storage**: Stores audio files and transcribed text files
12+
- **Azure Function (jobassistai-http-trigger-openai)**: Calls Azure Open AI to extract the main theme as `category` and provide a concise but complete summary as `details`, ensuring no critical information is lost from the transcribed text.
13+
- **Azure Function (jobassistai-save-insights)**: Stores insights in Azure Cosmos DB for effecient retreival.
14+
```python
15+
insight = {
16+
"id": str(uuid.uuid4()), # Unique ID for the insight
17+
"category": category, # Category of the insight provided by Job Coach
18+
"details": details, # Details of the insight provided by Job Coach
19+
"detailsVector": details_vector # Vector representation of the details
20+
}
21+
```
22+
- **Audio-based Input & Text Output**: Accepts audio in `WAV`, `MP3`, `OGG` and `FLAC` format and returns response message in text format.
23+
- **Modular Design**: Organized into reusable modules for configuration, clients.
24+
25+
## Directory Structure
26+
```
27+
jobassistai-save-insights/
28+
├── function_app.py # Main entry point with the Azure Function
29+
├── config.py # Environment variables and configuration
30+
├── clients.py # Client initializations (OpenAI and Blob)
31+
├── requirements.txt # Python dependencies
32+
├── README.md # Project documentation (this file)
33+
└── host.json # Azure Functions configuration (auto-generated or customized)
34+
└── transcribe_whisper.py # Download/Upload to Blob and Transcribe functionalities
35+
```
36+
37+
## Prerequisites
38+
39+
- **Azure Subscription**: Access to Azure AI services.
40+
- **Azure OpenAI's Whisper model**: The model is deployed and accessible.
41+
- **Azure Blob Storage**: The database must be configured to store audio files and transcribed text files.
42+
- **Azure Functions (jobassistai-http-trigger-openai & jobassistai-save-insights)**: The functions are deployed and accessible. Ensure the endpoints are configured correctly in environment variables.
43+
- **Python 3.12**: Compatible with Azure Functions Python runtime.
44+
- **Azure Functions Core Tools**: For local testing and deployment.
45+
46+
## Usage
47+
48+
1. **Upload a voice memo**: Sends a blob trigger.
49+
2. **Processing**:
50+
- The function transcribes audio file to text.
51+
- Uploads the transcribed text file to Blob storage.
52+
- Calls `jobassistai-http-trigger-openai` function to categorize the transcribed text as `category` and `detail`
53+
- Calls `jobassistai-save-insights` function to save the insight in Azure Cosmos DB.
54+
3. **Output**: Generated response is returned to the caller in text format.
55+
56+
## Dependencies
57+
58+
See `requirements.txt`:
59+
- `azure-functions>=1.18.0`
60+
- `python-dotenv>=1.0.0`
61+
- `azure-storage-blob>=12.19.0`
62+
- `openai>=1.0.0`

azure-functions/jobassistai-save-voice-insights/requirements.txt

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,4 +2,7 @@
22
# The Python Worker is managed by Azure Functions platform
33
# Manually managing azure-functions-worker may cause unexpected issues
44

5-
azure-functions
5+
azure-functions>=1.18.0
6+
python-dotenv>=1.0.0
7+
azure-storage-blob>=12.19.0
8+
openai>=1.0.0

0 commit comments

Comments
 (0)