Home
Hackathon
Apps
APIs
Contact Us
About
Sign In
Sign Up
WhatsApp Voice Transcriber
APIs used:
WhatsApp Cloud API
Whisper API
Github
Accurate transcriptions in seconds
This tutorial outlines the procedure for creating a real-time voice transcription bot capable of transcribing and translating incoming WhatsApp voice messages in various [supported languages](https://platform.openai.com/docs/guides/speech-to-text/supported-languages "supported languages") into English. This bot will use WhatsApp API for sending and receiving messages, and OpenAI's Whisper API for transcription. ## Create developer accounts #### Set up WhatsApp developer account Follow the [getting started guides](https://developers.facebook.com/docs/whatsapp/cloud-api/get-started "getting started guides") to set up a Meta developer account for WhatsApp. Retrieve your Access token from the Meta for Developers site `App Dashboard > WhatsApp > Getting Started` #### Set up OpenAI developer account Create an account with [OpenAI](https://platform.openai.com/signup "OpenAI") and generate an Access token to use their API. ## Create a new Node.js project Open your preferred IDE like Visual Studio Code to create a new Node.js project. Within your IDE, open the integrated terminal or command prompt and navigate to the directory where you want to create the project. Run the command`npm init --y`. This command will automatically generate a `package.json` file containing essential project information. ## Add and Install the SDKs to the project Add and install the necessary SDKs, namely the WhatsApp API SDK and the OpenAI API SDK, into your Node.js project. The SDKs will provide the tools and functionalities required for interacting with the WhatsApp and OpenAI APIs. Follow these steps for each SDK: #### Install the WhatsApp API SDK 1. Go to the [WhatsApp developer portal](https://www.apimatic.io/api-docs/whatsapp "WhatsApp developer portal"). 2. Download the TypeScript SDK. ![](https://apimatic-appstore.azurewebsites.net/Content/Upload/UCDescription/0d57794b-8085-4ee1-b3cd-58c81fcd6416_3_73f9283e-a0ae-4b49-84c1-39b4bdf7199f.png) 3. Unzip the SDK once the download is complete. 4. Copy the contents of the unzipped SDK folder into the root directory of your newly created Node.js project. 5. Open your terminal or command prompt and navigate to the SDK folder within your project. 6. Run the command `npm install` to install the necessary dependencies. #### Install the OpenAI API SDK Similarly, download the TypeScript SDK from the [OpenAI developer portal](https://www.apimatic.io/api-docs/openai "OpenAI developer portal") and follow the installation steps as you did for the WhatsApp SDK. Once you have installed both the SDKs successfully, run the command `npm install` to add the SDKs as dependencies to your project. ## Initialize the client Create instances of the SDKs to call the methods of the WhatsApp API and OpenAI API and provide the necessary authentication details. #### WhatsApp API Initialize the Whatsapp API client by providing the Access token. const client = new Client({ timeout: 0, accessToken: process.env.WHATSAPP_TOKEN, }); #### OpenAI API Similarly, instantiate an instance of the OpenAI API client by providing the Access token. const client = new Client({ timeout: 0, accessToken: process.env.OPENAI_API_KEY }); ## Create controller for WhatsApp webhook Create a controller in `index.js` file with 2 endpoints - `GET /webhook` and `POST /webhook`. The GET endpoint will be used to set up and verify the webhook while the POST endpoint will be used to receive webhook notifications from Whatsapp when a message is received. Refer to the [webhooks](https://developers.facebook.com/docs/whatsapp/cloud-api/webhooks/components "webhooks") documentation for information regarding the Notification Object payload. app.get("/webhook", (req, res) => { try { if ( req.query['hub.mode'] == 'subscribe' && req.query['hub.verify_token'] == token ) { res.send(req.query['hub.challenge']); } else { res.sendStatus(400); } } catch (err) { //Handle error } }); ------------ app.post("/webhook", async (req, res) => { // Add code to handle the incoming request }); ## Configure WhatsApp webhook Expose a publicly accessible URL to receive incoming messages from WhatsApp. You can use services like [ngrok](https://ngrok.com/ "ngrok") to create a tunnel to your local server so this webhook can be registered with Whatsapp. ## Create WhatsApp audio handler Use the `retrieveMediaURL` endpoint from the WhatsApp SDK to fetch the URL of the audio file and then make a request to it for downloading the audio file. Once the audio is downloaded, you can use the OpenAI SDK's `createTranscription` endpoint to obtain the text representation of the audio content. const downloadAudio = async (mediaID) => { try { const { result } = await mediaController.retrieveMediaURL(mediaID); const audioURL = result.url; //Add code to fetch the audio binary data using the retrieved URL } catch (error) { //Handle error } } const createTranscription = async audioFilePath => { //Add the necessary parameters for the audio transcription endpoint. try { const { result } = await openAIController.createTranscription(file,model,prompt,responseFormat,temperature,language); return result.text; } catch (error) { //Handle error } }; Call both the `downloadAudio`and `createTranscription` methods in the Transcription function. ```javascript async function Transcription(audioID) { const downloadMedia = await downloadAudio(audioID); // Add code to create a temporary directory try { // Add code to write audio file to the directory and to convert it from OGG to MP3 format const text = await createTranscription(mp3AudioPath); return text; } catch (error) { throw error; } finally { // Add code to remove the temporary directory } } ``` ## Call audio handler in post webhook controller To send the transcription as a message, we will use the `sendMessage` endpoint from the WhatsApp SDK and include appropriate error handling. const sendMessage = async (from, text) => { const phoneNumberID = process.env.WHATSAPP_PHONE_NUMBER_ID; const body = { messagingProduct: 'whatsapp', to: from, type: "text", text: { body: text } }; try { await messagesController.sendMessage(phoneNumberID, body); } catch (error) { //Handle error } } ` Lastly, call both the `sendMessage` and `Trascripiton` methods in the post webhook handler in `index.js`. async function handleAudioMessage(message) { try { //Add code to ignore the non audio messages const transcriptedMessage = await Transcription(audioID); //Add code to handle audio messages with no sound await sendMessage(from, transcriptedMessage); } catch (error) { console.log(error) } }
Post Comment
Comments
No comments yet.
4
downloads
1
likes
693
views
Categories
Media
Developer
Mehnoor Siddiqui
Languages
No comments yet.