Sprachassistent in Echtzeit mit ElevenLabs und Node.js bauen

If you’ve ever wanted a real‑time voice assistant that sounds natural, you’ve probably tried the usual suspects—Google TTS, Amazon Polly, or even open‑source models. They’re decent, but they often sound robotic or lack the nuance you need for a polished user experience. That’s where ElevenLabs shines. Their neural text‑to‑speech engine produces expressive, human‑like speech and even supports voice cloning, which means you can give your assistant a unique personality with just a few minutes of audio.

In this post we’ll walk through a complete Node.js setup:

By the end you’ll have a working AI voice assistant that you can plug into a chat UI, a smart speaker, or any other interactive app.

👉 Get your free ElevenLabs API key here: https://try.elevenlabs.io/kr07zfuqn1bp

mkdir ai-voice-assistant cd ai-voice-assistant npm init -y npm install express dotenv Enter fullscreen mode Exit fullscreen mode Create a.env file to keep your secret safe:

ELEVENLABS_API_KEY=your_api_key_here Enter fullscreen mode Exit fullscreen mode Tip: Never commit.env to version control. Add it to.gitignore.

Create index.js and paste the following:

import express from ' express '; import fs from ' fs '; import { pipeline } from ' stream '; import { promisify } from ' util '; import dotenv from ' dotenv '; dotenv. config (); const app = express (); app. use ( express. json ()); const ELEVENLABS_API_KEY = process. env. ELEVENLABS_API_KEY; const ELEVENLABS_TTS_URL = ' https://api.elevenlabs.io/v1/text-to-speech '; // Helper to call ElevenLabs TTS async function synthesize ( text, voiceId = ' EXAVITQu4vr4xnSDxMaL ' ) { const response = await fetch ( ` ${ ELEVENLABS_TTS_URL } / ${ voiceId } `, { method: ' POST ', headers: { ' xi-api-key ': ELEVENLABS_API_KEY, ' Content-Type ': ' application/json ', Accept: ' audio/mpeg ', }, body: JSON. stringify ({ text, model_id: ' eleven_monolingual_v1 ', voice_settings: { stability: 0.75, similarity_boost: 0.75, }, }), }); if (! response. ok ) { const err = await response. text (); throw new Error ( `ElevenLabs error: ${ err } ` ); } return response. body; // This is a readable stream of the MP3 } // Route: POST /speak { "text": "Hello world!" } app. post ( ' /speak ', async ( req, res ) => { const { text, voiceId } = req. body; if (! text ) return res. status ( 400 ). json ({ error: ' Missing "text" in body. ' }); try { const audioStream = await synthesize ( text, voiceId ); res. setHeader ( ' Content-Type ', ' audio/mpeg ' ); // Stream the MP3 directly to the client const pipe = promisify ( pipeline ); await pipe ( audioStream, res ); } catch ( err ) { console. error ( err ); res. status ( 500 ). json ({ error: err. message }); } }); const PORT = process. env. PORT || 3000; app. listen ( PORT, () => console. log ( `🚀 Server listening on http://localhost: ${ PORT } ` )); Enter fullscreen mode Exit fullscreen mode How It Works /speak receives JSON with a text field (and optionally a voiceId ).

It calls synthesize(), which POSTs to ElevenLabs’ /text-to-speech/{voice_id} endpoint. The API returns an audio/mpeg stream that we pipe straight to the HTTP response, avoiding temporary file writes. You can now start the server:

node index.js Enter fullscreen mode Exit fullscreen mode Test it with curl:

curl -X POST http://localhost:3000/speak \ -H "Content-Type: application/json" \ -d '{"text":"Hey there, I’m your new AI assistant!"}' \ --output hello.mp3 Enter fullscreen mode Exit fullscreen mode Play hello.mp3 —you should hear a smooth, natural voice.

ElevenLabs lets you clone a voice by uploading a short recording (≈30 seconds). The workflow is:

async function createCustomVoice ( name, audioFilePath ) { const audioBuffer = await fs. promises. readFile ( audioFilePath ); const response = await fetch ( ' https://api.elevenlabs.io/v1/voices/add ', { method: ' POST ', headers: { ' xi-api-key ': ELEVENLABS_API_KEY, }, body: JSON. stringify ({ name, files: [ { filename: audioFilePath, data: audioBuffer. toString ( ' base64 ' ), }, ], }), }); if (! response. ok ) { const err = await response. text (); throw new Error ( `Voice creation failed: ${ err } ` ); } const data = await response. json (); return data. voice_id; // Store this for later TTS calls } // Example usage: ( async () => { const voiceId = await createCustomVoice ( ' MyBrandVoice ', './my-brand-sample.wav ' ); console. log ( ' Created voice ID: ', voiceId ); })(); Enter fullscreen mode Exit fullscreen mode Note: Voice cloning is a premium feature. If you’re just experimenting, stick with the default voices (e.g., EXAVITQu4vr4xnSDxMaL ).

A minimal HTML page can fetch the /speak endpoint and play the result using the <audio> element:

<!DOCTYPE html> <html lang= "en" > <head> <meta charset= "UTF-8" > <title> AI Voice Assistant Demo </title> </head> <body> <textarea id= "prompt" rows= "4" cols= "50" > What’s the weather like today? </textarea><br> <button id= "talk" > Speak </button> <audio id= "player" controls ></audio> <script> document. getElementById ( ' talk ' ). onclick = async () => { const text = document. getElementById ( ' prompt ' ). value; const res = await fetch ( ' /speak ', { method: ' POST ', headers: { ' Content-Type ': ' application/json ' }, body: JSON. stringify ({ text }) }); const blob = await res. blob (); const url = URL. createObjectURL ( blob ); const player = document. getElementById ( ' player ' ); player. src = url; player. play (); }; </script> </body> </html> Enter fullscreen mode Exit fullscreen mode Serve this static file from Express (or any web server) and you have a click‑to‑talk demo in under a minute.

You now have a fully functional AI voice assistant powered by ElevenLabs and Node.js. The core pieces are:

Because the TTS happens on the server, you keep your API key safe, and you can easily extend the service to handle multi‑modal interactions (e.g., speech‑to‑text with Whisper, intent parsing with LLMs, etc.).

Ready to give your app a human voice? Grab your ElevenLabs API key now and start building: https://try.elevenlabs.io/kr07zfuqn1bp