Web application with multiple live audio streams.

[Ed LaFave]

Currently our application:

Has a front end written in JavaScript.

Has a back end written in Java, a JAX-RS/REST service, and is deployed using TomEE.

We need to add the following audio streaming functionality:

Server to client. The server will periodically generate audio that must be sent to every client. This will be initiated by the server.

Client to server. A client can speak into a microphone and that audio must be sent to the server where a speech to text algorithm will operate on it.

Client to client. A client can speak into a microphone and that audio must be sent to another user (possibly using the server as a stepping stone).

In all three cases we'd like to optimize network traffic by not broadcasting dead air because we'll know exactly when audio starts/stops (clients will have a "push to talk" interface and the server will have a well defined audio file). I don't mind using a dedicated Media Server in addition to TomEE if that is necessary.

What libraries, frameworks, and technologies should I investigate both on the client and server side?

[travis Haycock]

Im doing something similar and am encountering similar problems as you. i've been reading the last few days about this and from my understanding the biggest obstacle you will face is getting a server. as you would have to put your server side on a server constantly running and listening for clients but may not be permitted due to hosting restrictions. I have been investigating a few free options or to a certain threshold at least... I believe something on amazon cloud ( will find the name for you later) offers something where you can sit your server-side on and has free options/ add-ons The other option of course which I'm considering is just buying a used server on Ebay. If you've already addressed the server issue, then ignore anything i've just said otherwise we can bounce ideas back and forth.

[Karthik Shiraly]

Have you investigated WebRTC + WebSockets? My knowledge in those area is little, but here are some suggestions that may help you research some more.
These may not be the best possible solutions, so please prototype them first. Also keep in mind that these are fairly new and sometimes incomplete standards and browser implementations, so lack of browser support is something to be aware of.

Server to client. The server will periodically generate audio that must be sent to every client. This will be initiated by the server

One way is to encode the PCM audio in base64, broadcast it from server through websockets and play them on browsers using Web Audio API

Client to server. A client can speak into a microphone and that audio must be sent to the server where a speech to text algorithm will operate on it.

The ideal way would be if server itself can also act as a WebRTC peer so you can stream audio from client to it over low latency connection. Such a server implementation seems to exist (see http://doc-kurento.readthedocs.org/en/stable/tutorials/java/tutorial-recorder.html).
An alternate way is to use MediaStreamRecorder to record audio, upload micro batches of audio data to server over regular HTTP, and stream that data to your speech recognition algorithm.
I'm assuming you already have such a recognition algorithm (if not, try CMU Sphinx).

Client to client. A client can speak into a microphone and that audio must be sent to another user (possibly using the server as a stepping stone).

This one is the easiest because it's exactly the peer-to-peer communication use case WebRTC is meant for. See this site for audio conf (and many other WebRTC) examples.

Another site with some good examples: https://github.com/webrtc/samples

[Ed LaFave]

Thank you for the replies!

This work is part of an effort to transition from a desktop application to a web application. The legacy desktop application used RabbitMQ to stream all of the audio. My first prototype is going to use WebSockets on the web client and RabbitMQ's Web Stomp Plugin (https://www.rabbitmq.com/web-stomp.html) on the web server. Hopefully, that'll maximize reuse of the legacy application. Thank you for the suggestions, I'll have to look into them in more detail if this prototype doesn't work.