Streaming sound from Windows to Snowmix

buzzzy
2014-06-20
2014-07-04
  • buzzzy
    buzzzy
    2014-06-20

    Hello,

    we'd like to create a functionality in our project which let's to stream an audio feed from microphone on Windows to remote Linux server with Snowmix running and feeding that sound to it. What would be the best approach to do it?

    We think, that we could capture and stream (probably using RTP or RTSP) sound from mic with vlc and somehow capture it and feed it to Snowmix with gstreamer on Linux. What do you think, is it a good approach?

    Thank you,
    Buzzzy

     
    • Hi Buzzzy.

      I would assume both GStreamer and ffmpeg can do what you want with respect to streaming audio captured from microphone.

      In this thread, another is grabbing frames and audio under Windows to feed Snowmix:

      The command he uses is this:

      ffmpeg -re -i tcp://my.snowmix.ip:5010 -r 15 -c:v libx264 -crf:v 22 -maxrate 400k -bufsize 1835k -g 50 -preset veryfast -c:a libfaac -ar 22050 -crf:a 8 -f flv "rtmp://my.server.ip/application/streamname' flashver=FME/2.5\20(compatible;\20FMSc\201.0)'"
      

      Let me know how it goes and what works so others can benefit.

      Best regards
      Peter MM

       
      Last edit: Peter Maersk-Moller 2014-06-23
  • buzzzy
    buzzzy
    2014-06-27

    Hi Peter,

    I managed to stream sound from Windows machine with the following command:

    ffmpeg -f dshow -i audio="Microphone (Realtek High Defini" -acodec libmp3lame -f rtp rtp://192.168.1.77:1111
    

    I can receive the stream on Linux server with the ffplay command. Now, the question is how to use this sound as input stream via Gstreamer. Currently we are using the following script:

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    #!/bin/bash
    
    IP=127.0.0.1
    WIDTH=1024
    HEIGHT=576
    FRAMERATE=24
    
    if [ $# -lt 4 ] ; then
         echo parameter error. Usage $0 port source audio_feed_id feed_pipe [width] [height] [framerate]
         exit 1
    fi
    
    PORT=$1
    SRC=$2
    AUDIO_FEED_ID=$3
    FEED_PIPE=$4
    
    if [ $# -ge 7 ] ; then
      WIDTH=$5
      HEIGHT=$6
      FRAMERATE=$7
    fi
    
    echo $WIDTH x $HEIGHT
    
    rm $FEED_PIPE 2&>1 /dev/null
    
    (
    echo "audio feed ctr isaudio $AUDIO_FEED_ID";\
    gst-launch-1.0 -q filesrc location=$SRC ! decodebin name=decoder ! \
     videorate ! videoscale ! \
     videoconvert ! \
     video/x-raw, format=BGRA, pixel-aspect-ratio=1/1, interlace-mode=progressive, width=$WIDTH, height=$HEIGHT, framerate=$FRAMERATE/1 ! \
     shmsink socket-path=$FEED_PIPE shm-size=40000000 wait-for-connection=0         decoder.   ! queue ! audioconvert ! audioresample ! \
     audio/x-raw, format=S16LE, rate=48000, channels=2 ! fdsink fd=1 sync=true \
    ) | nc $IP $PORT
    
    exit 0
    

    What we want here to do is to start another audio input stream (the one from Windows) working in parallel with the script above. After that we want Snowmix to use Windows audio (mute the sounds from the video that is being shown). Finally, after Windows audio stream is finished, Snowmix would again start to use the sound of original video (script above). Is it possible to do?

    Thank You,
    B

     
    • Hi Buzzzy.

      There are several ways to do what you want. If your machines are network wise close and bandwidth sufficient, you could find a way to send raw audio to Snowmix. One way is to send raw audio S16E from the window machine prepended with the string 'audio feed ctr isaudio 1' plus a newline using TCP or you could have a helping process on the Linux/Snowmix machine like this:

          (echo 'audio feed ctr isaudio 1'
           nc -l 192.168.1.77 1111 ) | nc 127.0.0.1 9999
      

      receiving raw S16LE audio from the windows machine or you can have a pipeline like this using the mp3 over RTP over UDP from the windowsmachine:

          gst-launch-1.0 -v -e \
              udpsrc host=192.168.1.77 port=1111 \
              caps='application/x-rtp,media=audio, clock-rate=90000, payload=14' !\
              rtpmpadepay !\
              queue !\
              mpegaudioparse !\
              mad !\
              queue !\
              audioconvert !\
              autoaudiosink
      

      If this pipeline works or works when you have modified it, then you can use it for Snowmix like this

          (
           echo 'audio feed ctr isaudio 1'
           gst-launch-1.0 -v -e \
              udpsrc host=192.168.1.77 port=1111 \
              caps='application/x-rtp,media=audio, clock-rate=90000, payload=14' !\
              rtpmpadepay !\
              queue !\
              mpegaudioparse !\
              mad !\
              queue !\
              audioconvert !\
              'audio/x-raw,format=S16LE,channels=1,rate=44100,layout=interleaved' !\
              fdsink fd=3 3>&1 1>&2
          ) | nc 127.0.0.1 9999
      

      That was about howto connect an RTP MPEG audio stream to Snowmix. Now if you want to automate the things you write in your message, then you can have an external script connecting to Snowmix constantly querying it for the state of audio feeds and look at the RMS value for the channel and then change the volume for either audio feed depending on what you want to do exactly, which is not completely clear.

      In the next version of Snowmix, you can query values of RMS and state of audio feeds in the embedded Tcl interpreter and use the queryed values to do things automatically.

      Let me know how it goes.

      Best regards
      Peter

       
  • buzzzy
    buzzzy
    2014-06-30

    Hi, Peter,

    I appreciate your help very much. Great support. Your script for feeding mp3 sound to snowmix works perfectly, except I had to remove parameter "host=192.168.1.77" completely because the pipeline won't work.
    Now, I want to have 2 audio feeds in Snowmix and be able to switch between them. I have written them out in my .ini file, but the problem is that I don't know how to switch between them in the audio mixer:

    ...
    audio mixer source feed 1 5
    audio mixer source feed 1 4
    ...
    

    In this example (from my .ini that is attached) only feed 5 works. How do I switch to feed 4? What command should I use?

    Thank You,
    B

     
    Attachments
    • Hi Buzzzy

      Not sure what you problem is, however I can see that audio source 2 for the audio mixer 1 is muted.

      In your mixer definition you have written

      audio mixer source feed 1 5
      audio mixer source feed 1 4
      

      in that order. This means that 'audio feed 5 becomes source 1 of audio mixer 1 and 'audio feed 5' becomes source 2 of the same mixer. To unmute the sources, you need to execute both the following lines and not just the one you have

      audio mixer source mute off 1 1
      audio mixer source mute off 1 2
      

      Try also execute the snowcub utility and go to the audio tab and check your audio elements

      cd Snowmix-0.4.3/tcl
      ./snowcub.tcl 127.0.0.1 15000
      

      Best regards
      Peter MM

       
  • buzzzy
    buzzzy
    2014-06-30

    Hi, Peter,

    thank you again! That's what I was looking for. My problem was that tried to unmute 4th and 5th source although they don't exist.

    Thank you again,
    B

     
    Last edit: buzzzy 2014-06-30
  • buzzzy
    buzzzy
    2014-07-01

    Hello again,

    I was wondering if you could help me here again. The problem is that sound coming from windows microphone to snowmix is somehow distorted with a lot of glitches. I've tried the following script:

        gst-launch-1.0 -v -e \
        udpsrc port=1111 \
        caps='application/x-rtp,media=audio, clock-rate=90000, payload=14' !\
        rtpmpadepay !\
        queue !\ filesink location=mp3dump.mp3
    

    and everything fine here. There are no glitches in mp3dump.mp3 file. So I'm guessing there is something wrong with converting mp3 to raw S16LE format or sending it to snowmix. I've tried changing rates, formats, channels, but no luck. We're using following script for the videos and the sound is awesome:

    (
    echo "audio feed ctr isaudio $AUDIO_FEED_ID";\
    gst-launch-1.0 -q filesrc location=$SRC ! decodebin name=decoder ! \
     videorate ! videoscale ! \
     videoconvert ! \
     video/x-raw, format=BGRA, pixel-aspect-ratio=1/1, interlace-mode=progressive, width=$WIDTH, height=$HEIGHT, framerate=$FRAMERATE/1 ! \
     shmsink socket-path=$FEED_PIPE shm-size=40000000 wait-for-connection=0         decoder.   ! queue ! audioconvert ! audioresample ! \
     audio/x-raw, format=S16LE, rate=48000, channels=2 ! fdsink fd=1 sync=true \
    ) | nc $IP $PORT
    

    Script that we are using and which produces distorted sound:

    (
     echo "audio feed ctr isaudio $FEED_ID";\
     gst-launch-1.0 -v -e \
        udpsrc port=$PORT_SRC \
        caps='application/x-rtp,media=audio, clock-rate=90000, payload=14' !\
        rtpmpadepay !\
        queue !\
        mpegaudioparse !\
        mad !\
        queue !\
        audioconvert !\
        audioresample !\
        audio/x-raw,format=S16LE,channels=2,rate=48000 !\
        fdsink fd=3 3>&1 1>&2
    ) | nc $IP $PORT
    

    Thank you,
    B

     
    • Hi B.

      If you play the dumpmp3 file with VLC, try look at the codec information and check that it says it is 48000kHz Stereo.

      Try also adding 'do-timestamp=true' to last script for the udpsrc in the audio input script.

      Try also adding 'sync=true' to the fdsink in the same script.

      Please post your script running on the Windows machine, your ini file for Snowmix, if it is changed from your previous post, and the full scripts you are using for feeding audio and video into Snowmix. Then I'll try it out when I find a Windows box.

      Best regards

      Peter

       
  • buzzzy
    buzzzy
    2014-07-04

    Hi, Peter,

    I've updated my input script as you've suggested and made ffmpeg to stream audio that is at 48000kHz and everything works fine now! Below are the scripts that work for me:

    My ffmpeg script on Windows machine:

    ffmpeg.exe -f dshow -i audio="Microphone (Realtek High Defini" -acodec libmp3lame -ar 48000 -f rtp rtp://192.168.1.77:1111
    

    My input script on Linux machine:

    (
     echo 'audio feed ctr isaudio 5'
     gst-launch-1.0 -v -e \
        udpsrc port=1111 do-timestamp=true \
        caps='application/x-rtp,media=audio, clock-rate=90000, payload=14' !\
        rtpmpadepay !\
        queue !\
        mpegaudioparse !\
        mad !\
        queue !\
        audioconvert !\
        'audio/x-raw,format=S16LE,channels=2,rate=48000,layout=interleaved' !\
        fdsink fd=3 3>&1 1>&2 sync=true
    ) | nc 127.0.0.1 15000
    

    Thanks again!
    B

     


Anonymous


Cancel   Add attachments