Hearing My Child's Smiles and Laughter with ffmpeg + Nginx + RTMP on RaspberryPI

Overview

This post summarizes the steps for building nginx with the rtmp module on a RaspberryPI
and delivering video + audio captured by a WebCamera over HLS.

Background

At first, I used mjpeg-streamer to monitor our pet rabbit while I was out.
After my child was born, I found myself casually checking in to see whether they were doing well.

However, while rabbits don’t make noise, children cry and scream.
With mjpeg-streamer I could see their facial expressions, but I couldn’t hear my child’s voice.

So while looking for a way to deliver video + audio at low load, I came across ffmpeg.*1

What I Bought

Raspberry Pi 3 Model B V1.2 (日本製) 国内正規代理店品

• 出版社/メーカー: Raspberry Pi
• 発売日: 2016/02/29
• メディア: Tools & Hardware
• この商品を含むブログを見る

Punasi Raspberry Pi?源 5V 3A 1.8m Raspberry Pi 3 2Model B/B+ Pi A/A+ Zero Micro USB Androidタブレット スマートフォン

• 出版社/メーカー: Punasi
• メディア: エレクトロニクス
• この商品を含むブログを見る

LOGICOOL HDウェブカム フルHD動画対応 C615

• 出版社/メーカー: ロジクール
• 発売日: 2011/09/22
• メディア: Personal Computers
• 購入: 2人 クリック: 2回
• この商品を含むブログを見る

Kinobo???USB 2.0ミニマイクマイク" Makio"ノートパソコン/デスクトップPC用???VoIP/Skype/音声認識ソフトウェア

• 出版社/メーカー: Kinobo
• メディア: Personal Computers
• この商品を含むブログを見る

Nginx

I downloaded the latest version available at the time from https://nginx.org/en/download.html.

Downloading the Various Components

I downloaded the following.
`nginx-http-auth-digest` was added as a module to enable Digest authentication in Nginx.

• Nginx
• nginx-rtmp-module
• openssl
• nginx-http-auth-digest
• ffmpeg

// Let's say we work in the home directory.
pi$ cd ~

// nginx
pi$ wget https://nginx.org/download/nginx-1.15.2.tar.gz

// nginx-rtmp-module
pi$ wget -O rtmp.zip https://github.com/arut/nginx-rtmp-module/archive/master.zip
pi$ wget -O ssl.zip https://github.com/openssl/openssl/archive/master.zip

// nginx-http-auth-digest
pi$ git clone https://github.com/samizdatco/nginx-http-auth-digest.git
pi$ cd nginx-http-auth-digest
pi$ git clone https://gist.github.com/frah/3921741
pi$ patch -u < 3921741/patch-ngx_http_auth_digest_module.diff

// ffmpeg
pi$ git clone git://source.ffmpeg.org/ffmpeg.git
pi$ wget ftp://ftp.alsa-project.org/pub/lib/alsa-lib-1.1.6.tar.bz2

Extracting

pi$ tar xvzf nginx-1.15.2.tar.gz
pi$ unzip rtmp.zip
pi$ unzip ssl.zip
pi$ tar xjvf alsa-lib-1.1.6.tar.bz2

Building Nginx

pi$ cd nginx-1.15.2/
pi$ sudo ./configure --with-http_ssl_module --with-http_realip_module --add-module=../nginx-rtmp-module-master --with-openssl=../openssl-master --add-module=../nginx-http-auth-digest
pi$ sudo make
pi$ sudo make install

Checking the Nginx Version

pi$ /usr/local/nginx/sbin/nginx -V

nginx version: nginx/1.15.2
built by gcc 4.9.2 (Raspbian 4.9.2-10+deb8u1)
built with OpenSSL 1.1.1-pre9-dev  xx XXX xxxx
TLS SNI support enabled
configure arguments: --with-http_ssl_module --with-http_realip_module --add-module=../nginx-rtmp-module-master --with-openssl=../openssl-master --add-module=../nginx-http-auth-digest

Making Nginx Accessible from a Path via a Symbolic Link

pi$ sudo ln -s /usr/local/nginx/sbin/nginx /usr/bin/nginx

pi$ which nginx

Building ffmpeg

pi$ sudo apt-get install libomxil-bellagio-dev

pi$ cd alsa-lib-1.1.6
pi$ ./configure --prefix=/home/pi/ffmpeg
pi$ sudo make
pi$ sudo make install

pi$ cd /home/pi/ffmpeg
pi$ sudo ./configure  --enable-gpl  --enable-nonfree --enable-mmal --enable-omx-rpi --enable-omx --extra-cflags="-I/home/pi/ffmpeg/include" --extra-ldflags="-L/home/pi/ffmpeg/lib" --extra-libs=-ldl
pi$ sudo make -j4
pi4 sudo make install

I got the following error when I had not run `sudo apt-get install libomxil-bellagio-dev`.

ERROR: OMX_Core.h not found

Trying to Record

Checking the List of Video Capture Devices

pi$ v4l2-ctl --list-device

HD Webcam C615 (usb-3f980000.usb-1.3):
        /dev/video0

When you get an error like the following while running the command above,

Failed to open /dev/video0: No such file or directory

try the following command.

pi$ sudo pkill /dev/video0

Checking the List of Audio Input Devices

In my case,
card 1 is the WebCam,
card 2 is the microphone.

pi$ arecord -l

**** ハードウェアデバイス CAPTURE のリスト ****
カード 1: C615 [HD Webcam C615], デバイス 0: USB Audio [USB Audio]
  サブデバイス: 1/1
  サブデバイス #0: subdevice #0
カード 2: Device [USB PnP Sound Device], デバイス 0: USB Audio [USB Audio]
  サブデバイス: 1/1
  サブデバイス #0: subdevice #0

Now, Recording

Since card 2 is the input device, I specified `hw:2`.

pi$ ffmpeg -f alsa -ac 1 -i hw:2 -f v4l2 -s 640x480 -i /dev/video0 output.mpg

Download the generated output.mpg file to your Mac and try playing it back.

If it plays, you've confirmed that ffmpeg is working without any problems.

Next is the configuration for delivery.

Nginx Configuration

Creating a Directory for Configuration Files

pi$ sudo mkdir -p /usr/local/nginx/conf.d

Creating a Directory for HLS File Generation

pi$ sudo mkdir -p /var/www/html/live/hls

index.html for HLS Delivery

• Fetching hls.min.js

pi$ cd /var/www/html
pi$ wget https://cdn.jsdelivr.net/hls.js/latest/hls.min.js

• /var/www/html/index.html

<!DOCTYPE html>
<html lang="ja">
<head>
  <meta charset="utf-8"/>
  <script src="./hls.min.js"></script>
</head>

<body>
  <video id="video"></video>
  <script>
    if(Hls.isSupported()) {
      var video = document.getElementById('video');
      var hls = new Hls();
      hls.loadSource('/live/hls/stream.m3u8');
      hls.attachMedia(video);
      hls.on(Hls.Events.MANIFEST_PARSED,function() {
      video.play();
    });
   }
  </script>
</body>
</html>

Digest Authentication Configuration

pi$ cd /var/www
pi$ sudo htdigest -c .htdigest 'digest AuthNginx' hoge
password: <enter password>

Placing the Various Configuration Files

• /usr/local/nginx/conf.d/default.conf

server {
    listen 8090;
    proxy_set_header   X-Forwarded-For     $proxy_add_x_forwarded_for;
    access_log /var/log/nginx/access.log combined;
    error_log /var/log/nginx/error.log warn;

    location = /favicon.ico {
        access_log off;
        empty_gif;
        expires 30d;
    }

    location / {
        auth_digest "digest AuthNginx";
        auth_digest_user_file /var/www/.htdigest;

        root /var/www/html;
        index index.html;
        set_real_ip_from    127.0.0.1;
        real_ip_header      X-Forwarded-For;
    }
}

• /usr/local/nginx/conf.d/rtmp

rtmp {
    server {
        listen 1935;
        chunk_size 4096;
        allow play all;
        access_log /var/log/nginx/rtmp_access.log;

        application live {
            live on;
            hls on;
            record off;
            hls_path /var/www/html/live/hls;
            hls_fragment 1s;
            hls_type live;
        }
    }
}

• /usr/local/nginx/conf/nginx.conf

user  www-data;
worker_processes  1;

pid        /var/run/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include           mime.types;
    default_type      application/octet-stream;
    sendfile          on;
    keepalive_timeout 65;
    include /usr/local/nginx/conf.d/*.conf;
}

include /usr/local/nginx/conf.d/rtmp;

Nginx Startup Configuration File

• /lib/systemd/system/nginx.service

[Unit]
Description=The NGINX HTTP and reverse proxy server
After=syslog.target network.target remote-fs.target nss-lookup.target

[Service]
Type=forking
PIDFile=/var/run/nginx.pid
ExecStartPre=/usr/local/nginx/sbin/nginx -t
ExecStart=/usr/local/nginx/sbin/nginx
ExecReload=/usr/local/nginx/sbin/nginx -s reload
ExecStop=/bin/kill -s QUIT $MAINPID
PrivateTmp=true

[Install]
WantedBy=multi-user.target

Starting Nginx

pi$ sudo systemctl daemon-reload
pi$ sudo systemctl start nginx
pi$ sudo systemctl status nginx

At this point the HLS files have not been generated yet,
so even if you access `https://<RaspberryPI IP>:8090`, nothing is being delivered over HLS.

By starting ffmpeg, `stream.m3u8` is generated under the `/var/www/html/live/hls/` directory.

Starting ffmpeg

pi$ sudo ffmpeg \
-f alsa -ac 1 -thread_queue_size 8192 -i hw:2 \
-f v4l2 -thread_queue_size 8192 -input_format yuyv422 -video_size 432x240 -framerate 30 -i /dev/video0 \
-c:v h264_omx -b:v 768k -bufsize 768k -vsync 1 -g 16  \
-c:a aac -b:a 128k -ar 44100 \
-af "volume=30dB" \
-f flv rtmp://localhost/live/stream;

Trying to Access It

Let's access `https://<RaspberryPI IP>:8090`. You'll be prompted for Digest authentication, so enter the ID/PW you configured.

I confirmed that it's being delivered over HLS!

As for the load, it stays at around CPU 25 - 30%.

Summary

I was able to build HLS delivery on a RaspberryPI with ffmpeg + Nginx + RTMP.

In actual operation, it isn't always running; I set it up so it only starts when I want to watch, keeping the load as low as possible. For monitoring, Mackerel's free plan is sufficient for now.

I'd be glad if this serves as a useful reference for parenting hacks.

*1:Of course, this was set up with my family's consent.