Streaming Server

What media encoding choose?

The Android platform supports many media formats.

Are there better choices in the context of the library? Here under are some hints, based on experience. Do not take them as references, but as an aid to make your own opinion.

The most important point is:

For 3GPP and MPEG-4 containers, the moov atom must precede any mdat atoms, but must succeed the ftyp atom.

The mdat atom is a section containing media frames and is the biggest in size. The moov atom is a section for metadata, i.e. properties, indexes, offsets and other structural information. For an encoder, the natural order is to write the mdat first, then the moov, after knowing what the offsets are. For a decoder, it's the opposite: the moov has to be fetched first to know how to interpret the mdat. Which one comes before the other is not that critical with a file on disk, thanks to a direct access. But it really matters for progressive streaming, and even more with contents encrypted with block-chaining algorithms where accessing any part supposes an entire reading from the very beginning.

An inadequate order does not prevent the play, it incurs a delay to the start, proportional to the file size. Some improvements over the native Android implementation have been made in the library (since version 1.3) in order to speed up this step, with a gain up to 5.6 times for AES/CBC and 8.2 times for RC4. But a delay remains: for a rough order of magnitude, count about 2.3 seconds per 10 megabytes for AES/CBC and 1 second per 10 megabytes for RC4 (on a Nexus 7 2012). In addition, in this case and for devices as of Android 4.3, special caution should be given to the selected Security Provider.

How to check the order?

• A basic text editor. Look for the two 4-characters patterns.
• MediaInfo software. For the GUI version, check Debug / Advanced mode ; For the CLI version, use the --full option or this specific option: --inform=General;%IsStreamable%. In both cases, check that the IsStreamable property is Yes.
• MP4Box software. Use the --info option and check the presence of File suitable for progressive download (moov before mdat). When the file is not suitable, there is no negative mention.

How to generate a correct order?

Otherwise, use another tool. Here are some examples:

• MP4Box software. Run it as: mp4box -inter 500 in.mp4 -out out.mp4

What security provider choose?

Android 4.3 introduced AndroidOpenSSL as the preferred default provider, before BouncyCastle ("BC"). Initially, this new implementation didn't work as expected when the cipher is used a second time. This is particularly true with the atom order issue detailed in Media Encoding.

The bug has been partially fixed in 5.0. There is still a problem with algorithms having a padding. It may help to have a fix if you star the issue.

• 4.3 and 4.4, in any case
• 4.3 and more, for padded algorithms without the use of a necessary CipherFactory (case of the ECB mode)

  Unfortunately, the BC provider is deprecated and can't no more be imposed as of Android P (an exception is thrown). So this fallback must be restricted to versions before 9.

See also below some code samples.

New in v3.2.5 In the above mentioned situations, an alternative to the BC provider is to implement a CipherFactory, even if it doesn't seem mandatory otherwise. But it becomes the only solution for padded algorithms as of Android P (anyway, padded algorithms are discouraged, see next section).

What transformation (algorithm/mode/padding) choose?

In case of no need for forward/rewind seeking

However, you can use a block algorithm and select a mode that does not require padding to effectively use a block cipher as a stream cipher.

• AES/CFB/NoPadding
• AES/CTR/NoPadding
• AES/CTS/NoPadding
• AES/CBC/WithCTS
• AES/OFB/NoPadding

In case you don't have any criteria or legacy/compatibility constraints, the RC4 stream cipher is a good candidate, it's a widely used stream cipher and it's less processing intensive than the block ciphers.

The problem of forward/rewind seeking

There is no obvious solution to this problem. However, among the various block cipher modes of operation, some allow a random read access during decryption.

It doesn't change anything for the repeated hits from the MediaPlayer but at least the response comes quicker.

• Choose a mode allowing a random read access. The library supports these modes:
  • CTR. For example: AES/CTR/NoPadding
  • CFB (v3.2+)
  • CBC (v3.2+)
• Instead of a single Cipher, give to the LocalSingleHttpServer an instance of your implementation of a CipherFactory (see a code sample below)

Additionnal information on other modes:

OFB: This mode does not allow a random read access.

ECB: Although suitable for random access, it does not hide data patterns well and is not recommended in cryptographic.
Nevertheless, the library supports (v3.2+) this mode as well, and there is no need for a CipherFactory.

Data Sources

Basic Local File

File file = new File(
  Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_MOVIES),
  "path/to/video.mp4");
String path = mServer.getURL(file.getPath());

Entry in a Zip File

String path = mServer.getURL("/path/to/container.zip", "path/to/video.mp4");

Entry in an APK Expansion Zip File

int mainVersion = 1;
int patchVersion = 0;
String path = mServer.getURL(mainVersion, patchVersion, "path/to/video.mp4");

Raw Asset File

String path = mServer.getURL("asset://hd/video.mp4");

SMB Server

String path = mServer.getURL("smb://server/share/path/to/video.mp4");

HTTP or HTTPS Server

String path = mServer.getURL("http://media.mydomain.com/path/to/video.mp4");

FTP Server

String path = mServer.getURL("ftp://ftp.mydomain.com/path/to/video.mp4");

Code Samples

Playing <video> in WebView

You can design your application in two basic ways (or even a mix of both):

• Host-driven: The host application is the master and controls the video source.
• JavaScript-driven: The HTML page has some embedded interaction widgets to allow the user to select a media.

val webview = WebView(this)
webview.settings.javaScriptEnabled = true
// mServer = ...
var path = "/some/path"
path = mServer.getURL(path)
webview.webViewClient = object : WebViewClient() {
  override fun onPageFinished(view: WebView, url: String) {
    view.loadUrl("javascript:setURL('$path'); void(0);")
  }
}
val page = """
<html><body>
 <script>
  function setURL(url) {
   document.getElementById('v_id').src = url; }
 </script>
 <video id="v_id" controls></video>
</body></html>
"""
webview.loadData(page, "text/html", null)

final WebView webview = new WebView(this);
webview.getSettings().setJavaScriptEnabled(true);
// mServer = ...
String path = '/some/path';
path = mServer.getURL(path);
final String mediaPath = path;  // just because 'final' is mandatory
webview.setWebViewClient(new WebViewClient() {
  @Override
  public void onPageFinished(WebView view, String url) {
    view.loadUrl("javascript:setURL('" + mediaPath + "'); void(0);");
  }
});
String page = "<html><body>" +
    "<script>" +
    "function setURL(url) {" +
    "  document.getElementById('v_id').src = url; }" +
    "</script>" +
    "<video id=\"v_id\" controls></video>" +
    "</body></html>";
webview.loadData(page, "text/html", null);

val webview = WebView(this)
webview.settings.javaScriptEnabled = true
// mServer = ...
webview.addJavascriptInterface(
    mServer.jsInterfaceObject(),
    "serverObject")
val page = """
<html><body>
 <script>
  function setURL(url) {
   document.getElementById('v_id').src = url; }
  function setPath(path) {
   setURL(serverObject.getURL(path)); }
  // ... logic for:
  // setPath('/some/path');
 </script>
 <video id="v_id" controls></video>
</body></html>
"""
webview.loadData(page, "text/html", null);

final WebView webview = new WebView(this);
webview.getSettings().setJavaScriptEnabled(true);
// mServer = ...
webview.addJavascriptInterface(
    mServer.getJsInterfaceObject(),
    "serverObject");
String page = "<html><body>" +
    "<script>" +
    "function setURL(url) {" +
    "  document.getElementById('v_id').src = url; }" +
    "function setPath(path) {" +
    "  setURL(serverObject.getURL(path)); }" +
    "// ... logic for:" +
    "// setPath('/some/path');" +
    "</script>" +
    "<video id=\"v_id\" controls></video>" +
    "</body></html>";
webview.loadData(page, "text/html", null);

Playing a basic encrypted video

A partial sample of how to play an encrypted video in a VideoView widget:

• Kotlin
• Java

private fun myPlay(path: String) {
    mServer = LocalSingleHttpServer()
    mServer!!.setCipher(cipher)
    mServer!!.start()
    path = mServer!!.getURL(path)
    mVideoView.setVideoPath(path)
    mVideoView.start()
}
override fun onCompletion(mp: MediaPlayer) {  // MediaPlayer.OnCompletionListener interface
    mServer!!.stop()
    mServer = null
}
private val cipher: Cipher?
    @Throws(GeneralSecurityException::class)
    get() {
        val algorithm = "AES"
        val transformation = "AES/CTR/NoPadding"
        val key = "1234567890123456"
        val iv = ByteArray(16)
        val provider = selectProvider(transformation)
        val c = if (provider != null) Cipher.getInstance(transformation, provider)
                else Cipher.getInstance(transformation)
        c.init(Cipher.DECRYPT_MODE, SecretKeySpec(key.toByteArray(), algorithm), IvParameterSpec(iv)))
        return c
   }
companion object {
    private fun selectProvider(transformation: String): String? {
        /* Avoid the default security provider "AndroidOpenSSL" in some cases between Android 4.3 and 9
        http://libeasy.alwaysdata.net/network/#provider 
        */
        val ucName = transformation.toUpperCase()
        return if (Build.VERSION_CODES.JELLY_BEAN_MR2 <= Build.VERSION.SDK_INT
                    && (Build.VERSION.SDK_INT < Build.VERSION_CODES.LOLLIPOP
                        || (ucName.endsWith("PADDING") && !ucName.endsWith("/NOPADDING")
                            && Build.VERSION.SDK_INT < Build.VERSION_CODES.P))) "BC" else null
    }
}

private void myPlay(String path) {
    mServer = new LocalSingleHttpServer();
    mServer.setCipher(myGetCipher());
    mServer.start();
    path = mServer.getURL(path);
    mVideoView.setVideoPath(path);
    mVideoView.start();
}
public void onCompletion(MediaPlayer mp) { // MediaPlayer.OnCompletionListener interface
    mServer.stop();
    mServer = null;
}
private Cipher myGetCipher() {
    final String algorithm = "AES";
    final String transformation = "AES/CTR/NoPadding";
    final String key = "1234567890123456";
    final byte[] iv = new byte[16];
    final String provider = selectProvider(transformation);
    final Cipher c = provider != null ?
            Cipher.getInstance(transformation, provider) :
            Cipher.getInstance(transformation);
    c.init(Cipher.DECRYPT_MODE,
            new SecretKeySpec(key.getBytes(), algorithm),
            new IvParameterSpec(iv);
    return c;
}
private static String selectProvider(final String transformation) {
    /* Avoid the default security provider "AndroidOpenSSL" in some cases between Android 4.3 and 9
    http://libeasy.alwaysdata.net/network/#provider 
    */
    final String ucName = transformation.toUpperCase();
    return (Build.VERSION_CODES.JELLY_BEAN_MR2 <= Build.VERSION.SDK_INT
            && (Build.VERSION.SDK_INT < Build.VERSION_CODES.LOLLIPOP
                || (ucName.endsWith("PADDING") && !ucName.endsWith("/NOPADDING")
                    && Build.VERSION.SDK_INT < Build.VERSION_CODES.P))) ?
            "BC" : null;
}

Playing an encrypted video, with tolerance to forward/rewind seeks

A partial sample of how to implement a CipherFactory:

• Kotlin
• Java

private fun myPlay(path: String) {
    mServer = LocalSingleHttpServer()
    mServer!!.setCipherFactory(MyCipherFactory())
    mServer!!.start()
    path = mServer!!.getURL(path)
    mVideoView.setVideoPath(path)
    mVideoView.start()
}
override fun onCompletion(mp: MediaPlayer) {  // MediaPlayer.OnCompletionListener interface
    mServer!!.stop()
    mServer = null
}

private void myPlay(String path) {
    mServer = new LocalSingleHttpServer();
    mServer.setCipherFactory(new MyCipherFactory());
    mServer.start();
    path = mServer.getURL(path);
    mVideoView.setVideoPath(path);
    mVideoView.start();
}
public void onCompletion(MediaPlayer mp) { // MediaPlayer.OnCompletionListener interface
    mServer.stop();
    mServer = null;
}

• Library v3 case - Kotlin
• Library v3 case - Java
• Library v2 case - Java
• Library v1 case - Java

private class MyCipherFactory : CipherFactory {
    @Throws(GeneralSecurityException::class)
    override fun getCipher(): Cipher {
        // you are free to choose your own Initialization Vector
        val initialIV = ByteArray(16)
        return rebaseCipher(initialIV)
    }
    @Throws(GeneralSecurityException::class)
    override fun rebaseCipher(iv: ByteArray): Cipher {
        val c = Cipher.getInstance("AES/CTR/NoPadding")
        c.init(Cipher.DECRYPT_MODE,
                SecretKeySpec("1234567890123456".toByteArray(), "AES"),
                IvParameterSpec(iv))
        return c
    }
}

private class MyCipherFactory implements CipherFactory {
  @Override
  public Cipher getCipher() throws GeneralSecurityException {
    // you are free to choose your own Initialization Vector
    byte[] initialIV = new byte[16];
    return rebaseCipher(initialIV);
  }
  @Override
  public Cipher rebaseCipher(byte[] iv) throws GeneralSecurityException {
    final Cipher c = Cipher.getInstance("AES/CTR/NoPadding");
    c.init(Cipher.DECRYPT_MODE,
      new SecretKeySpec("1234567890123456".getBytes(), "AES"),
      new IvParameterSpec(iv));
    return c;
  }
}

private class MyCipherFactory implements LocalSingleHttpServer.CipherFactory {
  // ... same as v3

private class MyCipherFactory implements LocalSingleHttpServer.CipherFactory {
  @Override
  public Cipher getCipher() throws IOException {
    // you are free to choose your own Initialization Vector
    byte[] initialIV = new byte[16];
    return rebaseCipher(initialIV);
  }
  @Override
  public Cipher rebaseCipher(byte[] iv) throws IOException {
    Cipher c = null;
    try {
      // avoid the default security provider "AndroidOpenSSL" in Android 4.3+ (http://libeasy.alwaysdata.net/network/#provider)
      c = Cipher.getInstance("AES/CTR/NoPadding", "BC");
      c.init(Cipher.DECRYPT_MODE,
        new SecretKeySpec("1234567890123456".getBytes(), "AES"),
        new IvParameterSpec(iv));
    } catch (GeneralSecurityException e) {
      throw new IOException("Unable to create a cipher", e);
    }
    return c;
  }
}

How to generate an encrypted file?

Java code

public static void encrypt() throws Exception {
    final byte[] buf = new byte[8192];
    final Cipher c = Cipher.getInstance("AES/CTR/NoPadding");
    c.init(Cipher.ENCRYPT_MODE, new SecretKeySpec("1234567890123456".getBytes(), "AES"), new IvParameterSpec(new byte[16]));
    final InputStream is = new FileInputStream("/path/to/src/video.mp4");
    final OutputStream os = new CipherOutputStream(new FileOutputStream("/path/to/dst/video.mp4"), c);
    while (true) {
        int n = is.read(buf);
        if (n == -1) break;
        os.write(buf, 0, n);
    }
    os.close(); is.close();
}

node.js code

var crypto = require('crypto');
var algorithm = 'aes-128-ctr';
var key = new Buffer('1234567890123456');
var iv = new Buffer('\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0');
var c = crypto.createCipheriv(algorithm, key, iv);

var fs = require('fs');
var r = fs.createReadStream('/path/to/src/video.mp4');
var w = fs.createWriteStream('/path/to/dst/video.mp4');
r.pipe(c).pipe(w);

Python3 code

Note: code checked with M2Crypto version 0.35.2.

from M2Crypto.RC4 import RC4

key = bytes('BrianIsInTheKitchen', 'utf-8')
cipher = RC4(key)

src_path = '/path/to/src/video.mp4'
dst_path = '/path/to/dst/video.mp4'
with open(src_path, 'rb') as input, open(dst_path, 'wb') as output:
    output.write(cipher.update(input.read()))  # no need of cipher.final() for RC4

Command Line Tools

openssl enc -aes-128-ctr -in /path/to/src/video.mp4 -out /path/to/dst/video.mp4 -K 000102030405060708090A0B0C0D0E0F -iv 0

FAQ

Do you propose a similar library for iOS or Windows Phone?

No. We are focused on Android only. A React Native module is also available (again, for Android build only).

Is it usable for anything other than VideoView?

Yes. The decrypted stream can be delivered to a <video> or <audio> tag in a WebView, or to a Chromecast device.

I see these Error lines in LogCat:
"D/MediaPlayer: setDataSource IOException happend :"
" java.io.FileNotFoundException: No content provider: http://127.0.0.1: ..."

Note that it is not a thrown exception but a Debug message from the MediaPlayer, detailed with a stacktrace.
It's a normal behavior of MediaPlayer: whatever the path content is, it first tries it as a local resource and if it fails it will fallback to a remote resource. You see that on the next Debug message:
"D/MediaPlayer: Couldn't open file on client side, trying server side"

I see these Error lines in LogCat:
"Could not find class 'jcifs.smb.SmbFile', referenced ..."
"Could not find class 'com.android.vending.expansion.zipfile.ZipResourceFile', referenced ..."
together with a bunch of Warning or Debug lines prefixed with "VFY:".

Do you intend to use any of these optional data sources: SMB server or APK Expansion Zip file?
If not, just ignore the error, it doesn't prevent the app to work.
Otherwise, you forgot to embed an additional library in your app, so expect the dedicated getURL() methods to return null.

It works fine on a Wifi network, but on a mobile network it fails with:
"Server responded with http status 503", or 502.

A possible explanation is the setting of a proxy on the network interface, which routes all traffic to outside, even for localhost requests. On the Wifi interface, there is a "Bypass proxy for" option where "127.0.0.1" can be entered. Unfortunately this is not the case for mobile interface: in the Edit screen of an Access Point, you can find a Proxy and Port but no bypass input field.

The solution is to unset the Proxy setting on the mobile network interface.