Multimedia

Cloud Phone is designed for rich multimedia experiences, thanks to powerful server-side rendering. Many popular apps like YouTube and TikTok are available today.

Images

Cloud Phone processes HTML, CSS, and JavaScript into standard Puffin Vector Graphics Language (PVGL) to overcome limitations in native device rendering. For more refresher, learn about Cloud Phone architecture.

Supported Image Formats

• Bitmap (BMP)
• Portable Network Graphics (PNG)
• Joint Photographic Experts Group (JPEG)
• Scalable Vector Graphics (SVG)
• Graphics Interchange Format (GIF)
• AV1 Image File Format (AVIF)
• WebP

Responsive Images

Cloud Phone automatically downsamples (resizes & compresses) images to fit small screens and limited device random access memory (RAM), often as low as 16MB! Cloud Phone supports responsive images, including both the srcset attribute and <picture> element. Responsive images can save bandwidth, reduce cost, improve page load times, and ensure images are displayed correctly.

Unsupported Image Formats

Cloud Phone does not support image formats that are not packaged with Chromium by default. Unsupported formats include JPEG XL, High Efficiency Image File Format (HEIF/HEIC), Tag Image File Format (TIFF), Truevision Graphics Adapter (TGA), Silicon Graphics Image (SGI), PhotoShop Document (PSD), and other proprietary or RAW image formats.

Audio

Cloud Phone supports streaming remote audio over HTTPS, and local audio via Blob URLs.

Supported Audio Formats

Container Formats

• MP4
• Ogg
• WebM
• WAV

Codec Formats

• FLAC
• MP3
• Opus
• Vorbis
• AAC

WebAudio

The Web Audio API is not supported at this time.

Volume Manager API

W3C Audio/ Video Standards define the volume property to control playback volume between 0.0 and 1.0 but, “the loudest setting may be lower than the system’s loudest possible setting.” This is to prevent disruptive and awkward experiences where visiting a website unexpectedly causes a loud advertisement to play.

However, many Cloud Phone devices lack a physical volume rocker. This makes it difficult to raise and lower the volume without exiting an application.

To overcome these limitations, Cloud Phone provides the Volume Manager API to raise and lower the system volume. These APIs display the volume heads up display (HUD), which differs between devices in both style and number of increments.

The Volume Manager API lets users select an appropriate volume without leaving your widget. The API consists of three functions that all return void. While the HUD is visible, widgets cannot intercept the ArrowUp or ArrowDown keys as these raise and lower the volume, respectively.

// Displays the Volume HUD
// While visible:
// - ArrowUp raises volume
// - ArrowDown lowers volume
navigator.volumeManager.requestShow();

// Displays the Volume HUD and RAISES volume
navigator.volumeManager.requestUp();

// Displays the Volume HUD and LOWERS volume
navigator.volumeManager.requestDown();

interface VolumeManager {
    requestShow(): void;
    requestUp(): void;
    requestDown(): void;
}

declare global {
  interface Navigator {
    volumeManager?: VolumeManager;
  }
}

Autoplay

Autoplay is enabled on Cloud Phone. The autoplay attribute can be set on <audio> and <video> elements. If set, autoplay returns true when queried in JavaScript.

If your Cloud Phone widget shares a codebase with another platform, there are libraries like can-autoplay designed to detect when autoplay is enabled. The experimental Navigator.getAutoplayPolicy() and non-standard allowedToPlay attributes are not supported at this time.

Video

Codecs

Cloud Phone supports the MPEG-4 (MP4) video file format, including the H.264 standard with MPEG-4 audio, as well as the following codecs:

Codec Formats

• AV1
• VP8
• VP9
• H.264

HTTP Live Streaming (HLS)

Cloud Phone natively supports adaptive bitrate streaming without libraries like HLS.js.

<video width="240" height="320" controls>
  <source src="playlist.m3u8" type="application/x-mpegURL" />
</video>

Cloud Phone automatically selects the lowest bitrate stream to appropriate the resolution that most closely matches QQVGA or QVGA screens. This may select audio-only streams if those are included and lower bitrate than video streams. In the example M3U8 playlist file, the third audio-only stream has the lowest BANDWIDTH of 57544, so it would be selected over the two video streams.

#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=506651,RESOLUTION=640x480
https://example.com/stream?id=0
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=155108,RESOLUTION=320x240
https://example.com/stream?id=1
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=57544,CODECS="mp4a.40.2"
https://example.com/stream?id=2

Feature Detection

You can feature detect native HLS support. This is especially useful if your sharing a codebase with another platform.

function hasNativeHLS() {
  return videoEl.canPlayType('application/vnd.apple.mpegurl') === 'probably';
}

Unsupported APIs

Media Source Extensions API

Cloud Phone does not support the Media Source Extension API. Check for the presence of the MediaSource interface and fall back to other media sources as needed.

function hasMediaSource() {
  return window.MediaSource !== undefined;
}

Encrypted Media Extension API

Cloud Phone does not support the Encrypted Media Extension API needed to play content restricted by Digital Rights Management (DRM). Although APIs like navigator.requestMediaKeySystemAccess are available on Cloud Phone, the following key systems are unavailable at this time:

• Widevine: com.widevine.alpha
• PlayReady: com.microsoft.playready, com.youtube.playready
• ClearKey: webkit-org.w3.clearkey, org.w3.clearkey
• PrimeTime: com.adobe.primetime, com.adobe.access
• FairPlay: com.apple.fairplay

WebRTC

WebRTC (Web Real-Time Communication) APIs for candidate selection and peer-to-peer connections are not supported on Cloud Phone at this time.

You can connect to the microphone using navigator.mediaDevices.getUserMedia() which provides access to a MediaStream. This is available on Cloud Phone 3.0 or higher.

Summary

Cloud Phone has extensive support for streaming audio and video, including native support for live-streaming. With the same codec and container format as Chrome, you don’t need to make changes on Cloud Phone to accommodate the limited hardware of low-end feature phones.