Released: Virtual Webcam

While scrolling internal feed at workplace portal, I came across a hackathon project published by a couple of FB engineers internally. It used a virtual-webcam GitHub repo to provide a chrome extension which can mask the camera input to support certain effects, like overlay Gifs, stickers, dummy video, etc. Although the internal hack was to support some other things, I thought to try and build out what I conceived. My vision is to explore and understand how camera input can be intercepted and replaced with a pre-recorded video. I would play along with multiple things in the journey.

Goals

In short, Following are the reasons I decided to fork and play around with Virtual Webcam:

• Understand how to intercept camera input

• Understand how to add effects to the camera stream

• Understand the basics of building chrome extensions

• Understand testing chrome extensions

As POC, I made a simple website to achieve these: Virtual Webcam. I had observed that I may need to experiment more before it can be called as usable!

Flow

1. How to intercept camera input:

   
    // override the MediaDevices.getUserMedia() API
    const enumerateDevicesFn = MediaDevices.prototype.enumerateDevices;
    const getUserMediaFn = MediaDevices.prototype.getUserMedia;
   
    MediaDevices.prototype.enumerateDevices = async function () {
    const res = await enumerateDevicesFn.call(navigator.mediaDevices);
    // We could add "Virtual VHS" or "Virtual Median Filter" and map devices with filters.
    res.push({
        deviceId: "virtual",
        groupID: "uh",
        kind: "videoinput",
        label: "Virtual Chrome Webcam",
    });
    return res;
    };
   
    MediaDevices.prototype.getUserMedia = async function () {
    const res = await getUserMediaFn.call(navigator.mediaDevices, ...arguments);
    //// Intercepted here!!
    return res;
    };
    

2. How to add effects to the camera stream:

   
    // camera input
    const video = document.createElement("video");
    const canvas = document.createElement("canvas");
    this.canvas = canvas;
    this.renderer = new ShaderRenderer(this.canvas, video, shader);
   
    video.addEventListener("playing", () => {
    // Use a 2D Canvas.
    // this.canvas.width = this.video.videoWidth;
    // this.canvas.height = this.video.videoHeight;
   
    // Use a WebGL Renderer.
    this.renderer.setSize(this.video.videoWidth, this.video.videoHeight);
    this.update();
    });
    video.srcObject = stream;
    video.autoplay = true;
    this.video = video;
    //this.ctx = this.canvas.getContext('2d');
    this.outputStream = this.canvas.captureStream();
    

3. Basics of building chrome extensions:

   
    // having a manifest.json but cannot load any external resource
    {
    "name": "Virtual Webcam",
    "version": "1.0.0",
    "minimum_chrome_version": "10.0",
    "description": "Virtual webcam on all chrome tabs",
    "default_popup": "popup.html"
    },
    "content_scripts": [
        {
        "matches": [
            "<all_urls>"
        ],
        "js": [
            "js/inject.js"
        ],
        "run_at": "document_start",
        "all_frames": true
        }
    ],
        "permissions": [
        "tabs",
        "storage",
        "tabCapture",
        "<all_urls>",
        "http://*/*",
        "https://*/*"
        ],
        "manifest_version": 2,
            "web_accessible_resources": [
            "js/*"
            ]
    }
    &lt;/xmp&gt;&lt;/pre&gt;
   
   </all_urls></all_urls>

4. Testing chrome extensions:

   I needed to upload the manifest.json in chrome://extensions Load unpacked option and all files were picked up automatically on the go.

I felt it was not so straightforward to implement and it had a lot of dependencies to slow me down, especially around Chrome Extensions, Canvas, WebGL, etc. Having a simple website to overlay camera video with effects worked fine for me, but I am yet to test how it would perform for browser-based video calls.

For my demo, the source is hosted on GitHub repo: Virtual Webcam repo, quick sneak peek is below:

Default Camera Input

B&W Filter

Flicker Filter

Cartoonize Filter