OCR - OBS Plugin

Introduction

The OCR Plugin for OBS provides real-time Optical Characted Recognition (OCR) or Text Recognition abilities over any OBS Source that provides an image - can be Image, Video, Browser or any other Source. It is based on the incredible Tesseract open source OCR engine, compiled and running directly inside OBS for real-time operation on every frame rendered.

Reading scoreboards? Try https://scoresight.live free OCR tool specifically made (by us) for reading and broadcasting scoreboards.

If this free plugin has been valuable to you consider adding a ⭐ to this GH repo, subscribing to my YouTube channel where I post updates, and supporting my work: https://www.patreon.com/RoyShilkrot https://github.com/sponsors/royshil

Usage Tutorials

4 Minutes

Do more with OCR Plugin

OCR Plugin enables many use cases for enhancing your stream or recording:

Features

Available now:

• Add OCR Filter to any source with image or video output
• Choose from Scoreboard model or English, French, Spanish, German, Chinese, Japanese, Arabic, Turkish, Portugese, Hindi, Russian and Italian
• Choose the segmentation mode: Word, Line, Page, etc.
• "Semantic Smoothing": getting more consistent outputs with higher accuracy and confidence by "averaging" several text outputs
• Timing/Running modes: per X-milliseconds
• Output OCR result to an OBS Text Source
• Output to a text file (with/out aggregation)
• Output formatting (with inja): e.g. "Score: {{score}}"
• Output text detection to image source (draws boxes, text, etc.)
• Output to settings (e.g. for other plugins to use as triggers)
• Binarization methods (threshold, Otsu, Triangle, adaptive)
• Image Dilation
• Rescale (optimal Tesseract performance is at 35 pixels / character)

Coming soon:

• More languages built-in (pretrained Tesseract models)
• Allowing external model files
• More output capabilities e.g. Parsing, websocket event, etc.
• Detection area selection (to prevent using Crop/Pad Filter)
• Different timing/run modes: per X-frames, image change, etc.
• Image stabilization
• Optical flow tracking for fast moving text
• Image processing: Perspective warping, auto-cropping, etc.
• Advanced binarization: Niblack, Sauvola

Check out our other plugins:

• Background Removal removes background from webcam without a green screen.
• Detect will detect and track >80 types of objects in any OBS source.
• LocalVocal speech AI assistant plugin for real-time, local transcription (captions), translation and more language functions
• Polyglot translation AI plugin for real-time, local translation to hunderds of languages
• URL/API Source will connect to any URL/API HTTP and get the data/image/audio to your scene.
• 🚧 Experimental 🚧 CleanStream for real-time filler word (uh,um) and profanity removal from live audio stream

If you like this work, which is given to you completely free of charge, please consider supporting it https://github.com/sponsors/royshil or https://www.patreon.com/RoyShilkrot

Download

Check out the latest releases for downloads and install instructions.

Building

The plugin was built and tested on Mac OSX (Intel & Apple silicon), Windows and Linux.

Start by cloning this repo to a directory of your choice.

Mac OSX

Using the CI pipeline scripts, locally you would just call the zsh script. By default this builds a universal binary for both Intel and Apple Silicon. To build for a specific architecture please see .github/scripts/.build.zsh for the -arch options.

$ ./.github/scripts/build-macos -c Release

Install

The above script should succeed and the plugin files (e.g. obs-ocr.plugin) will reside in the ./release/Release folder off of the root. Copy the .plugin file to the OBS directory e.g. ~/Library/Application Support/obs-studio/plugins.

To get .pkg installer file, run for example

$ ./.github/scripts/package-macos -c Release

(Note that maybe the outputs will be in the Release folder and not the install folder like pakage-macos expects, so you will need to rename the folder from build_x86_64/Release to build_x86_64/install)

Linux (Ubuntu)

Use the CI scripts again

$ ./.github/scripts/build-linux.sh

Copy the results to the standard OBS folders on Ubuntu

$ sudo cp -R release/RelWithDebInfo/lib/* /usr/lib/x86_64-linux-gnu/
$ sudo cp -R release/RelWithDebInfo/share/* /usr/share/

Note: The official OBS plugins guide recommends adding plugins to the ~/.config/obs-studio/plugins folder.

Windows

Use the CI scripts again, for example:

> .github/scripts/Build-Windows.ps1 -Target x64 -CMakeGenerator "Visual Studio 17 2022"

The build should exist in the ./release folder off the root. You can manually install the files in the OBS directory.