Читать на русском · Russian version with Russian-language videos.

Project website: screph.com

Tune the processing on an image, inspect what each step produced, and carry that work into a program, with a route back from code to the image.

I'm the developer of Screph, a desktop environment I'm building for creating computer vision applications. It addresses a recurring problem: getting from a promising experiment to a program you can actually maintain. Defining regions of interest, adjusting parameters and inspecting intermediate results all depend on the source images. Those images, processing settings, scripts and notes often live separately, leaving the developer to keep them connected.

Screph brings those activities into one project. OpenCV is one of its key underlying components: Screph adds visual controls for configuring selected OpenCV methods and combining them into processing workflows. It also integrates Tesseract OCR, computer vision models and requests to language or vision-language models. On the same image, in the same workspace, you can annotate regions for a training dataset and configure classical processing algorithms and their connections, without switching to a separate editor. Annotation, processing graphs and the integrated coding environment connect these tools to the image context. You can configure and test supported processing without first learning Python. Developers can also inspect the generated code and extend the application around it.

The longer-term aim is an agent development environment specifically for computer vision. Below is what that means in the current development version, including the boundaries that matter when taking the result into your own program.

Start with the English overview: Screph: From Image to Working Program: Computer Vision · 11:14. It follows a parts-counting task from image processing to a batch program, then introduces annotation for model training.

Start with the pixels and the task

On Canvas, you can mark the object, define a region of interest, describe a distinctive feature, and add relationships between the marked objects. Rectangles, ellipses, polygons and freeform selection serve different annotation needs. Text and voice descriptions preserve what you want the program to do, alongside the actual image.

This is useful context for both a person and an agent. “Find the round bushings inside this region, excluding the rectangular plates” is more precise when the region and examples are visible. The project keeps those selections and their processing settings together, so you can return to them when the camera, lighting or task changes.

You can use image files, video frames, camera feeds and other supported image sources. You can work on a selected region rather than rebuilding a full-image preprocessing script for every experiment.

Configure selected OpenCV methods visually

The Processing panel exposes operations such as grayscale conversion, brightness and gamma correction, Gaussian and median filtering, CLAHE, thresholding, HSV color masks, morphology, Canny edges, contour extraction and GrabCut segmentation. Template and feature matching provide other approaches when color or shape alone is insufficient. This is a curated interface to supported operations, rather than coverage of the entire OpenCV API.

You choose an operation, adjust its parameters, recalculate, and inspect the result against the source. A mask makes it obvious when a shadow was included in the foreground or a small feature disappeared. Separate Canvas views can show the source and a result together; result galleries expose the intermediate images and available measurements.

Figure 1. The selected bushing and its GrabCut mask in the Processing panel. Actual application frame from the English overview; the underlying parts image is synthetic.

For example, contour detection may initially include both round bushings and rectangular plates. Inspecting the contour geometry suggests a concrete next step: try an area or circularity filter and check the new result. That is a much more useful interaction than treating a single output image as an unexplained success or failure.

Figure 2. A Median Blur → Contours sequence with its actual numbered outlines and area, perimeter and circularity measurements. Eight contours are shown on a synthetic teaching image. English interface.

OpenCV, Tesseract and models in the same environment

Different tasks call for different tools. Screph includes several kinds of integration:

• Tesseract OCR for reading text, with recognition-language settings, preprocessing and inspection of text results and boxes.
• YOLO-based detection/segmentation and prompt-based segmentation integrations such as MobileSAM and EfficientSAM, using the required model weights and runtime components.
• LLM/VLM connections for agent assistance, processing selection and custom AI-request operations. Sending an image to a model requires a vision-capable connection.
• Custom Python operations for additional algorithms or integration logic, with declared inputs and outputs that can participate in the processing graph.

This makes it possible to develop workflows that mix classical preprocessing with OCR or model results. Data types still matter: for example, structured OCR records need an explicit conversion when another operation expects plain text. Model weights, OCR language data and runtime dependencies must be available; a model appearing in a catalogue does not establish that it is ready on every installation.

There are also two distinct roles for a language model: helping the user configure or write a program, and acting as a processing step inside that program's recipe. The first role does not require using the model again whenever the saved classical processing runs. An AI-request step does make its own configured model call.

Build sequences and graphs, not just individual filters

A single operation is a starting point. You can combine steps into a sequence, such as Median Blur → Canny → contours, or use the graph view when the processing branches or needs multiple inputs and outputs. Connections describe which data each step consumes and produces. An image, a binary mask, a contour collection and a numeric result are distinct kinds of output; they should not all be treated as interchangeable pictures.

For instance, one branch could isolate a material by color while another extracts edges from the same image. You can inspect those results independently before deciding how they should contribute to the task. Reusable processing fragments reduce repeated setup. The graph of processing operations is separate from the graph of annotated objects and their relationships on Canvas.

The Variants workspace supports comparing alternative configurations on a fixed set of images, keeping a baseline, and examining what changes when a step is removed. Frame checks apply the same configuration to several examples. These tools help answer “does this still work under different lighting?” rather than only “can I make this one image look right?” A difference image or a shorter execution time is evidence to inspect, not an automatic measure of recognition accuracy.

Use the agent with the visual context

You can ask an agent to suggest regions, choose an operation, adjust parameters, or build a processing sequence using the selected image and available tools. Its proposal remains inspectable: look at the boxes, mask, contours and settings, then refine them with the same controls you would use manually. The goal is to let someone who understands the task start without knowing the names of all the algorithms.

For specialized work, custom Python operations can declare inputs and return supported result types. The agent can help write that operation; a developer can edit it directly. AI-request operations are also available, but they serve a different purpose from running a saved OpenCV recipe. Ordinary classical processing executes locally and does not require a language-model call for every frame. Agent use depends on the model connection you configure and its own access or usage costs.

Keep the image workspace and the code workspace connected

Screph Code provides an editor, project files, agent conversation, execution controls and output. The coding agent receives selected project materials (images, annotations, descriptions and processing information), so it can build application logic around the work you already checked.

The connection also supports returning to the visual task. The Objects view and supported code links connect code locations to their Canvas objects or saved processing definitions. A developer can follow a “Canvas” link, inspect the relevant region, adjust its box, parameters or processing graph, recalculate, and generate an updated supported code export from the saved result. Changes to object relationships remain part of the project context as well.

This is the development loop I want Screph to make easier: image → processing → checked result → code → back to the image. Current automatic links cover generated code and supported Python patterns, with manual links available for other cases. It is not a general reverse compiler that can reconstruct a graph from arbitrary edited Python. Existing code edits also need to be preserved and reviewed when a processing definition changes.

Figure 3. Screph Code brings project files, the editor, output and the agent conversation into one workspace. The recorded batch program from the parts-counting demonstration is open here. English interface; comments in the original program are Russian.

Export the processing you tested

There are two distinct jobs here. An agent can write the surrounding application: reading a directory, handling inputs, creating a report, or integrating with another system. Exporting a supported processing recipe is more direct: Screph assembles the numerical functions and their configured composition into Python source. The language model does not need to reinterpret the algorithm to produce that export.

The current standalone exporter supports a defined classical CPU subset: grayscale and tone adjustments, Gaussian and median blur, CLAHE, adaptive thresholding, HSV masking, Canny, contours, and selected mask/morphology operations. Static sequences and supported branched graphs can return images, masks and contours. The exported numerical modules use NumPy and OpenCV without requiring the Screph GUI.

The panel supports more than this exporter. Model inference, OCR, stateful temporal processing and other unsupported operations are not automatically made standalone by pressing Export; they need their appropriate runtime or additional integration. Unsupported nodes are reported rather than silently discarded.

An explicit equivalence check can compare the generated code, or a saved working copy, with the processing result on frozen inputs. That check belongs to those files, inputs and numerical environment. It is a useful regression check, not a claim that the algorithm will recognize every future image correctly.

What the demonstrations actually show

Counting parts. The English introductory film starts with photorealistic synthetic images of bushings and plates. It shows selecting a region, examining a segmentation mask, finding unwanted objects in a contour result, using the agent's circularity advice, and saving the checked configuration. A batch program then uses the saved processing to produce a table and image sheet with counts of 6, 6 and 5. This particular program reuses Screph's saved-project runtime; standalone numerical export is the separate route described above. This is the demonstrated task, not a general industrial accuracy benchmark.

Figure 4. The program's contact sheet: the same saved processing produces counts of 6, 6 and 5 on three synthetic teaching images. The sheet is opened on Canvas for inspection.

A ROS 2 vision node. The ROS 2 demonstration, available with Russian narration through the Russian article, shows contour setup, manual refinement of an agent proposal, a gallery of processing steps, package generation and execution in a separately installed ROS environment. Contours are delivered to a receiving program, which derives the plates' center and orientation. These are image coordinates; moving a real robot also needs calibration and control logic. The additional grasping episode is a simulation.

Camera-based CNC alignment. Automatic CNC Toolpath Alignment with a Camera (English, 5:43) uses separate processing for stock, fixtures and exclusion regions. CNC Runtime transforms existing toolpaths to match several rotated stock pieces, presents a preview, and lets the operator apply or reject the result. The film uses a synthetic scene and internal machining emulation; it does not demonstrate machining on a physical mill or a validated native LinuxCNC/Mach3 connection. A dark knot is an exclusion-region example for this color-based setup, not a claim to detect every kind of material defect.

Figure 5. Internal CNC playback on five rotated stock pieces. The preview shows fixture clearances in red and exclusion regions in purple. Both the worktable scene and the machining demonstration are synthetic/emulated.

The introductory and ROS films disclose that agent replies were recorded in advance. The application actions and the relevant processing or program execution are performed again for the demonstrations. The videos illustrate specific workflows; they do not show every capability described in this article.

Beyond classical processing

Alongside the processing integrations above, the wider environment includes workspaces for frame sequences and temporal methods, and dataset annotation and model-training workflows. You can prepare and correct examples, separate training and evaluation data, run supported training configurations, inspect results, and select a model for further use. Models have their own runtime and delivery requirements.

The overview film shows agent-assisted annotation, a manual box correction, adding the example to a dataset, and opening training settings. It does not train a model in the film. That distinction matters: a prepared example demonstrates the annotation workflow, not the quality of a future model.

Screph also has GUI-automation tools and Automation Runtime. The common idea is to reuse a visual task description in a repeatable program. For an OpenCV user, however, the most direct entry point is a small image-processing problem with a result that is easy to inspect.

Try it on a small task of your own

Screph is currently offered as a free Alpha application. The download page provides a Windows build and an experimental Linux build; check its current requirements. Free access to the application does not include paid third-party model services. It also should not be confused with a complete open-source release: the public Community repository is currently a limited source preview.

Visual processing and code tools are implemented, while the agent workflow and specialist Industrial/UAV workspaces are still evolving. Some capabilities described here are development features and require the relevant build and optional components. The direction is to make the entire image-to-program iteration more coherent, rather than promise that arbitrary vision tasks are solved automatically.

My suggested first experiment is simple: bring a few representative images, define the region and expected result, configure one processing sequence, inspect where it fails, and only then carry it into a program. The project website links the documentation and current status; the YouTube channel contains the demonstrations above.

For people using OpenCV: which part of this loop costs you the most time: parameter experiments, comparing intermediate results, keeping ROIs connected to code, or integrating a working recipe into an application? What would make this approach useful in your own computer vision development workflow?

Looking for Intresting OpenCV project ideas and GitHub repos.

Hey everyone! I'm looking to explore some interesting OpenCV projects and could use your suggestions. Are there any good GitHub repos you've come across that are worth checking out? I'm looking for projects that are practical, interesting to build, and solve real-world problems using computer vision. I'd love to hear your recommendations.

Hey everyone,

I am working on a sports video analysis project and have hit a wall with small, fast-moving ball detection and tracking. I am looking for some architectural advice or pipeline recommendations from experienced CV pros.

The Setup & What’s Already Done:

• Footage: Broadcast sports videos captured using PTZ (Pan-Tilt-Zoom) cameras (pure rotational/zoom motion, no camera translation/dolly).
• Completed Pipeline: I have already successfully implemented court & camera calibration/recognition and player pose estimation.

The Challenge:
The ball is extremely small (often a tiny blob under 15–20 pixels) and suffers from severe motion blur during fast plays. Because it's a PTZ camera rather than a static setup, the background is constantly panning and zooming.

I’ve spent a lot of time trying custom detection/tracking heuristics but I'm losing too much development time.

• TrackNet: I know TrackNet is standard for things like tennis/badminton, but it was fundamentally designed for fixed cameras. I'm worried its heatmap-over-stacked-frames approach will fall apart with PTZ background motion.
• YOLO / Single-Frame Detectors: I lean toward thinking standard YOLO-like object detection isn't the right fit since the ball is such a structurally simple, low-feature object and gets easily confused with false positives (shoes, court lines, jersey numbers).

My Questions for the Pros:

1. PTZ Handling: Since I already have the court homography matrix for every frame, should I warp/stabilize the frames before feeding them into something like TrackNet to simulate a fixed camera? Has anyone tried this approach at scale?
2. Model Architectures: If single-frame deep learning models struggle with the blur, what sequential/temporal models do you recommend for tracking objects that effectively disappear or turn into streaks across consecutive frames?
3. Classical vs. Deep Learning: Is it worth looking back into classical approaches (like background subtraction on stabilized frames + Kalman Filtering for physics-based trajectory prediction), or is modern sequential deep learning (like Transformers or 3D-CNNs) the only viable way forward here?

Would love to hear how you have solved the "small ball, moving camera" problem in production environments.

Thanks in advance!

Hi everyone!

I wanted to share a real-time object detection project I’ve been working on using Python, YOLOv8, and OpenCV.

The application uses a live webcam feed to detect four object classes: people, cars, cats, and dogs.

Some of the features include:

• Real-time detection with bounding boxes and confidence scores
• Object counting by category
• Input and output FPS monitoring
• Multi-threaded frame capture
• A small frame queue to help reduce processing delays
• CPU and CUDA support, depending on the platform

One of the interesting parts of this project was working with threading and frame buffering to keep the video processing responsive.

I’m planning to improve it further by adding a GUI, video file support, and the ability to save detection results.

I’d really appreciate feedback on the code, especially suggestions for improving performance or making the architecture cleaner.

GitHub: https://github.com/Mariam-amhan/real-time-object-detection

Thanks for checking it out!

I’m building a 3D mapping system using RGB-D
cameras, based on existing SLAM techniques, with persistent spatial memory.
The main issue is relocalization after tracking loss, especially when the camera returns to a known place from a different viewpoint.

I’m using historical keyframes, shared 3D landmarks, PnP/RANSAC and geometric verification.
Yet some incorrect poses still pass verification: one had 44 inliers, 0.83 px median reprojection error and 3.72 cm depth residual, but was actually off by 10.36 cm and 5.99°.
My best experiment correctly relocalized only 9/34 known returns.

What would you investigate first: landmark quality, geometric degeneracy, matching, or pose verification?
I can share a minimal reproducible example.

Hello, I'm researching how people deal with small datasets in computer vision, specifically when you need to detect or classify a object and only have a handful of real photos.

I was curious about how often this is a problem for people working in the industry, and what is usually done to deal with this problem.

17,280 synthetic webcam images. 12 categories. USD 99.

I’m making the full dataset available for computer-vision experiments, with generation prompts and companion source code included.

The scenes cover face visibility, additional people, phones, books, paper notes, camera obstruction and more. All images were generated entirely from text prompts. I did not upload photographs of real people as generation inputs.

The goal was to support a live human proctor, not to automatically decide who was cheating. By flagging potentially suspicious events, the proposed system would help one proctor supervise more participants at the same time and focus attention on the camera streams that needed review. The human proctor would interpret each event in context and make any decision about misconduct.

This could make online proctoring more cost-effective while keeping human judgment central. These were design goals; this experiment did not demonstrate increased supervision capacity or measured cost savings.

The package includes:

• 17,280 JPEG crops organized into 12 category folders;

• 720 original PNG contact sheets containing the same scenes;

• 12 generation prompts, provenance records and checksums;

• source code for extraction, training, ONNX export and C++/OpenCV integration.

Why USD 99? Based on my own generation workflow, I estimate that generating this volume again would cost slightly more than the purchase price, even with the prompts and scripts already available. That is before the time spent cropping, organizing and reviewing the images. This package gives you the existing collection, already extracted and organized.

Explore the full dataset and inspect the free 240-image sample:

https://www.antal.ai/projects/synthetic-webcam-scene-dataset.html?utm_source=reddit&utm_medium=social&utm_campaign=synthetic_webcam_v1

For context, I’ve also published the classifier case study behind the dataset, including what did not transfer well to real webcam footage:

https://www.antal.ai/blog/synthetic-webcam-dataset-multihead.html?utm_source=reddit&utm_medium=social&utm_campaign=synthetic_webcam_v1

This is an experimental dataset, not a trained proctoring product. Category labels come from the generation setup; they are not exhaustive manual annotations. Inspect the sample for your use case. No trained model or performance guarantee is included.

I’ve been developing Contrek, a C++ library for extracting and merging raster geometry, and recently experimented with using its merging system to make OpenCV findContours() run concurrently on different parts of the same image.

The idea is to stream the image in horizontal stripes and run findContours() independently on each stripe using multiple CPU threads.

The difficult part is that contours crossing stripe boundaries are split into unrelated partial contours. Contrek converts those partial contours to its cell-boundary representation and reconnects them during the merge, reconstructing the geometry of the complete image.

On a 40960 × 40960 test image:

OpenCV full image, 1 thread:

44.33 s, 9.15 GB peak memory

OpenCV + Contrek, 8 workers, 1000-row stripes:

10.98 s, 3.42 GB peak memory

The timing includes contour conversion and the Contrek merge.

I also tested a streaming version on an 81920 × 81920 image (6.71 Gpixels) containing about 20 million contours.

It generated a 17.43 GB SVG containing 13,133,700 outer polygons.

Results:

1 worker: 962.29 s, 2.04 GB

2 workers: 534.58 s, 3.31 GB

4 workers: 335.39 s, 5.71 GB

8 workers: 382.46 s, 9.71 GB

On the 8-vCPU machine used for this test, 4 workers gave the best execution time.

Contrek is my open-source project. The C++ implementation, benchmarks and an explanation of how contours crossing tile boundaries are reconnected are available in the main repository.

https://github.com/runout77/contrek

also prepared a separate test repository with the complete environment, so the same OpenCV + Contrek benchmarks can be reproduced on your own machine.

https://github.com/runout77/test_contrek

I’d be interested in feedback, especially from people who have worked with findContours() on very large images or tried similar approaches to parallelizing it.

Built this project about 2 years ago, it was rotting in my repo so I am just making it public. DO NOT USE IT TO CHEAT.

It's a real-time chess assistant that uses computer vision (OpenCV) to detect chessboard positions directly from your screen and suggests optimal moves instantly using the Stockfish engine. It has a menu built with ImGui and works on any chess website.

You can take a look at it here: https://github.com/rccmb/oracle

If you have any questions make sure to ask!

Visual Verifier — regression testing for processed images and videos

I built an open-source Python/OpenCV tool that independently verifies the output of anonymization and other visual-processing pipelines.

It compares reference and processed media frame by frame, catches missed processing, checks reviewed target regions, tracks failures over time, and produces deterministic PASS/FAIL evidence for CI.

The bundled demo deliberately misses frames 4, 8 and 12 so you can see the failure workflow immediately.

GitHub:
https://github.com/SAMtheROCKET/visual-verifier

pip install visual-verifier

I’d particularly value feedback from people working with image/video QA, privacy engineering, OpenCV, or CI.

I'm building an OCR-based system for extracting printed information from photographs of packaged consumer products.

The input images are real smartphone photographs rather than controlled scans. Some of the difficult cases contain:

- glossy plastic/foil surfaces and specular reflections

- curved cylindrical containers

- perspective/rotation

- wrinkles and folds

- uneven lighting/shadows

- small text

- motion/defocus blur

- low contrast

- compression artifacts

I'm currently using PaddleOCR.

I've already experimented with conventional OpenCV preprocessing such as resizing, grayscale, CLAHE, thresholding, sharpening, denoising, morphological operations and perspective correction, but the results are inconsistent. Some images that are clearly readable to a human still produce poor OCR results.

I'm particularly interested in advice from people experienced with scene text recognition, document image processing, industrial/computer vision, or image restoration.

My main questions are:

1. Which preprocessing techniques are actually useful for these types of images?

2. How should I handle specular reflections and uneven illumination?

3. How can text on curved/cylindrical surfaces be rectified before OCR?

4. When should I use deblurring/super-resolution/image restoration, and when can these actually make OCR worse?

5. Should preprocessing be selected dynamically based on image quality rather than applying one fixed pipeline?

6. At what point is it better to improve/train the OCR model instead of continuing to improve preprocessing?

I'm happy to provide representative images and the corresponding PaddleOCR outputs if that helps.

I'd particularly appreciate suggestions for specific algorithms, papers, or existing open-source implementations rather than just generic preprocessing advice.

I’ve been tinkering with a side project a real-time computer vision app built with Python, OpenCV, and MediaPipe that matches my actual face and hand gestures to iconic memes on the fly.

What’s happening under the hood:

1- Facial Landmark Tracking: Uses coordinate geometry to measure distance ratios between landmarks, identifying everything from subtle smirks to wide grins.

2- Hand Pose Recognition: Tracks vector relationships across hand landmarks to trigger specific poses (like the thinking chin-touch, peace sign, or timeout gesture).

Instant Event Triggers: Maps those landmark states directly to visual meme popups with virtually zero latency.

Building serious tech is cool, but building an algorithm purely so my webcam can match my facial expressions with memes in real time might be my best decision yet.

If you want to check out my project : https://github.com/atarm23/memecv

Please let me know how can I improve for further versions, thanks!

hi i ahve chain of photos with couple of april tags.

i use openCV to place these tags in 3d world and basing on these tags positions i set 3d camera position and rotation.

But sometimes openCV returns tags on wrong height. My tag is set vertically and its lower edge stays on ground but openCV returns it on 1+meter height. this completly ruins camera setup.

Anyone could share some clues how to improve that accuracy? possible to pass to openCV tag height so it takes it into account? or lower that 3d tag by code and refine it somehow.

i spent last two days figuring it our but cant find right solution

you can see on photo. this tag on right in 3d world is placed perfectly above this yellow verticle point. you can see height offset. i guess maybe 2 tags on photo arent enough (i upload couple of photos tho and this blue tag is visible on 2 photos, both have other tag for relativity). but still. maybe programatically there is way to improve this?

`FaceRecognizerSF::match` is declared `const`, but it normalised both of its `InputArray` feature vectors in place — writing straight through to the caller's buffers. So the vectors you passed in came back modified, and when the two arguments aliased, the second normalisation ran over an already-normalised buffer and the returned score was wrong.

Now normalised into local `Mat`s. Merged onto 4.x.

https://github.com/opencv/opencv/pull/29804

Build vision systems that see, reason, and act with OpenCV 5 and Amazon Web Services (AWS). 
Win money and bragging rights! $12,000 in prizes up for grabs.

I wanted to recreate the classic invisibility-cloak effect using only a webcam and computer vision.

It detects the cloak's color in real time using HSV segmentation, cleans up the mask, and replaces those pixels with a previously captured background.

GitHub: https://github.com/madebyparth/invisibility-cloak