Vida is a simple library that adds camera (or video) based motion detection and blob tracking functionality to p5js.

The library allows motion detection based on a static or progressive background; defining rectangular zones in the monitored image, inside which the occurrence of motion triggers the reaction of the program; detection of moving objects ("blobs") with unique index, position, mass, rectangle, approximated polygon.

The main guidelines of the library are to maintain the code in a compact form, easy to modify, hack and rework.

Vida is a part of the Tetoki! project and is developed thanks to substantial help and cooperation with the WRO Art Center and HAT Research Center.

link to statically hosted file:
https://tetoki.eu/vida/vida/p5.vida.min.js

Download current Vida release (v. 0.3.00a, 20.09.2019) with examples.
Examples

NOTE: Not all web browsers allow the use of a video camera (eg built into a laptop or connected via USB). This is not related to the construction of the Vida library, or even with p5js, but with the browser manufacturer's treatment of security. Sometimes this is related to the operating system (eg iOS in some cases blocks access to the camera), sometimes with security settings or certificates. Some of these settings can be changed (eg under macOS Safari supports the camera accessed from sites over HTTPS after enabling the WebRTC->Enable Legacy RTC API option in the Develop menu), but generally not all browsers work with cameras. Therefore, all examples of using the possibilities of Vida are doubled: each example exists in the variant using a video camera and variant using video file as video source. Currently the best browser on macOS, Windows and Linux to test p5js sketches hosted locally or loaded from a webserver and using video camera (and Vida library) is Mozilla Firefox.

Basics of image processing and Vida: working with a static background:


static background image + video camera


static background image + video file


Basics of image processing and Vida: working with progressive background:


progressive background image + video camera


progressive background image + video file


Basics of working with active zones:


active zones + static background image + video camera


active zones + static background image + video file


active zones + progressive background image + video camera


active zones + progressive background image + video file


More about active zones (accessing data, custom drawing, etc.):


(advanced) active zones + static background image + video camera


(advanced) active zones + static background image + video file


(advanced) active zones + progressive background image + video camera


(advanced) active zones + progressive background image + video file


Basics of working with blobs:


blobs + static background image + video camera


blobs + static background image + video file


blobs + progressive background image + video camera


blobs + progressive background image + video file


More about blobs (accessing data, custom drawing, etc.):


(advanced) blobs + static background image + video camera


(advanced) blobs + static background image + video file


(advanced) blobs + progressive background image + video camera


(advanced) blobs + progressive background image + video file

Reference

NOTE: Vida is using (with a few exceptions) normalized coords instead of pixel-based. Thus, the coordinates of the active zones, the location of detected moving objects (and some of their other parameters) are represented by floating point numbers within the range from 0.0 to 1.0. The use of normalized coordinates and parameters allows to manipulate the resolution of the image being processed (eg from a video camera) without having to change eg the position of active zones. Analogously, data describing moving objects is easier to use, if their values are not related to any specific resolution expressed in pixels. Names of all normalized parameters are preceded by the prefix "norm". The upper left corner of the image has the coordinates [0.0, 0.0]. The bottom right corner of the image has the coordinates [1.0, 1.0].

Vida

The main library pseudo-class. You must create an object derived from this class to use the library. The constructor requires one parameter: a "handle" to the p5js sketch, which contains and manages the created object, so the syntax looks like this:
var v = new Vida(this);
or (e.g. if you are using instance mode):
var v = new Vida(your_p5js_sketch);

activeZones[]

Array that stores defined active zones. Each array cell contains an object derived from VidaActiveZone pseudo-class.

addActiveZone(identifier, normX, normY, normW, normH, onChangeCallbackFunction)

This function adds active zone to the Vida instance. Parameters:
identifier - zone's identifier (integer or string);
normX, normY, normW, normH - normalized (!) rectangle;
onChangeCallbackFunction - optional parameter, callback function (triggered on change).

approximateBlobPolygonsFlag

Boolean variable that turns on or off detecting a polygons describing the blobs. It takes one of two values: "false" or "true".

backgroundImage

Variable (p5.Image) that stores current background image.

currentImage

Variable (p5.Graphics) that stores the image data passed to the Vida object.

getBlobs()

Function that returns an array of currently detected blobs. Each array cell contains one object derived from the VidaBlob pseudo-class.

handleActiveZonesFlag

Boolean variable that turns on or off handling active zones by the Vida object. It takes one of two values: "false" or "true".

handleBlobsFlag

Boolean variable that turns on or off detecting "blobs" in processed image. It takes one of two values: "false" or "true".

imageFilterFeedback

Value of the feedback for the procedure that calculates the background image in progressive mode. The value should be in the range from 0.0 to 1.0 (float). Typical values of this variable are in the range between ~0.9 and ~0.98.

imageFilterThreshold

Value of the threshold for the procedure that calculates the threshold image. The value should be in the range from 0.0 to 1.0 (float).

differenceImage

Variable (p5.Image) that stores current difference image.

drawActiveZones(x, y, w, h)

Function drawing active zones. Can be used with two or four parameters (w and h are optional). Parameters:
x, y - pixel-based position of the upper-left corner;
w, h - width and height expressed in pixels (if they are not explicitly provided, the function assumes that the width and height of the image being drawn will be equal to the width and height of the image processed by the Vida instance [last transferred in the update() function]);

drawBlobs(x, y, w, h)

Function drawing blobs. Can be used with two or four parameters (w and h are optional). Parameters:
x, y - pixel-based position of the upper-left corner;
w, h - width and height expressed in pixels (if they are not explicitly provided, the function assumes that the width and height of the image being drawn will be equal to the width and height of the image processed by the Vida instance [last transferred in the update() function]);

mirror

Variable that stores information about whether and how to flip the image passed in the update() function. It accepts values of one of the pseudo-constants MIRROR_XXXXXXX.

MIRROR_BOTH

Pseudo-constant (do not modify it's value!) that defines the horizontal and vartical flip of the image passed as a parameter to the update() function.

MIRROR_HORIZONTAL

Pseudo-constant (do not modify it's value!) that defines the horizontal flip of the image passed as a parameter to the update() function.

MIRROR_NONE

Pseudo-constant (do not modify it's value!) that defines "no flip" of the image passed as a parameter to the update() function.

MIRROR_VERTICAL

Pseudo-constant (do not modify it's value!) that defines the vertical flip of the image passed as a parameter to the update() function.

normMaxBlobArea

Highest acceptable normalized value for the area of the blob. The variable can take floating point values between 0.0 and 1.0.

normMaxBlobMass

Highest acceptable normalized value for the weight (mass) of the blob. The variable can take floating point values between 0.0 and 1.0.

normMinBlobArea

Lowest acceptable normalized value for the area of the blob. The variable can take floating point values between 0.0 and 1.0.

normMinBlobMass

Lowest acceptable normalized value for the weight (mass) of the blob.The variable can take floating point values between 0.0 and 1.0.

pointsPerApproximatedBlobPolygon

Variable that stores the value determining the number of points of the polygon describing each of the blobs. The variable accepts integer values greater or equal to 3.

progressiveBackgroundFlag

Boolean variable that turns on or off the progressive background mode. It takes one of two values: "false" or "true".

rejectBlobsMethod

Variable that holds the value of one of the pseudo-constants defining the blobs' filtering method due to their location in the monitored image (Vida may prefer smaller blobs located inside larger or the opposite: reject smaller blobs inside larger ones. The mechanism can also be completely disabled). The variable can take values of one of the pseudo-constants REJECT_XXXXXXX.

REJECT_INNER_BLOBS

Pseudo-constant (do not modify it's value!) that defines "reject-inner-blobs" mode.

REJECT_NONE_BLOBS

Pseudo-constant (do not modify it's value!) that defines no-reject mode.

REJECT_OUTER_BLOBS

Pseudo-constant (do not modify it's value!) that defines "reject-outer-blobs" mode.

removeActiveZone(identifier)

Function removes active zone from the Vida instance. The parameter indicates the zone identifier to be deleted.

setActiveZonesNormFillThreshold(threshold)

Function changing sensitivity of the active zones. The value (floating point number in the range from 0.0 to 1.0) passed to the function determines the movement intensity threshold which must be exceeded to trigger the zone (so, higher the parameter value = lower the zone sensitivity).

setBackgroundImage(image)

Function overwrites the state of the backgroundImage variable. Accepts p5.Image or p5.Graphics as a parameter. If the image flip mode is different than MIRROR_NONE the function will also overwrite the currentImage variable.

thresholdImage

Variable (p5.Image) that stores current threshold image.

trackBlobsFlag

If this boolean variable is set (and handleBlobsFlag value is set, too) Vida tries to find the blobs that seem to be the continuation of the previous ones and give them identifiers (id's) of probable "ancestors". It takes one of two values: "false" or "true".

trackBlobsMaxNormDist

Maximum allowable normalized distance between the blobs used by the tracking mechanism (enabled when the trackBlobsFlag is set). The variable can take floating point values between 0.0 and 1.0.

update(image)

Vida update function calculating a new state of the Vida instance, to which we pass the current image data. Usually this function is called in the draw loop (once per repetition). Accepts p5.Image or p5.Graphics as a parameter.

VidaActiveZone

Pseudo-class used to create objects that store active zone data. Objects of this class are usually not created manually by the user. The user uses rather addActiveZone function (from the Vida pseudo-class).

changedFrameCount

Variable storing the time (expressed in frames) when the zone changed it's status (if the value of the isChangedFlag was changed). Under normal conditions it is read-only.

changedTime

Variable storing the time (expressed in milliseconds) when the zone changed it's status (if the value of the isChangedFlag was changed). Under normal conditions it is read-only.

id

Text string or integer representing the zone identifier.

isChangedFlag

Boolean variable indicating whether the zone changed it's status in the current frame. Under normal conditions it is read-only and it's value is "false" or "true".

isEnabledFlag

Boolean variable that is enabling/disabling the testing of a the active zone without deleting it. It takes one of two values: "false" or "true".

isMovementDetectedFlag

Boolean variable indicating whether there was movement detected inside the zone. Under normal conditions it is read-only and it's value is "false" or "true".

normFillFactor

Normalized value defining the level of filling the active zone with detected movement. Under normal conditions it is read-only. It's value ranges from 0.0 to 1.0.

normFillThreshold

Normalized threshold value, which must exceed the movement level (value of the normFillFactor) to decide that there was traffic in the zone (to set the isMovementDetectedFlag).

normH

Normalized value specifying the height of the active zone. The variable can take floating point values from the 0.0 to 1.0 range.

normW

Normalized value specifying the width of the active zone. The variable can take floating point values from the 0.0 to 1.0 range.

normX

Normalized value specifying the horizontal position of the upper left corner of the active zone. The variable can take floating point values from the 0.0 to 1.0 range.

normY

Normalized value specifying the vertical position of the upper left corner of the active zone. The variable can take floating point values from the 0.0 to 1.0 range.

onChange(vidaActiveZone)

Function called when the zone changes it's status. Triggered zone (instance of the VidaActiveZone pseudo-class) will be passed to the called function as the parameter. In typical cases, the value of the variable that stores the "reference" to the callback function it will not be changed on-the-fly, by the user (because usually this reference is defined when creating the active zone with Vida's addActiveZone function).

VidaBlob

Pseudo-class used to create objects that store blob data. Objects of this class are usually not created or modified manually by the user (Vida creates them when the blob detection mode is enabled, by analyzing the content of the image being processed). Also individual values assigned to blob parameters under typical conditions are only read, not overwritten.

approximatedPolygon

Array storing the normalized coordinates of the approximate polygon describing the blob. If the polygon approximation is enabled (if Vida's approximateBlobPolygonsFlag is set), the array will be filled with objects storing normalized (floating point values from the 0.0 to 1.0 range) values defining the set of polygon points. Otherwise, the array will be empty.

creationFrameCount

Variable storing the time (expressed in frames) when the was created.

creationTime

Variable storing the time (expressed in milliseconds) when the was created.

id

Positive integer representing the unique identifier of the blob.

isNewFlag

Boolean variable that indicates whether the blob is a continuation of the previously detected, or a completely new entity. The variable value is calculated in the blob tracking procedure, so if Vida's trackBlobsFlag variable is not set all of the blobs will be marked as new.

normMass

Normalized value of the blob's "mass". The "mass" is calculated based on the ratio of the number of pixels occupied by the blob to the number of pixels in the image being processed.

normMassCenterX

Normalized (floating point values from the 0.0 to 1.0 range) value specifying the horizontal position of the blob's center of the mass. The mass center is calculated based on the average position of the pixels that make up the blob.

normMassCenterY

Normalized (floating point values from the 0.0 to 1.0 range) value specifying the vertical position of the blob's center of the mass. The mass center is calculated based on the average position of the pixels that make up the blob.

normRectH

Normalized (floating point values from the 0.0 to 1.0 range) value specifying height of the upper left corner of the blob's bounding box.

normRectW

Normalized (floating point values from the 0.0 to 1.0 range) value specifying width of the upper left corner of the blob's bounding box.

normRectX

Normalized (floating point values from the 0.0 to 1.0 range) value specifying the horizontal position of the upper left corner of the blob's bounding box.

normRectY

Normalized (floating point values from the 0.0 to 1.0 range) value specifying the vertical position of the upper left corner of the blob's bounding box.

Contact

write to: contact | at | tetoki | dot | eu

License
Creative Commons License
Vida by Paweł Janicki is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Based on a work at https://tetoki.eu.