Webcam Example

Webcam Service

The example robot also has webcams that can be accessed using the Python OpenCV libraries. Example simple_webcam_service_multi.py on GitHub is a program that exposes the webcams as a Robot Raconteur service. The example is intended to demonstrate the usage of the objref, pipe, and memory members that were not used in the iRobot Create examples.

#Service to provide sample interface to webcams

# This interface is for example only. Most cameras should use the standard com.robotraconteur.imaging.Camera interface
service experimental.simplewebcam3

# The current version of Robot Raconteur robdef standards is 0.10
stdver 0.10

struct WebcamImage
    field int32 width
    field int32 height
    field int32 step
    field uint8[] data
end

struct WebcamImage_size
    field int32 width
    field int32 height
    field int32 step
end

object Webcam
    property string name [readonly]
    function WebcamImage capture_frame()

    function void start_streaming()
    function void stop_streaming()
    pipe WebcamImage frame_stream [readonly]

    function WebcamImage_size capture_frame_to_buffer()
    memory uint8[] buffer [readonly]
    memory uint8[*] multidimbuffer [readonly]

end

object WebcamHost
    property string{int32} webcam_names [readonly]
    objref Webcam{int32} webcams
end

The service definition for the experimental.simplewebcam3 shown in above contains two objects: WebcamHost and Webcam. The Webcam object type represents a single camera, and the WebcamHost object allows for the client to determine the number of webcams and retrieve the Webcam objects through an objref member.

The class WebcamHost_impl implements the WebcamHost object type. The property webcam_names returns a map of the indexes and names of the cameras, and is an example of the string{int32} Robot Raconteur type. The function get_webcams implements the webcams objref. Note that the objref is implemented by prepending get\_ to the name of the objref member. The index may come as a string even though an int32 is expected, so convert the type to int before using. When returning an object from an objref, it is necessary to return the fully qualified Robot Raconteur type of the object as a second parameter.

Note: objects can only be registered as a service object ONCE. Objects cannot be returned by two separate objrefs. Objrefs must form a “tree” structure, where the child branches are the return objects from objrefs.

The Webcam_impl object implements the webcam functionality. The capture_frame function returns a single frame to the client. The start_streaming and stop_streaming functions begin or stop a thread implemented by the frame_threadfunc function that sends streaming frames to the connected clients through the frame_stream pipe.

A PipeBroadcaster is used for this example. The PipeBroadcaster is similar to the WireBroadcaster, sending packets to all connected clients. While a PipeBroadcaster can be inferred for a readonly pipe and the attribute set in the same manner as a WireBroadcaster, for this example the PipeBroadcaster is initialized by the object so the backlog can be specified. The backlog is used for flow control. If there are more packets “in-flight” than the specified maximum, more will not be sent. The property “FrameStream” is implemented, with the getter initializing the PipeBroadcaster. The function frame_threadfunc demonstrates using the PipeBroadcaster to send frames to the clients.

The final members of interest in the WebcamService are the two memories, buffer and

multidimbuffer. These two members demonstrate how to use two flavors of memories that are either single dimensional or multi-dimensional. Memories are useful when data greater than about 10 MB needs to be transmitted between client and server, when there is a random-access block of memory, or in the future for shared memory applications. The function capture_frame_to_buffer captures the data and saves it to the buffers. Note that multi-dimensional arrays in Python are simply multi-dimensional NumPy arrays. Some processing is done to place the data in “Matlab” style image formats. A structure of type experimental.simplewebcam3.WebcamImage_size is returned to tell the client how big the image is.

The two memories are implemented in Python using properties. The buffer member returns an ArrayMemory object, and the multidimbuffer returns a MultiDimArrayMemory object. Both contain their respective array and multi-dimensional array. In this example a new memory object is returned every time. This is not generally the best way to use the memory; instead, a persistent memory object should be used with a persistent memory block.

Webcam Client

Example simple_webcam_client.py on GitHub is a program that will read the webcams and display the images. The initialization and connection are similar to the iRobot Create example. The main difference is the use of the objrefs, which are used to get references to the webcams c1 and c2:

c1=c_host.get_webcams(0)

c2=c_host.get_webcams(1)

The rest of the program deals with OpenCV related functions to show the images.

Webcam Client (streaming)

Example simple_webcam_host_streaming.py is a program that provides a “live” view of the camera, although depending on the speed of the computer it may be fairly slow because Python is an interpreted language. The program connects and retrieves the webcam object reference c the same way as the previous example, and the connects to the pipe frame_stream. The pipe index is given as the argument, and -1 means any index.

p=c.frame_stream.Connect(-1)

Next, a callback is added so that the function new_frame will be called when a new pipe packet arrives.

p.PacketReceivedEvent+=new_frame

This function will be called from a different thread by Robot Raconteur when a new frame has arrived. In the new_frame function, the variable current_frame is updated with the new value. The Available property in the PipeEndpoint provides the number of packets waiting, and the ReceivePacket retrieves the next packet. Packets always arrive in order.

The rest of the program handles showing the images as they arrive and shutting down, including closing the pipe.

p.Close()

Note that this example uses the single webcam version of the service simple_webcam_service.py.

Webcam Client (memory)

Example simple_webcam_host_memory.py on GitHub demonstrates the use of the memories. The memories have functions Read and Write that allow for a segment of the memory to be read or written into or from a buffer. The memory position, buffer, buffer position, and count are passed. For multi-dimensional arrays, the memory position, buffer position, and count are lists. The ArrayMemory has the special property “Length” for the length of the array, and the MultiDimArrayMemory has the special properties Dims, DimCount, and Complex.

Note that this example uses the single webcam version of the service simple_webcam_service.py.