University of Udine (https://www.uniud.it), University of Trieste (https://www.units.it/)
Polytechnic Department of Engineering and Architecture (University of Udine), Department of Engineering and Architecture (University of Trieste)
Captain: Lorenzo Scalera
Lorenzo Scalera, PhD student in robotics at University of Udine (Italy), and prof. Paolo Gallina, associate professor of applied mechanics at University of Trieste (Italy), are the members of the team “Busker Robot”. Our innovative robotic painting system has been developed for rendering images into original watercolour artworks, by means of a six degrees of freedom robotic arm and a series of image processing and path-planning algorithms. Robotic painting is a process that generates interest and fascination and our challenge is to introduce a novel artistic and original expression, starting from a digital image and taking advantage of the beautiful effects generated by the watercolour technique. The painting machine we have used for our artworks is a UR10 collaborative robot, by Industrial Robot, which we have equipped for painting purposes. The robot is provided with collision-detecting systems and force limits and, therefore, the artist controlling the system can operate side by side with the robot during the whole painting process. The robot end-effector is equipped with an aluminium flange that allows an easy integration of the brushes by means of 3D printed plastic supports. Furthermore, an automatic brush change system has been designed and implemented. Prior the painting process is started, the robot and the drawing surface are calibrated in order to avoid planarity errors and the position of the colour cups, paper and brushes tool repository have to be known by the robotic system. Watercolour painting is a complex and challenging technique, and its properties are related to the semi-transparent suspensions of fine pigments in water and to the interaction between watered paint and paper. This technique allows to apply several layers of colour, one over others that have already dried, and allows the colours to be mixed and diluted, generating incontrollable effects such as backgrounds, granulation, flow-patterns and edge-darkening. For all these reasons, it can be understand how challenging the automation of watercolour painting could be. In order to better controlling the stroke width and the colour intensity along the strokes a characterization of strokes and brushes has been performed in a preliminary phase. The painting process of Busker Robot starts with the elaboration of a digital image. Several non-photorealistic rendering techniques have been developed and implemented in order to extract backgrounds and large areas, as well as contours and thin details from the input image. The output of these algorithms is a set of points that describes the extracted features of the image. Then, the points are elaborated into a series of trajectories for the robot, including the motions for the paint refill, the brush change and draining. Finally, the resulting script file is deployed on the robot controller. The algorithms, briefly explained in the following, have been implemented in Matlab environment by adopting the UR Script language to control the robot. We can group the algorithms that we have implemented into two main categories: algorithms for large areas and backgrounds and algorithms for details and contours. The elaboration of large areas is based on the concept of grey-scale threshold: given a specific intensity level, each pixel of the original image (firstly processes in grey-scale) is converted into a binary number equal to 1 or 0, depending on the intensity of that pixel with respect to the threshold one. The algorithms that we named Hatching and Gradient-Based Random Strokes are based on this operation. Hatching technique consists in a uniform filling of large areas (defined by a grey level) with parallel and closely spaced lines. The angular direction of the lines can be chosen by the artist and, in this manner, different thresholds, angles and distance between lines can be applied on the same image. Hatching has been applied to Trieste Seafront for the filling of light grey and black areas. Gradient-Based Random Strokes algorithm is a technique that allows the covering of a determined zone of an image with random strokes, perpendicular to the direction of the grey-scale gradient. The starting point of each line is randomly generated inside the area defined by a threshold and, in this way, each time the algorithm is run, a different outcome is obtained, resulting in an original and artistic covering. This algorithm has been applied for the processing of backgrounds and the large areas in all the artworks presented in this competition. The complete script for the Gradient-Based Random Stroke algorithm is available as example and it can be downloaded at the following link: https://sites.google.com/site/buskerrobot/home/gradient-based-random-strokes. For the process of details and contours different techniques have been implemented: Canny, Hough Transform, and Difference of Gaussians. Canny is one of the most famous algorithms for edge detection that we have adopted for the contours definition on the Trieste Seafront artwork. Hough Transform, adopted in the Sailboat, is a technique that is based on the recognition of lines in an image using a parametric representation. Finally, Difference of Gaussians (DOG) is based on the Gaussian blur of a grey-scale image at a specific standard deviation and window size, to suppress only high-frequency spatial information. The blurred version of the original image is then subtracted from another, less blurred version of the original, by allowing to obtain the final edge detection of the original image. The DOG algorithm has been applied to process the details and contours in Paolini, Goat and Amanda artworks. It has to be noticed that our algorithms are not used all together to elaborate a single image: it is the artist that choses the best combination of techniques in each case. A trial and error process, led by the artist’s aesthetic appreciation and fulfillment has usually been run before reaching a satisfactory outcome. Furthermore, the artist can influence the results by setting and adjusting the software parameters of each single algorithm, as well as changing the paper and brush type and all the parameters related to the interaction between the robot and the surrounding environment. The implemented algorithms and the results obtained with Busker Robot have been recently described in detail on scientific papers which have been submitted to international journals and conferences. The artworks produced by our robotic painting system have been appreciated by the public in recent international fairs and exhibitions as well as by local and international press. We are confident that, in future, Busker Robot could become a relevant example of robotic painting, attracting the interests of visitors and artists.