An X-box Kinect tracked the users movements. The laptop then sent serial commands to an Arduino that interfaced with 3 motor controllers to control the model crane’s functions. a pneumatic bucket was added using Lego components, actuated using a solenoid valve.
The Kinect SDK makes it easy to access the x, y & z coordinates of the users joints. For the crane we decided that the jib/bucket movements should mimic the users arm position. For example, positioning your arm lower causes the bucket to continuously move downwards.