Motion Tracking - ML5 with ZIM

Announcements
1

Just spent the afternoon working with ML5 with ZIM. Goodness, it was so easy. Why did we not do this years ago! I kept telling people, if you want hand tracking, face tracking, body tracking, etc. use TensorFlow. ML5 is based on TensorFlow and makes it easier. That is what P5js is using. But, I never went there myself to see.

image

https://zimjs.com/ml5

Will take a look at bringing the ml5 into perhaps the cam module so we can just import cam.

5 Likes
2

WOW!!! that so great!
But https://unpkg.com/ml5@1/dist/ml5.min.js didn't load in the html

3

Strange - it does for us - try just the URL do you see it?

We added a callback (modelLoaded) to the first picture one - maybe it was needed? Not sure, like I said, worked for us without it but on the cam ones, it did not work unless we added the callback. So try again, and if anyone has problems, let us know.

image
image739×144 6.53 KB

4

Hmmm... ml is 4mb minified so perhaps we will not automatically add it with the cam module. There is probably an es6 module or just use it as a script tag.

1 Like
5

Added a pinch example https://zimjs.com/ml5/pinch.html

image

We may internalize some of this

2 Likes
6

waauw so cool @abstract a dream comes true..

7

Found there is a general JavaScript videoWidth and videoHeight issue where we are getting 640x480 even though our camera is 1280x720. If we specify like this:

const cam = new Cam({
	width:1280,
	height:720,
	config:{
		width: 1280, // otherwise, seems to get the video as 640x480
		height: 720
	}
});

Then it seems to work - but not sure what happens on a 640x480 cam. Ran into a couple cases with different computers not doing it right. So will continue to test but for now am hardcoding the 1280x720

2 Likes