🕹 Yet Another Reason to Buy A Switch that Has Joy-Cons

  • where: RK
  • when: December 5, 2019

Resources

This page is playable as slides!

Press p to play as slides, then j and k to page.

Turning Switch's Joy-Con 🕹 into a clicker is non-trivial effort because you need to poll keypress events by yourself with the Gamepad API.

Yet Another Reason to Buy A Switch that Has Joy-Cons

  • Part 1 (overview)

Today I'm talking about the Nintendo Switch. How does that relate to our life as a React developer? Turns out, you can use your professional skills to turn part of your Switch, namely the Joy Cons, into a clicker that you can use to interact with a web page.

!!!

Some contexts

Before we dive in let me give you some more contexts. For nearly exactly how long this meetup exists I've been building the slides features for my site.

There are already many more existing arts

  • revealjs
  • slides.com
  • mdx deck
  • spectacle
  • dropbox paper
  • ...many developers choose to build their own

You may wonder, since there are already so many existing arts, why still create my own presentation slides? And I think that comes down to this very philosophical question:


"do chefs cook at home?"

So this is about my self ego as a developer who writes professional React components at work. Chefs can cook at home and feed everybody, what can I do with my professional skills?????

A good talk script should also be a good read

Also, since I write out the scripts for most talks I've done in my life, I appreciate good talks that are also good reads.

And since how I create the slides is roughly to take the headings out, add some emojis, and turn all the scripts into speaker notes. Why can't I render articles that can present?


It kinda shares the same spirit with Gatsby Theme Waves

Earlier today Raj shared about Gatsby Theme Waves. It kind of shares a same spirit, but a different product.

(initial version on rcs site)

So with all the motivations above I started building this with the birth of this meetup.

Then my appetite grew bigger

Then once this gets started you can't stop.

  • I need a timer...
  • can I have "speaker mode" synced with main slides?
  • can I have just speaker notes?
  • can I go full screen?
  • ...
  • can I remote control the slides?

there are many clickers out there

This actually can be trivial once you already have your keyboard shortcut up. Because there are many clickers out there already. Most of them will map to a key on your keyboard. Some even got very cool features such as a pointer.

omg what are these

Then I was at JSConfBP earlier this year. And during the closing keynote by Jake and Surma on the first day, they each is holding on to a Switch Joy Con.

  • Making things fast in world of build tools by Surma & Jake Archibald | JSConf Budapest 2019

all cool kids got wan

That's the thing a cool kid like everyone here in this room should get excited about. So I went ahead and borrowed a pair of Joy Cons, connected to my mac with bluetooth, it worked, and I tried to click it at my slides -- nothing works (apparently).

The Gamepad API

No magic will happen because those are not extra keyboards. They're gamepads. And we need to create the magic to make them work in browsers -- we need the Gamepad API.


By the way, what's your expectation?

So first, my naïve imagination was that I write something similar to keyboard event handlers that looks like this:


// some kind of key press events?
document.addEventListener('imaginary joycon press', event => {
  // do things?
});

WROOOOOOOONG

There are only two APIs

  • gamepadconnected
  • gamepaddisconnected

But no, WRONG. The only event handlers that the Gamepad API exposes are gamepadconnected and gamepaddisconnected. How to use them ah?

You need to poll key presses yourself

We need to start polling keypresses when gamepadconnected and stop this process when gamepaddisconnected.


dunno about you but I actually felt quite cheated.

- Brian's cat, name is "Gatsby"

so how to poll key presses ah

"game loop"

The main idea of game loop is that it's a snake that eats it's own tail. For each round, we do what we need to do, which normally is to do some updates. And once we're done, we call gameLoop again. And we should not forget to kick it off by calling it once first.

const gameLoop = () => {
  // update();

  // run again
  gameLoop();
};

gameLoop();

Interopting with rAF

But in real life we are "interopting" with the browser's event loop. In this case we are relying on the browser's requestAnimationFrame DOM API.

So we're weaving our updates into the browser's event loop, one frame at a time. We start this process when the gamepad is connected, and stop when it disconnects.

Everybody still with me?

let start;
const gameLoop = () => {
  // update();

  // run again
  start = requestAnimationFrame(gameLoop);
};

const gamepadConnect = evt => {
  start = requestAnimationFrame(gameLoop);
};

const gamepadDisconnect = () => {
  cancelAnimationFrame(start);
};

a bit more game loop

Here today we only very shallowly talked about game loop but that actually is a very interesting topic that by itself is worth a separate talk.

Its original intent was used to align game progress across different CPU time with our time. You have the chance to adjust your time, slice off extra or match your rendering with the progress with respect to real time. I learned last week from Shawn's talk that React also uses a "work loop" and "timeslicing" to keep time in check.

Implementing the game loop

Now, here's how we implement the game loop. Window's navigator API gives us access to our connected gamepads,

const gameLoop = () => {
  // get the gamepad
  var gamepads = navigator.getGamepads
    ? navigator.getGamepads()
    : navigator.webkitGetGamepads
    ? navigator.webkitGetGamepads
    : [];
  if (!gamepads) {
    return;
  }

  // do things...
  // run again...
};

when you peak into the gamepad object

then when we peak into what's inside the gamepad object, it'll be this giant object containing the data that represent our the most updated state of our gamepad. All of these are mutating.

Once again, on every single frame we have the most updated information of our gamepad.

The axes are for the joy stick, then there is an array of buttons one for each of the button on the gamepad, and we have pose for the gravity sensor.

{
  "axes": [1.2857142857142856, 0],
  "buttons": [
    {
      "pressed": false,
      "touched": false,
      "value": 0
    }
    // ... more (altogether 17) buttons
  ],
  "connected": true,
  "displayId": 0,
  "hand": "",
  "hapticActuators": [],
  "id": "57e-2006-Joy-Con (L)",
  "index": 0,
  "mapping": "standard",
  "pose": {
    "angularAcceleration": null,
    "angularVelocity": null,
    "hasOrientation": false,
    "hasPosition": false,
    "linearAcceleration": null,
    "linearVelocity": null,
    "orientation": null,
    "position": null
  },
  "timestamp": 590802
}

loop through all keys we care about and update for them

Now, having access to all those data available to us at every frame, we really have all the freedom however we want to use them. This is when, I think, how we organize our code matters. You can go for an object oriented approach, I've seen a couple of implementations that do that.

For my use case I find it makes most sense when I supply a list of buttons I care about, and supply the callbacks I want to fire if such and such buttons are pressed. Then during each frame of my game loop I can loop through those buttons only and update accordingly.

const gameLoop = () => {
  // get gamepad...

  // update for each "game" buttons
  gameButtons.forEach((button) => {
    if (buttonPressed(gamepad.buttons[button])) {
      typeof callbacks[button] === 'function' && callbacks[button]();
    }
  });

  // ... run again
};

const buttonPressed = key => {
  if (typeof key == 'object') {
    return key.pressed;
  }
  return false;
};

codepen earlier this year

Now maybe we're all excited to see some demos?

I happen to have drawn a Switch using CSS roughly a year ago 😅 So convenient, let's use that. If you're interested, here is the original CodePen for the Nintendo Switch art. And once again, the original design credits to its original creator Louie Mantia.

Original design Nintendo Switch art by Louie Mantia


I have a quick demo to get you started

beyond this

  • pointer?
  • more gestures?
  • "driver" for the lower level implementation?
  • design patterns - how to organize code

resources

If you want to join me, here are some resources that I found particularly helpful aside from playing around with code by yourself.

Links

Then finally there are some more links.