Map touch gestures to keyboard events

It is possible to handle touchscreen input on a client by simulating keyboard events.

The actual mapping between the touch event and the corresponding key event is done using the native language of the client platform. You then queue a built-in Pureweb command to transmit the converted event.

The Asteroids sample application provides a good example of this, as described in this section.

How-to

Simulate keyboard events

The logic for mapping touch events to keyboard events could be articulated as "if the user taps this area of the screen, behave as if the user had pressed the right arrow key; if the user taps that other area of the screen, behave as if the user had pressed the left arrow key", and so on.

The Asteroids sample application provides a good example of this. For instance, when the HTML5 Asteroids application runs on a mobile device, arrow buttons on the left-hand side of the screen allow users to play the game using touch input.

The mapping between the touch event and the corresponding key event is done using the native language of the client platform.

The PureWeb APIs come into play when you are ready to transmit the converted event to the service application. To do this, you would queue the built-in InputEvent command. The service APIs have an under-the-hood handler for this command, that responds as needed based on the event details provided in the command parameters. The parameters needed are:

  • EventType: the type of event, such as KeyUp or KeyDown
  • Path: the name of the view where the event occurred
  • Keycode: the non-character key that the touch event simulated (arrow, space, etc.). if applicable
  • CharacterCode: the character key that the touch event simulated, if applicable
  • Modifiers: the modifier key that the touch event simulated (Alt, Ctrl, etc), if applicable

See the example below.

Example

Using touch gestures with Asteroids

This example, taken from the Asteroids sample application, illustrates how to convert touch input to keyboard events, and send them as an InputEvent command with parameters.

HTML5

Set up the buttons with ontouch functions in the html file.

<div id="leftButton" ontouchstart="simKeyDown(event, LEFT_KEYCODE);" ontouchend="simKeyUp(event, LEFT_KEYCODE);" onmspointerup="simKeyUp(event, LEFT_KEYCODE);" onmspointerdown="simKeyDown(event, LEFT_KEYCODE);"></div>

<div id="rightButton" ontouchstart="simKeyDown(event, RIGHT_KEYCODE);" ontouchend="simKeyUp(event, RIGHT_KEYCODE);" onmspointerup="simKeyUp(event, RIGHT_KEYCODE);" onmspointerdown="simKeyDown(event, RIGHT_KEYCODE);"></div>

<div id="forwardButton" ontouchstart="simKeyDown(event, THRUST_KEYCODE);" ontouchend="simKeyUp(event, THRUST_KEYCODE);" onmspointerup="simKeyUp(event, THRUST_KEYCODE);" onmspointerdown="simKeyDown(event, THRUST_KEYCODE);"></div>

<div id="reverseButton" ontouchstart="simKeyDown(event, REVERSE_KEYCODE);" ontouchend="simKeyUp(event, REVERSE_KEYCODE);" onmspointerup="simKeyUp(event, REVERSE_KEYCODE);" onmspointerdown="simKeyDown(event, REVERSE_KEYCODE);"></div>

<div id="fireButton" ontouchstart="touchDown(event);" ontouchend="touchUp(event);" onmspointerup="touchUp(event);" onmspointerdown="touchDown(event);"></div>

<div id="shieldsButton" ontouchstart="simKeyDown(event, SHIELDS_KEYCODE);" ontouchend="simKeyUp(event, SHIELDS_KEYCODE);" onmspointerup="simKeyUp(event, SHIELDS);" onmspointerdown="simKeyDown(event, SHIELDS);"></div>

Define the ontouch handlers in the .js file.

//Key codes for simulating key events
var FIRE_KEYCODE = 32; //Space key
var THRUST_KEYCODE = 38; //Up cursor key
var REVERSE_KEYCODE = 40; //Down cursor key
var LEFT_KEYCODE = 37; //Left cursor key
var RIGHT_KEYCODE = 39; //Right cursor key
var SHIELDS_KEYCODE = 83; //'s' key
					
//Simulate a key up event
function simKeyUp(e, keyCode) {
    //Suppress the default action
    e.preventDefault();

    //Send the event as a key up event
    queueKeyboardEvent('KeyUp', keyCode);
}

//Simulate a key down event
function simKeyDown(e, keyCode) {
    //Suppress the default action
    e.preventDefault();

    //Send the event as a key down event
    queueKeyboardEvent('KeyDown', keyCode);
}

//Transmit the converted event to the service using a PureWeb command
function queueKeyboardEvent(eventType, keyCode) {

    //Create the array of event parameters as a JS object
    var parameters = {
        'EventType': eventType,
        'Path': 'AsteroidsView',
        'KeyCode': keyCode,
        'CharacterCode': 0,
        'Modifiers': 0
    };
    //Send the PureWeb InputEvent command
    pureweb.getClient().queueCommand('InputEvent', parameters);
}

iOS (Swift)

The iOS Asteroids client provides examples of simulating keyboard presses in Swift. The directional pad is implemented as a UIVisualEffectView custom class, which is bound to a View element on the story board. Touches within the view are handled with the following code:

override func touchesBegan(touches: Set<UITouch>, withEvent event: UIEvent?) 
{
    for touch in touches
    {
        processTouch(touch, withMode:.Started);
    }
}
							
override func touchesEnded(touches: Set<UITouch>, withEvent event: UIEvent?) 
{
    for touch in touches
    { 
        processTouch(touch, withMode:.Ended); 
    }
}
								
func processTouch(touch: UITouch, withMode mode: ActionMode) 
{
    let location = touch.locationInView(self)
    if location.y > location.x 
    {
        if location.y > (self.frame.size.height - location.x)
        { 
            bottomWithMode(mode)
        }
        else
        {
            leftWithMode(mode)
        }
    }
    else {
        if location.y > (self.frame.size.height - location.x)
        {
            rightWithMode(mode)
        }
        else { 
            topWithMode(mode); 
        }
    }
}

func leftWithMode(mode: ActionMode) {
    if mode == .Started 
    {
        queueKeyPress("KeyDown",  keycode: .Left,  modifiers:0);
    }
    else {
        queueKeyPress("KeyUp", keycode: .Left, modifiers:0);
    }
}
    
func rightWithMode(mode: ActionMode) {
    if mode == .Started
    {
        queueKeyPress("KeyDown",  keycode: .Right,  modifiers:0);
    }
    else {
        queueKeyPress("KeyUp", keycode: .Right, modifiers:0);
    }
}
    
func topWithMode(mode: ActionMode) {
    if mode == .Started 
    {
        queueKeyPress("KeyDown",  keycode: .Up,  modifiers:0);
    }
    else {
        queueKeyPress("KeyUp", keycode: .Up, modifiers:0);
    }
}
    
func bottomWithMode(mode: ActionMode) {
    if mode == .Started 
    {
        queueKeyPress("KeyDown",  keycode: .Down,  modifiers:0);
    }
    else {
        queueKeyPress("KeyUp", keycode: .Down, modifiers:0);
    }
}

The queueKeyPress function transmits the converted event to the service using the InputEvent command.

func queueKeyPress(eventType: String, keycode: KeyCode, modifiers: Int)
{
    // Create the array of event parameters to send with the command
    let cmdParams = [ "EventType" : eventType, "Path" : "AsteroidsView", "KeyCode" : "\(keycode.rawValue)", "Modifiers" : "\(modifiers)" ]
    
    // Send the PureWeb InputEvent command
    PWFramework.sharedInstance().client().queueCommand("InputEvent", withParameters:cmdParams);
}

The fire and shield buttons work similarly, with the methods being bound to view elements on the storyboard for the individual buttons (see screenshot).

@IBAction func fireBegan(sender: AnyObject) {
    queueKeyPress("KeyDown", keycode: .Space, modifiers: 0)
}
   
@IBAction func fireEnd(sender: AnyObject) {
    queueKeyPress("KeyUp", keycode: .Space, modifiers: 0)
}

@IBAction func shieldBegan(sender: AnyObject) {
    queueKeyPress("KeyDown", keycode: .KeyCodeS, modifiers: 0)
}
    
@IBAction func shieldEnd(sender: AnyObject) {
    queueKeyPress("KeyUp", keycode: .KeyCodeS, modifiers: 0)
}
    
func queueKeyPress(eventType: String, keycode: KeyCode, modifiers: Int)
{
    let cmdParams = [ "EventType" : eventType, "Path" : "AsteroidsView", "KeyCode" : "\(keycode.rawValue)", "Modifiers" : "\(modifiers)" ]
        
    PWFramework.sharedInstance().client().queueCommand("InputEvent", withParameters:cmdParams);
}

Android

The onTouch method handles game button touches. It translates the button presses/releases into keyboard events. This method calls queueKeyboardEvent, described further down.

@Override
public boolean onTouch(android.view.View v, MotionEvent event)
{
    // determine whether this is a key down or up event
    KeyboardEventType eventType;
	int action = event.getAction();

	switch (action)
	{
		case MotionEvent.ACTION_DOWN:
		case MotionEvent.ACTION_POINTER_1_DOWN:
		case MotionEvent.ACTION_POINTER_2_DOWN:
		case MotionEvent.ACTION_POINTER_3_DOWN:
		    eventType = KeyboardEventType.KeyDown;
		break;

		case MotionEvent.ACTION_UP:
		case MotionEvent.ACTION_POINTER_1_UP:
		case MotionEvent.ACTION_POINTER_2_UP:
		case MotionEvent.ACTION_POINTER_3_UP:
		    eventType = KeyboardEventType.KeyUp;
		break;
		...
	}
	pointerIndex = event.getPointerId(pointerIndex);
						
	// determine which button was pressed (if any)
	if (eventType == KeyboardEventType.KeyDown)
	{
		if (pointerIndex == 0)
		{
			pressedButtonId[0] = v.getId();
		} else
		{
			int[] xy = new int[2];
			getScreenCoordinates(v.getId(), (int)event.getX(pointerIndex), (int)event.getY(pointerIndex), xy);
			pressedButtonId[pointerIndex] = getGameButtonId(xy[0], xy[1]);
			...
			queueKeyboardEvent(pressedButtonId[pointerIndex], eventType);
		...
}				

The queueKeyboardEvent function transmits the converted event to the service using the InputEvent command. It uses a switch statement with a case for each of the game buttons.

private void queueKeyboardEvent(int button, KeyboardEventType eventType) {
    KeyCode keyCode;

	if (button == R.id.fire) {
		keyCode = KeyCode.Space;
	} else if (button == R.id.shields) {
		keyCode = KeyCode.S;
	} else if (button == R.id.forward) {
		keyCode = KeyCode.Up;
	} else if (button == R.id.reverse) {
		keyCode = KeyCode.Down;
	} else if (button == R.id.left) {
		keyCode = KeyCode.Left;
	} else if (button == R.id.right) {
		keyCode = KeyCode.Right;
	} else {
		return; // not a game button - ignore!
	}
							
	// Create the array of event parameters to send with the command
    Map<String, Object> parameters = new HashMap<String, Object>();
	parameters.put("EventType", eventType);
	parameters.put("Path", "AsteroidsView");
	parameters.put("KeyCode", keyCode.getKeyCode());
	parameters.put("CharacterCode", 0);
	parameters.put("Modifiers", Modifiers.None.toInt());

	// Send the PureWeb InputEvent command
	framework.getWebClient().queueCommand("InputEvent", parameters);
	}
}