The SDL forums have moved to discourse.libsdl.org.
This is just a read-only archive of the previous forums, to keep old links working.


SDL Forum Index
SDL
Simple Directmedia Layer Forums
tvOS swipe left/right gesture support (patch enclosed)
oviano


Joined: 05 Mar 2016
Posts: 24
I found that events weren't being generated for the simple directional swipes on the Apple TV touch remote.

While it's possible to use the touch press/release/motion events to detect simple swipes of this nature (and I had this working well enough) it struck me that this was re-inventing the wheel.

As such, I have written a patch that recognises the proper swipe up/down/left/right events issued in UIKit and passes these on as directional key presses. This also means that the touch remote operates the same way as the old-style IR remote in this respect.

Code:

diff -r 007dfe83abf8 src/video/uikit/SDL_uikitview.m
--- a/src/video/uikit/SDL_uikitview.m   Wed Oct 19 20:50:33 2016 -0700
+++ b/src/video/uikit/SDL_uikitview.m   Sun Dec 18 14:57:42 2016 +0300
@@ -48,7 +48,20 @@
 #if !TARGET_OS_TV
         self.multipleTouchEnabled = YES;
 #endif
-
+#if TARGET_OS_TV
+        UISwipeGestureRecognizer *swipeUp = [[UISwipeGestureRecognizer alloc] initWithTarget:self action:@selector(swipeGesture:)];
+        swipeUp.direction = UISwipeGestureRecognizerDirectionUp;
+        [self addGestureRecognizer:swipeUp];
+        UISwipeGestureRecognizer *swipeDown = [[UISwipeGestureRecognizer alloc] initWithTarget:self action:@selector(swipeGesture:)];
+        swipeDown.direction = UISwipeGestureRecognizerDirectionDown;
+        [self addGestureRecognizer:swipeDown];
+        UISwipeGestureRecognizer *swipeLeft = [[UISwipeGestureRecognizer alloc] initWithTarget:self action:@selector(swipeGesture:)];
+        swipeLeft.direction = UISwipeGestureRecognizerDirectionLeft;
+        [self addGestureRecognizer:swipeLeft];
+        UISwipeGestureRecognizer *swipeRight = [[UISwipeGestureRecognizer alloc] initWithTarget:self action:@selector(swipeGesture:)];
+        swipeRight.direction = UISwipeGestureRecognizerDirectionRight;
+        [self addGestureRecognizer:swipeRight];
+#endif
         touchId = 1;
         SDL_AddTouch(touchId, "");
     }
@@ -56,6 +69,27 @@
     return self;
 }
 
+#if TARGET_OS_TV
+- (void)swipeGesture:(UISwipeGestureRecognizer *)sender
+{
+    switch (sender.direction)
+    {
+    case UISwipeGestureRecognizerDirectionUp:
+        SDL_SendKeyboardKey(SDL_PRESSED, SDL_SCANCODE_UP);
+        break;
+    case UISwipeGestureRecognizerDirectionDown:
+        SDL_SendKeyboardKey(SDL_PRESSED, SDL_SCANCODE_DOWN);
+        break;
+    case UISwipeGestureRecognizerDirectionLeft:
+        SDL_SendKeyboardKey(SDL_PRESSED, SDL_SCANCODE_LEFT);
+        break;
+    case UISwipeGestureRecognizerDirectionRight:
+        SDL_SendKeyboardKey(SDL_PRESSED, SDL_SCANCODE_RIGHT);
+        break;
+    }
+}
+#endif
+
 - (void)setSDLWindow:(SDL_Window *)window
 {
     SDL_WindowData *data = nil;
tvOS swipe left/right gesture support (patch enclosed)
Alex Szpakowski
Guest

SDL is already supposed to send left/right/up/down key presses when “press” events are detected on the Apple TV Remote ( wrote:
I found that events weren't being generated for the simple directional swipes on the Apple TV touch remote.While it's possible to use the touch press/release/motion events to detect simple swipes of this nature (and I had this working well enough) it struck me that this was re-inventing the wheel.As such, I have written a patch that recognises the proper swipe up/down/left/right events issued in UIKit and passes these on as directional key presses. This also means that the touch remote operates the same way as the old-style IR remote in this respect.Code:diff -r 007dfe83abf8 src/video/uikit/SDL_uikitview.m--- a/src/video/uikit/SDL_uikitview.m Wed Oct 19 20:50:33 2016 -0700+++ b/src/video/uikit/SDL_uikitview.m Sun Dec 18 14:57:42 2016 +0300@@ -48,7 +48,20 @@ #if !TARGET_OS_TV self.multipleTouchEnabled = YES; #endif-+#if TARGET_OS_TV+ UISwipeGestureRecognizer *swipeUp = [[UISwipeGestureRecognizer alloc] initWithTarget:self action:@selector(swipeGesture:)];+ swipeUp.direction = UISwipeGestureRecognizerDirectionUp;+ [self addGestureRecognizer:swipeUp];+ UISwipeGestureRecognizer *swipeDown = [[UISwipeGestureRecognizer alloc] initWithTarget:self action:@selector(swipeGesture:)];+ swipeDown.direction = UISwipeGestureRecognizerDirectionDown;+ [self addGestureRecognizer:swipeDown];+ UISwipeGestureRecognizer *swipeLeft = [[UISwipeGestureRecognizer alloc] initWithTarget:self action:@selector(swipeGesture:)];+ swipeLeft.direction = UISwipeGestureRecognizerDirectionLeft;+ [self addGestureRecognizer:swipeLeft];+ UISwipeGestureRecognizer *swipeRight = [[UISwipeGestureRecognizer alloc] initWithTarget:self action:@selector(swipeGesture:)];+ swipeRight.direction = UISwipeGestureRecognizerDirectionRight;+ [self addGestureRecognizer:swipeRight];+#endif touchId = 1; SDL_AddTouch(touchId, ""); }@@ -56,6 +69,27 @@ return self; } +#if TARGET_OS_TV+- (void)swipeGesture:(UISwipeGestureRecognizer *)sender+{+ switch (sender.direction)+ {+ case UISwipeGestureRecognizerDirectionUp:+ SDL_SendKeyboardKey(SDL_PRESSED, SDL_SCANCODE_UP);+ break;+ case UISwipeGestureRecognizerDirectionDown:+ SDL_SendKeyboardKey(SDL_PRESSED, SDL_SCANCODE_DOWN);+ break;+ case UISwipeGestureRecognizerDirectionLeft:+ SDL_SendKeyboardKey(SDL_PRESSED, SDL_SCANCODE_LEFT);+ break;+ case UISwipeGestureRecognizerDirectionRight:+ SDL_SendKeyboardKey(SDL_PRESSED, SDL_SCANCODE_RIGHT);+ break;+ }+}+#endif+ - (void)setSDLWindow:(SDL_Window *)window { SDL_WindowData *data = nil;

_______________________________________________SDL mailinghttp://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org
[/quote]
oviano


Joined: 05 Mar 2016
Posts: 24
That's great, but it doesn't work and no such events come through.

If you can point me in the direction of where the directional swipe gestures are handled in the code I can try and fix it, but I couldn't see it.

If you're talking about the old (non touch) remote then yes that generates the events correctly.

I'm talking about the swipes on the touch remote.
tvOS swipe left/right gesture support (patch enclosed)
Alex Szpakowski
Guest

wrote:
That's great, but it doesn't work and no such events come through.If you can point me in the direction of where the directional swipe gestures are handled in the code I can try and fix it, but I couldn't see it.If you're talking about the old (non touch) remote then yes that generates the events correctly.I'm talking about the swipes on the touch remote.
_______________________________________________SDL mailinghttp://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org
[/quote]
tvOS swipe left/right gesture support (patch enclosed)
Alex Szpakowski
Guest

I just tested on the tvOS Simulator and the key press events work for me with the latest SDL code from hg.
Quote:
On Dec 18, 2016, at 10:45 AM, Alex Szpakowski wrote:
wrote:
That's great, but it doesn't work and no such events come through.If you can point me in the direction of where the directional swipe gestures are handled in the code I can try and fix it, but I couldn't see it.If you're talking about the old (non touch) remote then yes that generates the events correctly.I'm talking about the swipes on the touch remote.
_______________________________________________SDL mailinghttp://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org




[/quote]
oviano


Joined: 05 Mar 2016
Posts: 24
Ok I'll try the simulator later but you're testing the swipe gestures, correct?
tvOS swipe left/right gesture support (patch enclosed)
Alex Szpakowski
Guest

The Apple TV Remote touchpad “tap” actions, as described here: https://developer.apple.com/tvos/human-interface-guidelines/remote-and-controllers/#buttons

The ‘tap’ actions map pretty cleanly to button presses, in terms of what their behaviour describes. SDL doesn’t really have analogous user-facing APIs that describe OS-handled swipe gestures in the same manner, however you can access the Remote’s touchpad as joystick axes in order to get a more granular description of what the user is doing with it.
Quote:
On Dec 18, 2016, at 11:55 AM, oviano wrote:
Ok I'll try the simulator later but you're testing the swipe gestures, correct?
_______________________________________________SDL mailinghttp://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org
oviano


Joined: 05 Mar 2016
Posts: 24
The up/down/left/right swipe gestures on Apple TVs are the primary method to navigate around button/menu systems on the device so assuming that some SDL apps will have button or menu systems that require the user to navigate from one button to another then it's far better to hook into these gestures...

Otherwise you are expecting the user to re-implement these themselves and inevitably there will be subtle differences in "feel" compared to the official ones leading to a substandard user experience.
tvOS swipe left/right gesture support (patch enclosed)
Alex Szpakowski
Guest

2 out of the 3 gesture types outlined here are implemented as key presses (click and tap) https://developer.apple.com/tvos/human-interface-guidelines/remote-and-controllers/#gestures

Given the fact that key presses can’t represent swipe strength which is one of the main purposes of a swipe, and the fact that the direction key presses are already used by the tap gestures, how do you propose swipes should be exposed in SDL’s APIs?
Quote:
On Dec 18, 2016, at 12:32 PM, oviano wrote:
The up/down/left/right swipe gestures on Apple TVs are the primary method to navigate around button/menu systems on the device so assuming that some SDL apps will have button or menu systems that require the user to navigate from one button to another then it's far better to hook into these gestures...Otherwise you are expecting the user to re-implement these themselves and inevitably there will be subtle differences in "feel" compared to the official ones leading to a substandard user experience.
_______________________________________________SDL mailinghttp://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org
tvOS swipe left/right gesture support (patch enclosed)
Alex Szpakowski
Guest

Quote:
Given the fact that key presses can’t represent swipe strength which is one of the main purposes of a swipe

Actually I suppose the Apple documentation is a bit misleading about this. Smile
_______________________________________________
SDL mailing list

http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org
tvOS swipe left/right gesture support (patch enclosed)
Alex Szpakowski
Guest

I pushed a commit which should do what you want: https://hg.libsdl.org/SDL/rev/a6babd973955

Quote:
On Dec 18, 2016, at 12:41 PM, Alex Szpakowski wrote:


Quote:
Given the fact that key presses can’t represent swipe strength which is one of the main purposes of a swipe

Actually I suppose the Apple documentation is a bit misleading about this. Smile

_______________________________________________
SDL mailing list

http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org
oviano


Joined: 05 Mar 2016
Posts: 24
Well the swipe gesture I'm talking about hooking into, and which my patch uses is just a simple standard swipe with direction only. There is concept of strength:

https://developer.apple.com/reference/uikit/uiswipegesturerecognizer

So it maps pretty easily to the arrow key presses.

It's funny - I actually had no idea (despite owning a touch-based Apple TV since they were available) that you could tap to produce the direction "buttons", I'd always swiped which is why it seems to me such a big omission. I simply had no way to navigate within the UI of my application (a video player) Smile
oviano


Joined: 05 Mar 2016
Posts: 24
There is *no* concept of strength, I should say.
oviano


Joined: 05 Mar 2016
Posts: 24
Thanks for making the change - I haven't tested your slight variation on my code yet but I will at some point and let you know if I encounter any problems.