Making a Software Piano Keyboard for iPhone and iPad

I've recently been doing a lot of working involving audio on the iPhone and iPad. I thought it would be cool to make a multi-touch software keyboard. Making an on-screen keyboard has two main challenges: Laying out the images of the piano keys correctly and handling multiple touches accurately.

Laying out the keyboard:

There are two ways to lay out a keyboard: programatically and manually. To lay out the keyboard manually, you would create a new view in interface builder; then add the images of the keys to the view setting their positions manually. This is easy, but it lacks flexibility. If you want to add more keys, you would need to do it manually. If you wanted to change the scale of the keyboard, again, you would have to do it manually. After some consideration, I decided to lay the keyboard out programatically. To do this, you just need to loop over the number of white keys in the keyboard. For each loop, you create a new white key image and increase it's x-position by the necessary amount. After that you add the balck keys - the position of the black key is a bit more complex to work out and will depend on the dimensions of the images you're using for the keys:

  1. // Psudo-code
  2.  
  3. for (NSInteger i=0; i <= numberOfWhiteKeys; i++) {
  4. // Create new white key image
  5. UIImageView key = [new white key image]
  6.  
  7. // set the x - position
  8. x = i * [white key image width]
  9.  
  10. // Add the key
  11. [self addSubView: key]
  12. }
  13.  
  14. // Loop over the keys again and this time add the black keys

Handling Touch:

Next lets look at how to make the keyboard interactive. Here are the requirements for handling user touches:

  • Multiple touches must be handles correctly
  • A key must be activated when it receives a touch down event
  • A key must be deactivated when it receives a touch up event
  • If the user slides their finger over the keyboard, when the finger enters a key's boundary, the new key must be activated and the old key must be deactivated

The best way to achieve these requirements isn't immediately obvious. Handling touch down and touch up events is straightforward - we could use a button. However a keyboard presents a special challenge in that we need the user to be able to slide their finger up the keyboard activating and de-activating as it goes.

Attempt 1: Create subclass of UIImageView

I started off by sub-classing the UIImageView and trying to handle touches by overriding the: touchesBegan, touchesMoved etc... methods. This works for touch down and touch up events but doesn't work when the user slides their finger up the keyboard. I hoped that when the user slid their finger from one UIImageView to the next, the touches moved event would start being triggered on the new UIImageView. Unfortunately it doesn't.

Attempt 2: Using buttons

Next I decided to try using buttons. From the API it looks like buttons have useful functions which inform you when certain events like: touch drag enter or touch drag exit happen. This looked perfect!

  1. UIButton * button = [[UIButton alloc] init];
  2. [button addTarget:self action:@selector(touchDragEnter) forControlEvents:UIControlEventTouchDragEnter];

With this information we could disable the key when a "touch drag exit" is recorded and enable it when a "touch drag enter" is recorded. Unfortunately, this suffers from the same problem as before. If you start a drag on a button, when your finger leaves the button the drag exit event is triggered. So far so good. Unfortunately, the touch event seems to be isolated to that button, this means that the drag enter event is never called on the next button.

Attempt 3: Using a super view

Finally, I tried putting the controller code in the view which contained the keyboard key images. Here it's necessary to override the same touch methods:

  1. -(void) updateTouches: (NSSet *)touches withEvent:(UIEvent *)event {
  2. // Loop over the black keys
  3. CGPoint point;
  4. BOOL inside;
  5.  
  6. for (UIImageView * key in _blackKeys) {
  7.  
  8. // Loop over the touches
  9. for (UITouch * touch in [event allTouches]) {
  10.  
  11. // Find the point touched in the key
  12. point = [touch locationInView:key];
  13. // Check whether the point is inside the key
  14. inside = [key pointInside:point withEvent:event];
  15.  
  16. if (inside) {
  17. // If the touch is inside the key activate the key
  18. // otherwise de-activate it
  19. }
  20. }
  21. }
  22. // Do the same for the white keys

This function should be called from the overridden: touchesBegan and touchesMoved methods. Then a slightly different approach needs to be taken in the touchesEnded function. In the touches ended function a positive touch inside the key should de-activate the key. Using this approach, its possible to slide your finger over the keyboard and the keys activate and de-activate correctly. It also supports multi-touch which means you can play up to 10 notes simultaneously!

A video of the final working keyboard can be found here.

Overall this keyboard took about 3 days to get fully working. The final result is light weight, high performance and very flexible. If you want to save yourself some time, the keyboard plugin is available here.

Tweet: 

Add new comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • You can enable syntax highlighting of source code with the following tags: <code>, <blockcode>, <c>, <cpp>, <drupal5>, <drupal6>, <java>, <javascript>, <php>, <python>, <ruby>. The supported tag styles are: <foo>, [foo].
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.