There is no doubt that for the vast majority of professional audio practitioners, a computer of some kind is at the centre of our studio. This has largely been focused around a conventional computer, with a CPU, mouse, keyboard and screen, with most of the human interaction focussed around the mouse and keyboard. But will it always be like that? Perhaps not?
We have been looking at some recent successful patent applications to see how computer developments could change the way we relate to the computer in the studio.
For example, imagine a laptop with two screens, one in the conventional location with the second replacing the keyboard and trackpad and filling the entire surface of the lower section.
The US Patent and Trademark Office has finally granted a patent to Apple for a dual-display MacBook with a virtual keyboard replacing the traditional keyboard. What no keyboard?
Yes, That’s right, no physical keyboard. Instead, the lower section would have a virtual keyboard for normal use, but because it’s no longer a real keyboard, this offers the opportunity for a much wider range of input devices than the conventional QWERTY keyboard.
The patent, which has taken 3 years to be approved, shows examples that include dedicated button arrays, the option for a rest-on rotatable dial as well as more conventional laptop things like a fingerprint sensor, Touch ID and wireless recharging.
In the patent Apple suggests…
“because the images of the keys of the virtual keyboard are produced by a display below the top case, different keyboard layouts could be supported like in FIG. 16B using an ergonomic keyboard layout. As another example, we’re able to see in patent FIG. 16C above that dual-display based notebook could use the virtual keyboard area and use another available user interface that could double as a game controller.”
This could allow the virtual keyboard to be rearranged at will perhaps swapping the trackpad and keyboard, different language keyboards available at the click of a virtual button, as well as dedicated controllers for specific applications.
Imagine a virtual keyboard with a dedicated Pro Tools virtual controller layout giving you direct shortcuts to a bit like the options on the Avid Control app.
However, for those who prefer real buttons to press or real faders to push, the idea of using virtual controls on a glass screen doesn’t really deliver. But could it, in the future?
Haptic Feedback - Feel The Buttons And Faders
It would appear that Apple has considered this. In a number of patents, Apple has been covering haptic feedback on relatively conventional laptops, including making a virtual trackpad that would feel like you were clicking a real button.
However, in a patent published in August 2021, Apple appears to have moved what is possible with haptic feedback by designing zoned haptic feedback behind a normal display. This opens up all kinds of possibilities.
In this latest Haptic Feedback patent, Apple notes that haptics using vibratory motors typically vibrate the entire electronic device and are not able to provide a haptic output for a specific localised area. Apple's patent application reveals that in the future, haptics could be found in very specific areas of a device and only vibrate that area, such as behind a single app icon, keyboard key, on a MacBook touch display or a device casing such as a Magic Mouse
Consider the virtual keyboard described earlier in this article. With zoned haptic feedback it wouldn’t feel like typing on glass, as it does currently when you use the virtual keyboard on a tablet. With this design, it would feel like you were typing on a real keyboard because you would feel each of the keys depress.
Apple goes on to discuss further options for haptic feedback that include a beam-type structure within a surface or some other component. The beam structure would be coupled to a piezoelectric element and could deflect in different directions, depending on the current, voltage or other input signals that are applied to the piezoelectric element.
As the beam structure deflects, this causes a haptic output that creates a tactile sensation. In some embodiments, the haptic output may be provided to an input surface (e.g., a surface, structure, designed to receive an input from a user). This could offer the option to not only touch and control the faders on your DAW from the screen but more importantly ‘feel’ the fader as you move it.
To date, Apple has not offered touch displays on their laptop, but it is difficult to see how this technology if implemented, would not be used on their laptop range, so maybe we see an Apple laptop with touch displays with virtual controller interfaces with haptic feedback. Surely that would warrant the often overused phrase as being a ‘game-changer’?
Possible But Not Yet
Now before you get too excited, all these patents show is that Apple has undertaken a proof of concept, sufficient to be able to submit a patent based on what they have developed. It is not a statement of intent fr a future product. We have no way of knowing when or even if this will ever materialise on an actual product, but it does give us a sense of what might be possible.
Would this be a direction of travel that you would find interesting?