Like image sprites, sound sprites collect several distinct sounds into a single file, which reduces network lag while downloading.
This How-to shows how we applied that to speed up our very first HTML5 web game that used sounds, and shows unforeseen set-backs to consider.
When we built Trezjr, our HTML5 and Canvas implementation of the most successful video game, ever, we experimented with ways to play sounds (bleeps, tunes, and themes) when gamers pressed buttons (left, right, rotate, drop) or when the game executed certain actions (you won! You lost! Countdown to play!).
The onset of HTML5 brought us the HTMLAudioElement, and with it came a unified javacript API to load, play, and stop audio sources. We discovered several inconsistencies in browser support. This article shows actual html5 and javascript coding from our own solution, which has been proven to work.
First we have a section that allows the player to control whether sound is off or on. It sports an onChange event that is hard-coded rather than attached lazily, because in this web app we do not care about unobtrusive javascript: no javascript = no game.
Then we have a debug section that normally stays invisible to the player, but can be made visible for testing purposes. In that debug section we dropped our Audio Element, and a button to play its audio.
We needed that button because it allowed us to test whether the audio javascript API reacted to our game buttons appropriately. And we needed to attach events to it lazily (unobtrusively), because that is what we would be doing to our game buttons, too.
Note that we use just 1 HTMLAudioElement, and that the sources it references are various formats of 1 and the same sounds file. That is the file with sound sprites. We converted it to often-used and widely-supported audio formats. In our HTML coding, we first addressed the smallest file: the browser will download the first (and only the first) source file of which it supports the type. Thus we improve the download speed of the browsers that support the mime type of smallest audio file.
Note that we use our own Cross-browser Javascript Library, zjrJS for some actions, but you can safely use any other. Currently, the lodash library is gaining support.
We believe this script straightforward: create an HTMLAudioElement and test whether it has a member named “play”. If so, we assume audio is supported.
Why do we create a new audio element in-memory, rather than using the html-coded one? Because if our HTML developer had happened to include an attribute or a child node named “play”, we would have hit a false positive.
Mind you that we did not check whether “play” is a callable function (don't you hate prototype-changing scripts?) and whether or not it actually plays any source file (because that is tested by automated unit tests).
When we started coding this game, we were unsure on how many sound files we would need, so we created a system that had the potential of addressing and marking several. Thus, in our configuration singleton, we declared a soundData singleton that contains individual sound file singletons, which could, if they'd exist, contain a sprite declaration. Each sprite declares a time range which marks the begin and end of its audio sample in the source file, in seconds.
Note how we kept at least half a second of space in-between each sample? That is because some browsers are notoriously bad at stopping on time, causing the start of the next sample to be heard. Allowing a gap of half a second took away the symptom.
Note how we named each time range? This lets us play a sprite by name, using the following code:
Note that “zjrJS.Doc.eid(name)” loads the named element by ID in a backwards-compatible, cross-browser compatible manner. You are welcome to substitute these calls with any javascript library of your choosing.
Notice we approach this quite defensively: the desired audio must exist in the HTML, it must have a soundData configuration, the sprite must be defined, and if anything goes wrong too often, we automatically shut down audio support to keep game-play running smoothly.
Also notice that we did not, in fact, stop the sprites from playing. We do that automatically, by attaching an event listener to any declared audio files. And while we are at it, we should also make sure that the browser actually loads the declared source files: some mobile browsers refuse to load anything untill a user interacts, or untill we call the load() function specifically. The preload attribute is not heeded in those browsers.
We combine that into a single sound initialisation function:
The “zjrJS.Doc.aE()” method attaches events in a backwards-compatible, cross-browser compatible manner.
The actual function to automatically stop the sound sprites from playing looks like this:
Here we take advantage of zjrJS’ built-in method “zjrJS.Doc.eS(evt)” to recognise which HTML element is the source of the event, whether your player is using Microsoft’s, Google’s, Opera’s, Mozilla’s, Apple’s, or any other kind of web browser.
There are more methods and properties, but we did not need them for this purpose. Look them up in Mozilla’s great documentation for Using the Audio and Video Elements in HTML5
and we specified that cache manifest as follows:
Notice anything missing? We did not cache the audio source files! Why is that?
Well, that is because Apple made it impossible, in all its wisdom, to let Mobile Safari 5 play audio files from the browser’s cache. Whether a player is on-line or off-line does not matter: as soon as those audio sources get cached, Mobile Safari refuses to play them.
Unfortunately that means no sound for off-line players, which is why our playSound() function automatically shuts down audio support afer 5 failures.
If you find a way around that, or limit it to users of Apple’s mobile browser only, we are happy to hear from you!
The onset of HTML5 brought us the HTMLAudioElement, and with it came a unified javacript API to load, play, and stop audio sources. We discovered several inconsistencies in browser support. This article shows actual html5 and javascript coding from our own solution, which has been proven to work.
The HTML
<div><details>
<summary><h3>Options</h3></summary>
<p><label for="soundToggle"><input type=checkbox checked id=soundToggle onchange="Trezjr.UI.soundToggle(this)" /> Sound</label></p>
</details></div>
<div class=debug>
<audio id="soundSprites" preload="auto" controls>
<source src="m/trezjr-soundsprites.aac" type="audio/aac" />
<source src="m/trezjr-soundsprites.m4a" type="audio/m4a" />
<source src="m/trezjr-soundsprites.m4a" type="audio/mpeg" />
<source src="m/trezjr-soundsprites.ogg" type="audio/ogg" />
<source src="m/trezjr-soundsprites.wav" type="audio/wav" />
</audio>
<br />
<input type=reset class=btn id=playAudio value="Play Audio" />
</div>First we have a section that allows the player to control whether sound is off or on. It sports an onChange event that is hard-coded rather than attached lazily, because in this web app we do not care about unobtrusive javascript: no javascript = no game.
Then we have a debug section that normally stays invisible to the player, but can be made visible for testing purposes. In that debug section we dropped our Audio Element, and a button to play its audio.
We needed that button because it allowed us to test whether the audio javascript API reacted to our game buttons appropriately. And we needed to attach events to it lazily (unobtrusively), because that is what we would be doing to our game buttons, too.
Note that we use just 1 HTMLAudioElement, and that the sources it references are various formats of 1 and the same sounds file. That is the file with sound sprites. We converted it to often-used and widely-supported audio formats. In our HTML coding, we first addressed the smallest file: the browser will download the first (and only the first) source file of which it supports the type. Thus we improve the download speed of the browsers that support the mime type of smallest audio file.
Our Javascript
Testing whether the browser supports audio
Trezjr.UI.checkSoundSupport = function () {
"use strict";
var callee = "Trezjr.UI.checkSoundSupport()", check = null, supported = false;
try {
check = zjrJS.Doc.cE("audio");
if (typeof check !== "undefined") {
if ("play" in check) {
supported = true;
}
}
} catch (checkError) {
zjrJS.Debug.log(callee + " encountered this error: " + checkError.toString());
}
if (supported) {
if (Trezjr.Config.debug) {
zjrJS.Debug.log(callee + " determined sound is supported.");
}
} else {
zjrJS.Debug.log(callee + " determined lack of sound support.");
}
return supported;
};Note that we use our own Cross-browser Javascript Library, zjrJS for some actions, but you can safely use any other. Currently, the lodash library is gaining support.
We believe this script straightforward: create an HTMLAudioElement and test whether it has a member named “play”. If so, we assume audio is supported.
Why do we create a new audio element in-memory, rather than using the html-coded one? Because if our HTML developer had happened to include an attribute or a child node named “play”, we would have hit a false positive.
Mind you that we did not check whether “play” is a callable function (don't you hate prototype-changing scripts?) and whether or not it actually plays any source file (because that is tested by automated unit tests).
Marking the Starts and Ends of the Audio Samples
var Trezjr.Config = ({
/* other configuration */
soundData: ({
soundSprites: ({
sprites: ({
removedLine: [0, 0.45],
left: [1, 1.2],
right: [2, 2.2],
drop: [3, 3.5],
turnRight: [4, 4.35],
turnLeft: [5, 5.35],
newBlock: [6, 6.45],
gameOver: [7, 9]
})
})
})
});When we started coding this game, we were unsure on how many sound files we would need, so we created a system that had the potential of addressing and marking several. Thus, in our configuration singleton, we declared a soundData singleton that contains individual sound file singletons, which could, if they'd exist, contain a sprite declaration. Each sprite declares a time range which marks the begin and end of its audio sample in the source file, in seconds.
Note how we kept at least half a second of space in-between each sample? That is because some browsers are notoriously bad at stopping on time, causing the start of the next sample to be heard. Allowing a gap of half a second took away the symptom.
Note how we named each time range? This lets us play a sprite by name, using the following code:
Playing a Sound Sprite by Name
Trezjr.UI.playSound = function (name, sprite) {
"use strict";
var audio = null, d = null, noErrors = true, startPosition = 0, endPosition = 0, START = 0, END = 1;
if (Trezjr.UI.soundSupported && Trezjr.UI.soundInitiated) {
audio = zjrJS.Doc.eid(name);
if (audio) { try {
if (sprite === null) {
audio.endTime = (audio.duration) ? audio.duration : 9;
} else {
d = Trezjr.Config.soundData;
if (d) {
if (d[name]) {
if (d[name].sprites) {
if (d[name].sprites[sprite]) {
startPosition = d[name].sprites[sprite][START];
endPosition = d[name].sprites[sprite][END];
audio.currentTime = startPosition;
audio.endTime = endPosition;
}
}
}
}
}
try {
audio.play();
} catch (playError) {
zjrJS.Debug.log("Trezjr.UI.playSound() encountered error while trying to play '" + name + "': " + playError.toString());
noErrors = false;
Trezjr.UI.soundErrorAmount += 1;
if (Trezjr.UI.soundErrorAmount > 5) {
Trezjr.UI.soundSupported = false;
}
}
} catch (someError) {
zjrJS.Debug.log("Trezjr.UI.playSound() encountered an error: " + someError.toString());
noErrors = false;
Trezjr.UI.soundErrorAmount += 1;
if (Trezjr.UI.soundErrorAmount > 5) {
Trezjr.UI.soundSupported = false;
}
} }
}
audio = d = null;
return noErrors;
};Note that “zjrJS.Doc.eid(name)” loads the named element by ID in a backwards-compatible, cross-browser compatible manner. You are welcome to substitute these calls with any javascript library of your choosing.
Notice we approach this quite defensively: the desired audio must exist in the HTML, it must have a soundData configuration, the sprite must be defined, and if anything goes wrong too often, we automatically shut down audio support to keep game-play running smoothly.
Also notice that we did not, in fact, stop the sprites from playing. We do that automatically, by attaching an event listener to any declared audio files. And while we are at it, we should also make sure that the browser actually loads the declared source files: some mobile browsers refuse to load anything untill a user interacts, or untill we call the load() function specifically. The preload attribute is not heeded in those browsers.
We combine that into a single sound initialisation function:
Initialising Audio Elements
Trezjr.UI.initSounds = function () {
"use strict";
var audio = null, name = "", noErrors = true, ui = Trezjr.UI, soundData = Trezjr.Config.soundData;
Trezjr.UI.soundErrorAmount = 0;
for (name in soundData) {
try {
audio = zjrJS.Doc.eid(name);
if (audio) {
audio.load();
zjrJS.Doc.aE(audio, "timeupdate", Trezjr.UI.stopSoundAutomatically);
} else {
zjrJS.Debug.log("Trezjr.UI.initSounds() did not locate the audio element identified as '" + name + "'.");
noErrors = false;
}
} catch (initSoundError) {
zjrJS.Debug.log("Trezjr.UI.initSounds() encountered an error while adding sounds: " + initSoundError.toString());
noErrors = false;
}
}
audio = name = ui = soundData = null;
return noErrors;
};The “zjrJS.Doc.aE()” method attaches events in a backwards-compatible, cross-browser compatible manner.
The actual function to automatically stop the sound sprites from playing looks like this:
Automatically Stopping a Sound Sprite from Playing
Trezjr.UI.stopSoundAutomatically = function (evt) {
"use strict";
var audio = zjrJS.Doc.eS(evt);
if (audio.endTime) {
if (audio.currentTime >= audio.endTime) {
audio.pause();
}
}
};Here we take advantage of zjrJS’ built-in method “zjrJS.Doc.eS(evt)” to recognise which HTML element is the source of the event, whether your player is using Microsoft’s, Google’s, Opera’s, Mozilla’s, Apple’s, or any other kind of web browser.
TL;DR: The Audio Javascript API
If you have read this far: thank you! Now you should be familiar with the API for audio files:audio.load()audio.play()audio.pause()There are more methods and properties, but we did not need them for this purpose. Look them up in Mozilla’s great documentation for Using the Audio and Video Elements in HTML5
A Word about Caching
We like to cache our web apps and web games in the users’s browsers so they can play them while off-line. For that purpose, we declared a cache manifest in our HTML:<html lang="en" manifest="game-cache.37.manifest">and we specified that cache manifest as follows:
CACHE MANIFEST
# For Trezjr, Version 37
# https://omegajunior.globat.com/code/trezjr/
# Contact omegajunior@protonmail.com
CACHE:
../zjrjs/zjrJS.20121110t0214.js
../Zjramble5/fb.18.js
channel.html
game.36.js
style.9.css
im1/trezjr-icon-294x294.png
im1/trezjr-icon-156x156.png
im1/trezjr-icon-114x114.png
im1/trezjr-icon-72x72.png
im1/trezjr-icon-57x57.png
NETWORK:
*Notice anything missing? We did not cache the audio source files! Why is that?
Well, that is because Apple made it impossible, in all its wisdom, to let Mobile Safari 5 play audio files from the browser’s cache. Whether a player is on-line or off-line does not matter: as soon as those audio sources get cached, Mobile Safari refuses to play them.
Unfortunately that means no sound for off-line players, which is why our playSound() function automatically shuts down audio support afer 5 failures.
If you find a way around that, or limit it to users of Apple’s mobile browser only, we are happy to hear from you!
Need problem solving?
Talk to me. Let's meet for coffee or over lunch. Mail me at “omegajunior at protonmail dot com”.