Commit 3b1a09d0 authored by crogers@google.com's avatar crogers@google.com

Add MediaPlayer::audioSourceProvider() method for audio stream access by the Web Audio API.

https://bugs.webkit.org/show_bug.cgi?id=66398

Reviewed by Eric Carlson.

Source/WebCore:

* html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::HTMLMediaElement):
(WebCore::HTMLMediaElement::prepareForLoad):
(WebCore::HTMLMediaElement::loadNextSourceChild):
(WebCore::HTMLMediaElement::ensureMediaPlayer):
(WebCore::HTMLMediaElement::createMediaPlayer):
    MediaPlayer creation has now been put into createMediaPlayer() instead of it being created
    separately in these various methods.  It has been made thread-safe for use with the Web Audio API
    in conjunction with an MediaElementAudioSourceNode.
(WebCore::HTMLMediaElement::setAudioSourceNode):
(WebCore::HTMLMediaElement::audioSourceProvider):
* html/HTMLMediaElement.h:
(WebCore::HTMLMediaElement::audioSourceNode):
    The HTMLMediaElement now keeps track of an optional MediaElementAudioSourceNode if
    the audio stream is being processed using the Web Audio API.

* platform/graphics/MediaPlayer.cpp:
(WebCore::MediaPlayer::audioSourceProvider):
* platform/graphics/MediaPlayer.h:
* platform/graphics/MediaPlayerPrivate.h:
(WebCore::MediaPlayerPrivateInterface::audioSourceProvider):
    MediaPlayer now has an audioSourceProvider() method with a default "empty" implementation.
    audioSourceProvider() returns an AudioSourceProvider object which the Web Audio API's
    MediaElementAudioSourceNode uses (indirectly through an HTMLMediaElement method) to get the rendered audio stream.

* webaudio/AudioContext.cpp:
(WebCore::AudioContext::createMediaElementSource):
* webaudio/AudioContext.h:
* webaudio/AudioContext.idl:
    AudioContext::createMediaElementSource() is now more careful to check that an HTMLMediaElement
    doesn't already have a MediaElementAudioSourceNode attached to it.

* webaudio/MediaElementAudioSourceNode.cpp:
(WebCore::MediaElementAudioSourceNode::MediaElementAudioSourceNode):
(WebCore::MediaElementAudioSourceNode::~MediaElementAudioSourceNode):
(WebCore::MediaElementAudioSourceNode::process):
(WebCore::MediaElementAudioSourceNode::lock):
(WebCore::MediaElementAudioSourceNode::unlock):
* webaudio/MediaElementAudioSourceNode.h:
    Implement thread-safe processing (replacing the old stub implementation).

LayoutTests:

* webaudio/mediaelementaudiosourcenode-expected.txt:
* webaudio/mediaelementaudiosourcenode.html:

git-svn-id: http://svn.webkit.org/repository/webkit/trunk@93903 268f45cc-cd09-0410-ab3c-d52691b4dbfc
parent ab6cca3f
2011-08-26 Chris Rogers <crogers@google.com>
Add MediaPlayer::audioSourceProvider() method for audio stream access by the Web Audio API.
https://bugs.webkit.org/show_bug.cgi?id=66398
Reviewed by Eric Carlson.
* webaudio/mediaelementaudiosourcenode-expected.txt:
* webaudio/mediaelementaudiosourcenode.html:
2011-08-23 Stephen White <senorblanco@chromium.org>
Assertion fires if canvas is resized while save() active
......@@ -8,6 +8,7 @@ PASS connect() exception thrown for illegal destination AudioNode.
PASS connect() exception thrown for illegal output index.
PASS connect() exception thrown for illegal input index.
PASS audioNode.connect(context.destination) succeeded.
PASS createMediaElementSource() threw error when called twice on same HTMLMediaElement.
PASS successfullyParsed is true
TEST COMPLETE
......
......@@ -70,6 +70,14 @@ function runTest() {
} catch(e) {
testFailed("audioNode.connect(context.destination) failed.");
}
// Try creating another MediaElementAudioSourceNode using the same audio element.
try {
mediaSource = context.createMediaElementSource(audioElement);
testFailed("createMediaElementSource() should throw if called twice on same HTMLMediaElement.");
} catch(e) {
testPassed("createMediaElementSource() threw error when called twice on same HTMLMediaElement.");
}
finishJSTest();
}
......
2011-08-26 Chris Rogers <crogers@google.com>
Add MediaPlayer::audioSourceProvider() method for audio stream access by the Web Audio API.
https://bugs.webkit.org/show_bug.cgi?id=66398
Reviewed by Eric Carlson.
* html/HTMLMediaElement.cpp:
(WebCore::HTMLMediaElement::HTMLMediaElement):
(WebCore::HTMLMediaElement::prepareForLoad):
(WebCore::HTMLMediaElement::loadNextSourceChild):
(WebCore::HTMLMediaElement::ensureMediaPlayer):
(WebCore::HTMLMediaElement::createMediaPlayer):
MediaPlayer creation has now been put into createMediaPlayer() instead of it being created
separately in these various methods. It has been made thread-safe for use with the Web Audio API
in conjunction with an MediaElementAudioSourceNode.
(WebCore::HTMLMediaElement::setAudioSourceNode):
(WebCore::HTMLMediaElement::audioSourceProvider):
* html/HTMLMediaElement.h:
(WebCore::HTMLMediaElement::audioSourceNode):
The HTMLMediaElement now keeps track of an optional MediaElementAudioSourceNode if
the audio stream is being processed using the Web Audio API.
* platform/graphics/MediaPlayer.cpp:
(WebCore::MediaPlayer::audioSourceProvider):
* platform/graphics/MediaPlayer.h:
* platform/graphics/MediaPlayerPrivate.h:
(WebCore::MediaPlayerPrivateInterface::audioSourceProvider):
MediaPlayer now has an audioSourceProvider() method with a default "empty" implementation.
audioSourceProvider() returns an AudioSourceProvider object which the Web Audio API's
MediaElementAudioSourceNode uses (indirectly through an HTMLMediaElement method) to get the rendered audio stream.
* webaudio/AudioContext.cpp:
(WebCore::AudioContext::createMediaElementSource):
* webaudio/AudioContext.h:
* webaudio/AudioContext.idl:
AudioContext::createMediaElementSource() is now more careful to check that an HTMLMediaElement
doesn't already have a MediaElementAudioSourceNode attached to it.
* webaudio/MediaElementAudioSourceNode.cpp:
(WebCore::MediaElementAudioSourceNode::MediaElementAudioSourceNode):
(WebCore::MediaElementAudioSourceNode::~MediaElementAudioSourceNode):
(WebCore::MediaElementAudioSourceNode::process):
(WebCore::MediaElementAudioSourceNode::lock):
(WebCore::MediaElementAudioSourceNode::unlock):
* webaudio/MediaElementAudioSourceNode.h:
Implement thread-safe processing (replacing the old stub implementation).
2011-08-23 Stephen White <senorblanco@chromium.org>
Assertion fires if canvas is resized while save() active
......@@ -88,6 +88,10 @@
#include "HTMLTrackElement.h"
#endif
#if ENABLE(WEB_AUDIO)
#include "MediaElementAudioSourceNode.h"
#endif
using namespace std;
namespace WebCore {
......@@ -179,6 +183,9 @@ HTMLMediaElement::HTMLMediaElement(const QualifiedName& tagName, Document* docum
, m_loadInitiatedByUserGesture(false)
, m_completelyLoaded(false)
, m_havePreparedToPlay(false)
#if ENABLE(WEB_AUDIO)
, m_audioSourceNode(0)
#endif
{
LOG(Media, "HTMLMediaElement::HTMLMediaElement");
document->registerForDocumentActivationCallbacks(this);
......@@ -531,7 +538,7 @@ void HTMLMediaElement::prepareForLoad()
scheduleEvent(eventNames().abortEvent);
#if !ENABLE(PLUGIN_PROXY_FOR_VIDEO)
m_player = MediaPlayer::create(this);
createMediaPlayer();
#else
if (m_player)
m_player->cancelLoad();
......@@ -682,7 +689,7 @@ void HTMLMediaElement::loadNextSourceChild()
#if !ENABLE(PLUGIN_PROXY_FOR_VIDEO)
// Recreate the media player for the new url
m_player = MediaPlayer::create(this);
createMediaPlayer();
#endif
m_loadState = LoadingFromSourceElement;
......@@ -2467,7 +2474,7 @@ void HTMLMediaElement::defaultEventHandler(Event* event)
void HTMLMediaElement::ensureMediaPlayer()
{
if (!m_player)
m_player = MediaPlayer::create(this);
createMediaPlayer();
}
void HTMLMediaElement::deliverNotification(MediaPlayerProxyNotificationType notification)
......@@ -2807,6 +2814,37 @@ void* HTMLMediaElement::preDispatchEventHandler(Event* event)
return 0;
}
void HTMLMediaElement::createMediaPlayer()
{
#if ENABLE(WEB_AUDIO)
if (m_audioSourceNode)
m_audioSourceNode->lock();
#endif
m_player = MediaPlayer::create(this);
#if ENABLE(WEB_AUDIO)
if (m_audioSourceNode)
m_audioSourceNode->unlock();
#endif
}
#if ENABLE(WEB_AUDIO)
void HTMLMediaElement::setAudioSourceNode(MediaElementAudioSourceNode* sourceNode)
{
ASSERT(!m_audioSourceNode);
m_audioSourceNode = sourceNode;
}
AudioSourceProvider* HTMLMediaElement::audioSourceProvider()
{
if (m_player)
return m_player->audioSourceProvider();
return 0;
}
#endif
}
#endif
......@@ -39,6 +39,10 @@
namespace WebCore {
#if ENABLE(WEB_AUDIO)
class AudioSourceProvider;
class MediaElementAudioSourceNode;
#endif
class Event;
class HTMLSourceElement;
class MediaControls;
......@@ -203,6 +207,13 @@ public:
bool isPlaying() const { return m_playing; }
#if ENABLE(WEB_AUDIO)
MediaElementAudioSourceNode* audioSourceNode() { return m_audioSourceNode; }
void setAudioSourceNode(MediaElementAudioSourceNode*);
AudioSourceProvider* audioSourceProvider();
#endif
protected:
HTMLMediaElement(const QualifiedName&, Document*);
virtual ~HTMLMediaElement();
......@@ -221,6 +232,8 @@ protected:
virtual bool isMediaElement() const { return true; }
private:
void createMediaPlayer();
virtual void attributeChanged(Attribute*, bool preserveDecls);
virtual bool rendererIsNeeded(const NodeRenderingContext&);
virtual RenderObject* createRenderer(RenderArena*, RenderStyle*);
......@@ -435,6 +448,13 @@ private:
bool m_loadInitiatedByUserGesture : 1;
bool m_completelyLoaded : 1;
bool m_havePreparedToPlay : 1;
#if ENABLE(WEB_AUDIO)
// This is a weak reference, since m_audioSourceNode holds a reference to us.
// The value is set just after the MediaElementAudioSourceNode is created.
// The value is cleared in MediaElementAudioSourceNode::~MediaElementAudioSourceNode().
MediaElementAudioSourceNode* m_audioSourceNode;
#endif
};
} //namespace
......
......@@ -873,6 +873,13 @@ void MediaPlayer::characteristicChanged()
m_mediaPlayerClient->mediaPlayerCharacteristicChanged(this);
}
#if ENABLE(WEB_AUDIO)
AudioSourceProvider* MediaPlayer::audioSourceProvider()
{
return m_private->audioSourceProvider();
}
#endif // WEB_AUDIO
}
#endif
......@@ -41,6 +41,10 @@
#include <wtf/PassOwnPtr.h>
#include <wtf/text/StringHash.h>
#if ENABLE(WEB_AUDIO)
#include "AudioSourceProvider.h"
#endif
#if USE(ACCELERATED_COMPOSITING)
#include "GraphicsLayer.h"
#endif
......@@ -309,6 +313,10 @@ public:
void setPrivateBrowsingMode(bool);
#if ENABLE(WEB_AUDIO)
AudioSourceProvider* audioSourceProvider();
#endif
private:
MediaPlayer(MediaPlayerClient*);
void loadWithNextMediaEngine(MediaPlayerFactory*);
......
......@@ -146,6 +146,10 @@ public:
void clearMediaCacheForSite(const String&) { }
virtual void setPrivateBrowsingMode(bool) { }
#if ENABLE(WEB_AUDIO)
virtual AudioSourceProvider* audioSourceProvider() { return 0; }
#endif
};
}
......
......@@ -312,12 +312,27 @@ PassRefPtr<AudioBufferSourceNode> AudioContext::createBufferSource()
return node;
}
PassRefPtr<MediaElementAudioSourceNode> AudioContext::createMediaElementSource(HTMLMediaElement* mediaElement)
PassRefPtr<MediaElementAudioSourceNode> AudioContext::createMediaElementSource(HTMLMediaElement* mediaElement, ExceptionCode& ec)
{
ASSERT(mediaElement);
if (!mediaElement) {
ec = INVALID_STATE_ERR;
return 0;
}
ASSERT(isMainThread());
lazyInitialize();
// First check if this media element already has a source node.
if (mediaElement->audioSourceNode()) {
ec = INVALID_STATE_ERR;
return 0;
}
RefPtr<MediaElementAudioSourceNode> node = MediaElementAudioSourceNode::create(this, mediaElement);
mediaElement->setAudioSourceNode(node.get());
refNode(node.get()); // context keeps reference until node is disconnected
return node;
}
......
......@@ -110,7 +110,7 @@ public:
// The AudioNode create methods are called on the main thread (from JavaScript).
PassRefPtr<AudioBufferSourceNode> createBufferSource();
PassRefPtr<MediaElementAudioSourceNode> createMediaElementSource(HTMLMediaElement*);
PassRefPtr<MediaElementAudioSourceNode> createMediaElementSource(HTMLMediaElement*, ExceptionCode&);
PassRefPtr<AudioGainNode> createGainNode();
PassRefPtr<BiquadFilterNode> createBiquadFilter();
PassRefPtr<WaveShaperNode> createWaveShaper();
......
......@@ -59,7 +59,8 @@ module webaudio {
// Sources
AudioBufferSourceNode createBufferSource();
MediaElementAudioSourceNode createMediaElementSource(in HTMLMediaElement mediaElement);
MediaElementAudioSourceNode createMediaElementSource(in HTMLMediaElement mediaElement)
raises(DOMException);
// Processing nodes
AudioGainNode createGainNode();
......
......@@ -30,6 +30,7 @@
#include "AudioContext.h"
#include "AudioNodeOutput.h"
#include "MediaPlayer.h"
namespace WebCore {
......@@ -46,20 +47,54 @@ MediaElementAudioSourceNode::MediaElementAudioSourceNode(AudioContext* context,
addOutput(adoptPtr(new AudioNodeOutput(this, 2)));
setType(NodeTypeMediaElementAudioSource);
initialize();
}
MediaElementAudioSourceNode::~MediaElementAudioSourceNode()
{
m_mediaElement->setAudioSourceNode(0);
uninitialize();
}
void MediaElementAudioSourceNode::process(size_t)
void MediaElementAudioSourceNode::process(size_t numberOfFrames)
{
AudioBus* outputBus = output(0)->bus();
// FIXME: implement MediaPlayer abstraction to get audio stream from <audio> and <video>
outputBus->zero();
if (!mediaElement()) {
outputBus->zero();
return;
}
// Use a tryLock() to avoid contention in the real-time audio thread.
// If we fail to acquire the lock then the HTMLMediaElement must be in the middle of
// reconfiguring its playback engine, so we output silence in this case.
if (m_processLock.tryLock()) {
if (AudioSourceProvider* provider = mediaElement()->audioSourceProvider())
provider->provideInput(outputBus, numberOfFrames);
else
outputBus->zero();
m_processLock.unlock();
} else
outputBus->zero();
}
void MediaElementAudioSourceNode::reset()
{
}
void MediaElementAudioSourceNode::lock()
{
ref();
m_processLock.lock();
}
void MediaElementAudioSourceNode::unlock()
{
m_processLock.unlock();
deref();
}
} // namespace WebCore
#endif // ENABLE(WEB_AUDIO)
......@@ -28,6 +28,7 @@
#include "AudioSourceNode.h"
#include "HTMLMediaElement.h"
#include <wtf/PassRefPtr.h>
#include <wtf/Threading.h>
namespace WebCore {
......@@ -37,16 +38,22 @@ class MediaElementAudioSourceNode : public AudioSourceNode {
public:
static PassRefPtr<MediaElementAudioSourceNode> create(AudioContext*, HTMLMediaElement*);
virtual ~MediaElementAudioSourceNode();
HTMLMediaElement* mediaElement() { return m_mediaElement.get(); }
// AudioNode
virtual void process(size_t framesToProcess);
virtual void reset();
void lock();
void unlock();
private:
MediaElementAudioSourceNode(AudioContext*, HTMLMediaElement*);
RefPtr<HTMLMediaElement> m_mediaElement;
Mutex m_processLock;
};
} // namespace WebCore
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment