Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.
Now, I haven't tested the code, and some of the things you will hit playing with it are due to the difference between the version of Swift you are using and Swift 3. But I think this demonstrates more what I was going for with my suggestions. Take it or leave it as you see fit.

Awesome, thanks ! I already skimmed through your code, but I will definitely incorporate it into my code and test it out. I think that having concrete code to look at is better than us going back n forth with abstract ideas.

I don't know if you remember, but I did have some problems when I put playback code on the queue (i.e. outside the main thread). The calculated duration went to zero. I suspect some sort of thread-scoped isolation going on. As long as I access playerNode.sampleTime in the main thread, I get the correct value, but outside of the main thread, it's zero. I'll have to investigate this further.

Anyway, thanks again for all your input and your time and effort in writing this code, which I will put to use somehow (if not all of it, surely parts of it). I will let you know what happens.

BTW, I have to ask ... since you've actually cloned my repo and played with my code ... did you actually run my app ? ... I ask because I have not been able to test my app on any machines other than mine, and I'm curious about compatibility and how this performs on other systems ... esp. other MacOS versions (mine is Yosemite). If you tried running it, did it work at all ? This is my first time "distributing" a MacOS app, so I'm totally clueless as to portability challenges, even though this is native OS X code, after all. Any data you can provide would be appreciated.
 
Last edited:
Awesome, thanks ! I will definitely take a look at it. I think that having concrete code to look at is better than us going back n forth with abstract ideas.

I don't know if you remember, but I did have some problems when I put playback code on the queue (i.e. outside the main thread). The calculated duration went to zero. I suspect some sort of thread-scoped isolation going on. As long as I access playerNode.sampleTime in the main thread, I get the correct value, but outside of the main thread, it's zero. I'll have to investigate this further.

Yeah, so there are ways you can get around this, which is to dispatch to the main thread when you need to. For UI interactions, for example, it's pretty common to have a pattern to dispatch to a background thread to do some expensive task, and then dispatch back to the UI/main thread with the result. Because both the main queue and your buffer queue are both serial, you can dispatch back and forth relatively easily, as long as you don't reach across to modify data that the other queue is responsible to dealing with. For example, you cannot really do much with NSViews off the main thread, so it's common to dispatch to the main thread before updating them with progress from a background task.

Thinking a bit, I've had some more thoughts on the code (keep in mind, I'm new to this particular piece of framework, so I'm reading the docs as I go).
  • The completion handler of scheduleBuffer() should probably dispatch back into the buffer's dispatch queue to avoid parallel access to variables like currentBufferCount. AVAudioPlayerNode calls you back on a background thread that isn't the queue you created.
  • EventRegistry comes across as a stealth dependency. Not fun if you want to start unit testing.
  • I'm not entirely sure of how AVFoundation expects AVAudioPlayerNode to be used in terms of threading. Seems like some of it is thread safe, but other parts are not. I may actually just ensure that access to playerNode happens on the main thread.

Anyway, thanks again for all your input and your time and effort in writing this code, which I will put to use somehow (if not all of it, surely parts of it). I will let you know what happens.

BTW, I have to ask ... since you've actually cloned my repo and played with my code ... did you actually run my app ? ... I ask because I have not been able to test my app on any machines other than mine, and I'm curious about compatibility and how this performs on other systems ... esp. other MacOS versions (mine is Yosemite). If you tried running it, did it work at all ? This is my first time "distributing" a MacOS app, so I'm totally clueless as to portability challenges, even though this is native OS X code, after all. Any data you can provide would be appreciated.

Unfortunately, I don't have a ton of time to dig. The code I wrote up was done in maybe 10 minutes in a playground to point out the egregious errors. No guarantees it even compiles as-is.

I've been working on a side project of my own on top of AVFoundation (more the video side of things), and having "fun" dealing with the quirks of when playing and looping video. There's some annoying limitations there that I've had to spend time crafting answers to. And now I've been sucked into implementing drag and drop into the UI (it is an iOS app).
 
  • Like
Reactions: 0002378
Thanks for the additional tips.

No worries (about time to dig) ... this is just a hobby/educational project. Ultimately, the only person this app needs to please is me :D You've been very helpful.

I'm currently working on trying to add real-time visualization to this MP3 app (using the Accelerate framework to perform FFT). So far, it appears to be the most interesting challenge/learning experience yet !
 
Mark, interestingly, Instruments doesn't let me profile for energy with my platform being OS X. Did the developers of Instruments forget that Apple makes laptops that run OS X, that run on batteries ?

30 seconds of playback data takes up 10.09 MB vs 1.68 MB for a 5 second buffer. I don't know if that's a small difference. Maybe, in the grand scheme of things (assuming a modern machine with > 4GB of memory).

Wow, that's a big oversight. I wonder if that's still missing in Xcode 9. (I'm an iOS developer)

Ideally your app doesn't use more than a couple or so MB of RAM. It's really the 'working set' that you need to tune. So whatever memory that you're using to get real work done in your tightest loop or most time sensitive code, should fit in CPU cache, which varies but can be about 3 MB on some Intel CPUs.

So as long as your play buffer and whatever code uses it are under that then it's all good. You can see from memory paging (cache to RAM paging, not RAM to disk paging, which would be even worse).

Ideally you would not have a distinct play buffer and read ahead buffer, since that implies copying MBs of data from one to the other. As I think someone else here mentioned, you'd want to play directly from a portion of your read ahead buffer while reading ahead into other portions. This could be accomplished by using several buffers, at least two, but maybe 3 to 4 of the 5 second buffers. Or by having a single buffer and managing what ranges are being read from and written to.
 
  • Like
Reactions: 0002378
Wow, that's a big oversight. I wonder if that's still missing in Xcode 9. (I'm an iOS developer)

Apple historically does some odd things with Instruments. iOS and macOS have both been denied very useful tools that the other has gotten over the years.

Ideally you would not have a distinct play buffer and read ahead buffer, since that implies copying MBs of data from one to the other.

Thankfully, that's not how AVAudioPlayerNode seems to work. But if you do a big read ahead, and then schedule it, then you have to keep the entirety of the large buffer in memory until the player node is done with it. So that's a bit of a waste to use buffers that are too big that way too. Your comments about cache size are good though, since I totally forgot to consider that aspect. I'd probably double the number of buffers I queue, and halve the buffer length to get it under 2MB per buffer.
 
Wow, that's a big oversight. I wonder if that's still missing in Xcode 9. (I'm an iOS developer)

Ideally your app doesn't use more than a couple or so MB of RAM. It's really the 'working set' that you need to tune. So whatever memory that you're using to get real work done in your tightest loop or most time sensitive code, should fit in CPU cache, which varies but can be about 3 MB on some Intel CPUs.

So as long as your play buffer and whatever code uses it are under that then it's all good. You can see from memory paging (cache to RAM paging, not RAM to disk paging, which would be even worse).

Ideally you would not have a distinct play buffer and read ahead buffer, since that implies copying MBs of data from one to the other. As I think someone else here mentioned, you'd want to play directly from a portion of your read ahead buffer while reading ahead into other portions. This could be accomplished by using several buffers, at least two, but maybe 3 to 4 of the 5 second buffers. Or by having a single buffer and managing what ranges are being read from and written to.

Mark, thanks for the response.

I think that either I misunderstood your comment about buffering or you misunderstood how I'm scheduling the buffers. The two buffers I mentioned (play and lookahead) are totally mutually exclusive wrt their data. Each buffer is an atomic unit of playback (the first one containing seconds 0 - 5, the second one being 5 - 10, etc). While seconds 0-5 are being played, the 5-10 buffer is waiting in line on the queue, and the moment 0-5 is done, it schedules 10-15, while 5-10 starts playing. I'm not looking inside any of them or breaking them up into chunks. Both are scheduled directly on the playback queue, and there are no duplicate buffers and no copying from one to another.

In pseudocode, this is what goes on:

Code:
func play() {

// Put one buffer on the player queue for immediate playback
scheduleNextBuffer()

// Start playing file
player.play()

// Schedule another buffer on the playback queue (this is the lookahead)
scheduleNextBuffer()

// And, the completion handler of each player.scheduleBuffer() call takes care of scheduling the next one upon completion of its playback.

}


This first implementation might be somewhat simplistic. I've never written this kind of logic before, so it's really a first attempt at getting something working. I'm not sure what the best/standard practices are for this kind of buffer scheduling.

I will consider the tips you guys have mentioned and improve the code in iterations. Thanks for chipping in.
 
Last edited:
Mark, thanks for the response.

I think that either I misunderstood your comment about buffering or you misunderstood how I'm scheduling the buffers. The two buffers I mentioned (play and lookahead) are totally mutually exclusive wrt their data. Each buffer is an atomic unit of playback (the first one containing seconds 0 - 5, the second one being 5 - 10, etc). While seconds 0-5 are being played, the 5-10 buffer is waiting in line on the queue, and the moment 0-5 is done, it schedules 10-15, while 5-10 starts playing. I'm not looking inside any of them or breaking them up into chunks. Both are scheduled directly on the playback queue, and there are no duplicate buffers and no copying from one to another.

Ahh ok, gotcha. So while 0 - 5 seconds are playing, are you reading into the 5 - 10 second buffer from disk? When are the non-currently-playing buffers being filled-in?

For example, using 3 buffers of 5 seconds, and re-filling each buffer right after playing, to conservatively ensure audio ready, but doing disk I/O ungrouped (except in beginning):
Code:
Time in seconds ->
012345678901234567890123456789

READ 0 - 5
|
 READ 5 - 10
 |
  READ 10 - 15
  |

 PLAY 0 - 5
 |---|
       PLAY 5 - 10
       |---|
             PLAY 10 - 15
             |---|

       READ 15 - 20   (UN-GROUPED READS)
       |
             READ 20 - 25   (UN-GROUPED READS)
             |
                  READ 25 - 30   (UN-GROUPED READS)
                  |
For that, I'm not sure there's a point having more than two buffers, unless you're doing some extra processing on them.

Or, for example, always grouping reads, to maximize the disk powering down:
Code:
Time in seconds ->
012345678901234567890123456789

READ 0 - 5
|
 READ 5 - 10
 |
  READ 10 - 15
  |

 PLAY 0 - 5
 |---|
       PLAY 5 - 10
       |---|
             PLAY 10 - 15
             |---|

              READ 15 - 20    (GROUPED READS)
              |
               READ 20 - 25   (GROUPED READS)
               |
                  READ 25 - 30   (GROUPED READS)
                  |

                   PLAY 0 - 5
                   |---|
                         PLAY 5 - 10
                         |---|
                               PLAY 10 - 15
                               |---|

For the second approach, again there might as well just be two buffers, the key is that the last one refill right after it plays, but the other ones don't refill as soon as they can, instead they wait to refill once the last one has already started playing, and maybe not right at the beginning but a little bit in, depending how large it is.

(EDITED this following section, please re-read)
So now let's take this a step further. By using asymmetrically sized buffers, we can keep the simple algorithm of refilling a buffer right after it's played, but then the reads are grouped.
Code:
Time in seconds ->
012345678901234567890123456789

READ A  0 - 2
|
 PLAY A  0 - 2
 ||
 READ B  2 - 15
 |
   PLAY B  2 - 15
   |-----------|
   READ A  15 - 17    (GROUPED READ with start)
   |
                PLAY A  15 - 17
                ||
                READ B  17 - 30   (GROUPED READS)
                |
                  PLAY B  17 - 30
                  |-----------|
                  READ A  30 - 32  (GROUPED READS)
                  |

                               PLAY A  30 - 32
                               ||
                               READ B  32 - 45   (GROUPED READS)
                               |
                                 PLAY B  32 - 45
                                 |-----------|
                                 READ A  45 - 47  (GROUPED READS)
                                 |
 
Last edited:
Ahh ok, gotcha. So while 0 - 5 seconds are playing, are you reading into the 5 - 10 second buffer from disk? When are the non-currently-playing buffers being filled-in?

I think the best way to explain what I'm doing is to point you to my code. Otherwise, I think we'll keep going in circles. What I'm doing is actually quite simple, a nice consequence of being naive :)

(*** only if you have time ***)

https://github.com/maculateConception/aural-player/blob/master/Aural/BufferManager.swift

:)
 
Yeah you're evenly creating the buffers over time, but you're not reusing them but rather always creating new ones as you go. I don't know the API well enough but is there some way you can recycle AVAudioPCMBuffer objects once they're done playing?
 
Yeah you're evenly creating the buffers over time, but you're not reusing them but rather always creating new ones as you go. I don't know the API well enough but is there some way you can recycle AVAudioPCMBuffer objects once they're done playing?

That's probably a good idea. There doesn't seem to be any reason you can't, as long as you wait for the buffer to become unscheduled before refilling it. You could even use that to drive the buffering logic: allocate all the buffers you want at once (hopefully keeps the allocations near each other), and store them in a collection if they are ready to be filled.
 
Yeah you're evenly creating the buffers over time, but you're not reusing them but rather always creating new ones as you go. I don't know the API well enough but is there some way you can recycle AVAudioPCMBuffer objects once they're done playing?

That's probably a good idea. There doesn't seem to be any reason you can't, as long as you wait for the buffer to become unscheduled before refilling it. You could even use that to drive the buffering logic: allocate all the buffers you want at once (hopefully keeps the allocations near each other), and store them in a collection if they are ready to be filled.

1 - Thanks for the suggestion. Other than reducing the garbage collection (or Swift's equivalent) overhead and the one additional thing temporarily in memory (i.e. the buffer that just finished playing and is now waiting to be collected), is there another benefit to reusing buffers ?

Keeping the added complexity of your solution in mind (i.e the risk of re-playing old data which is still sitting in the same buffer), is the benefit worth it ?

Again, this is a balancing act. I don't doubt that your suggestion has benefits. I'm just wondering whether it is really worthwhile to reuse buffers, given the risk/complexity it comes with.

2 - A new question for both of you - I'm thinking of a way to potentially speed up playback start (i.e when a user clicks play or next/prev track) ... when the app loads up initially, I eagerly "prepare" the first track for playback, and whenever a track starts playing, I eagerly prepare the next track for playback, so that it is ready (to the extent possible) when the user clicks next or the previous one finishes playing. Of course, the prepping of the next track will be done asynchronously in a background thread, while the current track begins playing (or when the app initially loads up and nothing is playing yet). This "prepping" includes 1 - reading metadata like sampleRate and total frames, and 2 - possibly even reading that first small buffer of actual audio data ... to keep it ready for when the track is actually played. Of course, the assumption is that the user will not alter the playback sequence between the time the eager prepping is done and the prepped track actually starts playing. If he does, by double clicking a random track that hasn't been prepared yet, I'll just load it normally (just in time), but accounting for the most common case (where the app is just running in the background and the user is busy doing something else), it is fairly easy to figure out which track will play next and eagerly load some or all of the initial data required (1 - metadata, and 2 - the first small buffer of audio data), one track at a time. So, to summarize, I can always "look ahead" one track into the future, and keep it ready.

So, the question is - how would y'all do something like this ? Are there any potential downfalls to doing something like this ?

In pseudocode, I'm currently doing something like this:

Code:
func play(track) {

    player.play(track)

    dispatchAsyncToGlobalQueue {
  
        // Figure out which track is next in the current playback sequence, depending on repeat/shuffle mode
        let nextTrack = figureOutTheNextTrack()

        // Eagerly prepare it for playback
        prepare(nextTrack)
    }
}
 
Next track buffer is a good idea, although probably just 5 seconds.

As long as you can be told when the buffer is done being used, and reset it back to an empty state, then there's no risk.
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.