Confused about Mixer output

[bjoerngiesler]

I'm trying to hook a mixer up to a volume stream. The final application I have in mind is something that mixes several sound sources (an audio file player, a signal generator, and others) into one output, and that output getting a final "master" level control. However I'm not clear on how to do this. Here's (part of) my code:

//#define USE_SETSTREAM
#ifdef USE_SETSTREAM // This dumps core
    sineGen_.begin(info_, N_B5);
    squareGen_.begin(info_, N_B5);
    sawtoothGen_.begin(info_, N_B5);

    sigGenStream_.setInput(sineGen_);
    sigGenStream_.begin(info_);

    sigGenVolume_.setStream(sigGenStream_);
    sigGenVolume_.begin(info_);

    mixer_.setOutputCount(NUM_MIXER_CHANNELS); // currently 1
    mixer_.begin(AUDIO_BUFFER_SIZE);
    mixerCopiers_[0].begin(mixer_, sigGenVolume_);

    mixerVolume_.setStream(mixer_);
    mixerVolume_.begin(info_);
#else // This doesn't, but logs a NPE in the mixerVolume_ stream
    sineGen_.begin(info_, N_B5);
    squareGen_.begin(info_, N_B5);
    sawtoothGen_.begin(info_, N_B5);

    sigGenStream_.setInput(sineGen_);
    sigGenStream_.begin(info_);

    sigGenVolume_.setStream(sigGenStream_);
    sigGenVolume_.begin(info_);

    mixer_.setOutputCount(NUM_MIXER_CHANNELS);
    mixer_.begin(AUDIO_BUFFER_SIZE);

    mixerCopiers_[0].begin(mixer_, sigGenVolume_);

    mixer_.setOutput(mixerVolume_);
    mixerVolume_.begin(info_);
#endif

In the first bit, I thought the same semantics that apply for the relation between sigGenStream_ and sigGenVolume_ might apply also for the relation between mixer_ and mixerVolume_, so mixerVolume_.setStream() might be the right way. This however dumps core. The second bit doesn't dump core, and it's in line with the Mixer examples (where the output stream is given to the mixer's constructor, mirrored here by mixer_.setOutput(). But this makes the VolumeStream log a null pointer exception as a warning. Looking into the code, that is caused by p_inp being nullptr. But setting it to the mixer using setStream causes the same coredump as in the first block above. What to do?

Edit: Since OutputMixer is not a Stream but a Print, I can't use a StreamCopy to copy from it to the mixerVolume_ either. TBH I find this all rather confusing...

Edit once more: Should I be using an InputMixer instead? With that, it seems to work.

[pschatzmann]

It is pretty diffcult to understand what you are doing if you omitt the variable definitions. If some code paths are crashing, analyse the backtrace: this will give you the root cause!

In general, I recommend to build the processing chain with the help of the constructors: this is easier to read and makes sure that the relevant output is define before you call begin.

Here is an example that is pretty close with what you are trying to implement....

ps. Instead of using a separate Volume for each GeneratedSoundStream, you could try to use the amplitude of the generator or just adjust the weights in the mixer: This should be more efficient

[bjoerngiesler]

Understood. I'm never sure how much code to post... the full code of the Player implementation is here: Player.h, Player.cpp. I have since converted it to use InputMixer instead of OutputMixer, which seems to work. The distinction wasn't clear to me, but it looks like InputMixer is the one to use here as it is a Stream and not just a Print.

I'm of course looking at the backtrace, but I'm using PlatformIO so I can't install the nifty backtrace analyzer tool... the textual part just states a Guru meditation error on Core 1.

I have studied your example before trying to implement mine, and you seem to be supplying the final output to the mixer. Since you're directly hooking up the mixer to the hardware driver, there's no VolumeStream in the loop, but the VolumeStream is giving me the NPE.

Apologies, still trying to figure this thing out. It's a fantastically powerful library but the mindset behind its construction takes a bit to understand.

I tried adjusting the mixer weights but if one of the channels has no data (eg a decoder channel fed from a file, that will stop eventually), that channel's weight is apparently not considered in norming, and the active channel gets maximum loudness again. Using the amplitude is a good idea, thank you!

[pschatzmann]

in platform.ino make sure you have:
monitor_filters = esp32_exception_decoder
You also might need to add
build_type = debug

Please note that when you call begin with audioInfo, the system will call setAudioInfo() on the dependent objects: so if you don't use the constructors to define the processing chain: either start at the end or call setStream/setOutput first on all objects, before calling begin.

Or you can switch the automatic notifications off by calling setNofiyActive(false) before calling begin.

If you find the root cause, I can add some additional checks...

[pschatzmann]

One final advice: first document your processing chain: this will show you what needs to be connected: e.g.

GeneratedAudioStream -> VolumeStream - copy-> OutputMixer -> VolumeStream -> I2SStream

This gives you the information what needs to be connected: e.g. the objects passed in the constructor:

• VolumeStream needs to read the data from GeneratedAudioStream
• OutputMixer needs to write the data to VolumeStream
• VolumeStream needs to write the data to I2SStream

[bjoerngiesler]

Omg. Thank you thank you thank you on the esp32_exception_decoder thing. I didn't know this existed and it's soooo helpful. I'll have more questions tomorrow as I'm trying to integrate an EncodedAudioStream with the mixer setup (it talks to the I2S stream directly just fine but doesn't work with the mixer), but I'll call it a day for now. So grateful for your help!