FormantShifter Class Reference

#include <FormantShifter.h>

Collaboration diagram for FormantShifter:

Public Member Functions
	FormantShifter (int sampleRate, double cutoffQuefrency, FormantShifterLoggerInterface &logger)
void	Reset (size_t fftSize)
void	Reset ()
void	Process (const float *powerSpectrum, std::complex< float > *spectrum, double factor)
	Processes spectrum in place, or does nothing if Reset(fftSize) wasn't called or Reset() was called since. More...

Public Attributes
const double	cutoffQuefrency

Private Attributes
const int	mSampleRate
FormantShifterLoggerInterface &	mLogger
std::unique_ptr< staffpad::audio::FourierTransform >	mFft
staffpad::SamplesComplex	mEnvelope
staffpad::SamplesReal	mCepstrum
std::vector< float >	mEnvelopeReal
std::vector< float >	mWeights

Detailed Description

Definition at line 27 of file FormantShifter.h.

Constructor & Destructor Documentation

FormantShifter()

FormantShifter::FormantShifter	(	int	sampleRate,
		double	cutoffQuefrency,
		FormantShifterLoggerInterface &	logger
	)

Definition at line 43 of file FormantShifter.cpp.

    : cutoffQuefrency { cutoffQuefrency }
    , mSampleRate { sampleRate }
    , mLogger { logger }
{
}

Member Function Documentation

Process()

void FormantShifter::Process	(	const float *	powerSpectrum,
		std::complex< float > *	spectrum,
		double	factor
	)

Processes spectrum in place, or does nothing if Reset(fftSize) wasn't called or Reset() was called since.

"Shifts" the frequency-domain envelope of the input signal. typically used for formant preservation. Tuned to work best with voice.

Parameters

powerSpectrum	The power of spectrum, i.e., powerSpectrum[i] = norm(spectrum[i]), i.e., the square root was NOT taken.
spectrum	The complex spectrum of the input signal.
factor	The factor by which to scale the position of the formants on the frequency axis.

Precondition

powerSpectrum and spectrum are not null and have size fftSize / 2 + 1.

factor > 0.

Definition at line 68 of file FormantShifter.cpp.

{
   assert(factor > 0);
   if (factor <= 0 || cutoffQuefrency == 0 || !mFft)
      return;
 
   const auto fftSize = mFft->getSize();
   const auto numBins = fftSize / 2 + 1;
 
   mLogger.Log(fftSize, "fftSize");
 
   // Take the log of the normalized magnitude. (This assumes that
   // the window averages to 1.)
   std::complex<float>* pEnv = mEnvelope.getPtr(0);
   const float normalizer = FastLog2(fftSize);
   std::transform(powSpec, powSpec + numBins, pEnv, [&](float power) {
      return .5f * FastLog2(power) - normalizer;
   });
 
   // Get the cosine transform of the log magnitude, aka the cepstrum.
   mFft->inverseReal(mEnvelope, mCepstrum);
   auto pCepst = mCepstrum.getPtr(0);
   mLogger.Log(pCepst, fftSize, "cepstrum");
 
   // "Lifter" the cepstrum.
   const auto binCutoff = int(mSampleRate * cutoffQuefrency * factor);
   if (binCutoff < fftSize / 2)
      std::fill(pCepst + binCutoff + 1, pCepst + fftSize - binCutoff, 0.f);
   mLogger.Log(pCepst, fftSize, "cepstrumLiftered");
 
   // Get the envelope back.
   mFft->forwardReal(mCepstrum, mEnvelope);
   std::transform(
      pEnv, pEnv + numBins, mEnvelopeReal.begin(),
      [fftSize = fftSize](const std::complex<float>& env) {
         return std::exp2(env.real() / fftSize);
      });
   mLogger.Log(mEnvelopeReal.data(), numBins, "envelope");
 
   // Get the weights, which are the ratio of the desired envelope to the
   // current envelope (which has the effect of downsampling).
   std::transform(
      mEnvelopeReal.begin(), mEnvelopeReal.end(), mWeights.begin(),
      [](float env) { return std::isnormal(env) ? 1.f / env : 0.f; });
 
   const auto lastNonZeroedBin =
      ResampleFreqDomain(mEnvelopeReal.data(), fftSize, factor);
 
   mLogger.Log(mEnvelopeReal.data(), numBins, "envelopeResampled");
   std::transform(
      mEnvelopeReal.begin(), mEnvelopeReal.end(), mWeights.begin(),
      mWeights.begin(), [](float env, float weight) {
         // Limit the weights to 100, which corresponds to 20dB.
         // Our purpose is to add (or remove) energy to formants, and it doesn't
         // need to be by more than that. This way we also avoid unreasonable
         // amplification.
         return std::min(env * weight, 100.f);
      });
 
   // Say the signal was downsampled to pitch it up. The factor is then less
   // than 1, and the resampler had to zero out the upper part of the envelope
   // bins. For these, rather than zeroing the spec too, it sounds better
   // to keep the original, even if no envelope correction is applied, else the
   // signal looses a bit of clarity. At such high frequencies, we probably
   // don't need a smooth frequency-domain transition and a jump is fine. (This
   // is visible in the spec, in case you're curious.)
   std::fill(mWeights.begin() + lastNonZeroedBin, mWeights.end(), 1.f);
 
   mLogger.Log(mWeights.data(), mWeights.size(), "weights");
 
   mLogger.Log(
      spec, numBins, "magnitude",
      [fftSize = fftSize](const std::complex<float>& spec) {
         return std::abs(spec) / fftSize;
      });
 
   // Now apply the weights.
   std::transform(
      spec, spec + numBins, mWeights.begin(), spec,
      std::multiplies<std::complex<float>>());
 
   mLogger.Log(
      spec, numBins, "weightedMagnitude",
      [fftSize = fftSize](const std::complex<float>& spec) {
         return std::abs(spec) / fftSize;
      });
 
   mLogger.ProcessFinished(spec, fftSize);
}

References cutoffQuefrency, FastLog2(), staffpad::SamplesFloat< T >::getPtr(), FormantShifterLoggerInterface::Log(), mCepstrum, mEnvelope, mEnvelopeReal, mFft, mLogger, mSampleRate, mWeights, fast_float::detail::power(), FormantShifterLoggerInterface::ProcessFinished(), and anonymous_namespace{FormantShifter.cpp}::ResampleFreqDomain().

Referenced by anonymous_namespace{StaffPadTimeAndPitch.cpp}::CreateTimeAndPitch().

Here is the call graph for this function:

Here is the caller graph for this function:

Reset() [1/2]

void FormantShifter::Reset

(

)

Definition at line 63 of file FormantShifter.cpp.

{
   mFft.reset();
}

References mFft.

Reset() [2/2]

void FormantShifter::Reset

(

size_t

fftSize

)

Definition at line 52 of file FormantShifter.cpp.

{
   mFft = std::make_unique<staffpad::audio::FourierTransform>(
      static_cast<int32_t>(fftSize));
   const auto numBins = fftSize / 2 + 1;
   mEnvelope.setSize(1, numBins);
   mCepstrum.setSize(1, fftSize);
   mEnvelopeReal.resize(numBins);
   mWeights.resize(numBins);
}

References mCepstrum, mEnvelope, mEnvelopeReal, mFft, mWeights, and staffpad::SamplesFloat< T >::setSize().

Referenced by StaffPadTimeAndPitch::OnFormantPreservationChange(), and StaffPadTimeAndPitch::StaffPadTimeAndPitch().

Here is the call graph for this function:

Here is the caller graph for this function:

Member Data Documentation

cutoffQuefrency

const double FormantShifter::cutoffQuefrency

Definition at line 30 of file FormantShifter.h.

Referenced by Process().

mCepstrum

staffpad::SamplesReal FormantShifter::mCepstrum

private

Definition at line 64 of file FormantShifter.h.

Referenced by Process(), and Reset().

mEnvelope

staffpad::SamplesComplex FormantShifter::mEnvelope

private

Definition at line 63 of file FormantShifter.h.

Referenced by Process(), and Reset().

mEnvelopeReal

std::vector<float> FormantShifter::mEnvelopeReal

private

Definition at line 65 of file FormantShifter.h.

Referenced by Process(), and Reset().

mFft

std::unique_ptr<staffpad::audio::FourierTransform> FormantShifter::mFft

private

Definition at line 62 of file FormantShifter.h.

Referenced by Process(), and Reset().

mLogger

FormantShifterLoggerInterface& FormantShifter::mLogger

private

Definition at line 61 of file FormantShifter.h.

Referenced by Process().

mSampleRate

const int FormantShifter::mSampleRate

private

Definition at line 60 of file FormantShifter.h.

Referenced by Process().

mWeights

std::vector<float> FormantShifter::mWeights

private

Definition at line 66 of file FormantShifter.h.

Referenced by Process(), and Reset().

---

The documentation for this class was generated from the following files:

• FormantShifter.h
• FormantShifter.cpp