How to test plugins?

ABX and blind testing are the best techniques for objectively comparing audio plugins by removing biases that influence our perceptions.

ABX Testing: This involves comparing two audio samples (A and B) and testing a third sample (X) to determine whether it matches A or B. The tester is unaware of which sample is A or B during the test, ensuring judgment is based purely on what they hear.

Blind Testing: Similar to ABX testing, blind testing hides the identities of the audio samples, ensuring decisions are not influenced by brand names, price tags, or preconceived notions.

These methods eliminate several psychological biases:

• Confirmation Bias: Ensures expectations do not color audio quality perception.
• Brand Bias: Focuses decisions solely on sound by hiding brand identities.
• Price Bias: Emphasizes quality over cost by obscuring price information.
• Expectation Bias: Prevents marketing and reviews from skewing judgment.

Tools for ABX and Blind Testing

Several tools facilitate ABX and blind testing, making the process more accessible and efficient.

• Overview: Lacinato ABX/Shootouter is a cross-platform ABX and shootout blind testing software designed for audio. It allows double-blind ABX testing of audio files and shootouts, where you can pick your favorite from a shuffled, anonymized list.
• Features: Supports ABX testing, ABCD..X testing (more than two files), shootouts with fine-grained scoring, and convenient file switching. It maintains the current play position, automatically detects changes in audio files, and supports drag-and-drop functionality.
• Pros: User-friendly, lightweight, free to download and use, no need to install Java despite being Java-based, runs on Linux, Windows, and Mac.
• Cons: Limited advanced features compared to some other tools, automatic change detection doesn't work on Windows due to a Java bug.

• Overview: Designed for easy and effective blind tests, 4U+ BlindTest by HOFA allows for blind listening comparisons of multiple audio files.
• Features: Blind test setup, multiple sample comparisons, randomized playback.
• Pros: Intuitive interface, flexible comparison options, great for testing multiple plugins.
• Cons: Focused primarily on blind testing, lacks some of the advanced comparison features of MCompare.

There is a free version (3 tracks limit) and a paid version. 

• Overview: MCompare offers both blind testing and comprehensive audio comparison features.
• Features: Blind testing, loudness matching, integrated A/B comparison, reference track loading.
• Pros: Versatile, professional-grade tool with extensive features, excellent for detailed comparisons.
• Cons: More complex interface, may have a steeper learning curve for beginners.

Bertom's EQ Curve Analyzer is a powerful free tool designed to analyze the frequency response (magnitude and phase) of any plugin or hardware. This plugin is particularly useful for understanding the impact of EQs and other audio effects on your sound. For instance you can easily match EQ curves of hardware emulations with your favorite parametric EQ like Pro-Q 3. 

Also it easily detects "hidden gain" a lot of plugins introduce in their default state. You can easily uncover things like that, making sure you won't be biased just because a plugin introduces clean gain in addition to it's original processing. 

Features and Benefits:

• Educational Tool: Ideal for visualizing the effects of classic EQ emulations and understanding their behavior.
• Real-Time Comparison: The "group" system allows you to compare two analyses in real-time on the same graph, providing immediate visual feedback on the differences between settings or plugins.
• Isolated Analysis: Note that while capturing the frequency response, the analysis is isolated between the two instances, meaning you won't be able to hear the audio during this process.

EQ Curve Analyzer simplifies the process of frequency analysis, making it an invaluable tool for both learning and precise audio engineering.

Understanding a plugin's harmonic response helps you assess its impact on your sound. This can be done using a test tone generator or oscillator with a frequency analyzer.

Concept Overview:

1. Test Tone Generator/Oscillator: Generates a pure tone (sine wave) at a specific frequency, like 1 kHz.
2. Frequency Analyzer: Visualizes the frequency spectrum, showing both the fundamental frequency and any harmonics introduced by the plugin.

How to Perform Harmonic Response Analysis:

1. Generate a Test Tone: Use the generator to produce a pure tone.
2. Insert the Plugin: Place the plugin you want to analyze after the test tone generator.
3. Analyze the Output: Use a frequency analyzer to observe the fundamental frequency and any additional harmonics.

TDR Prism is a modern frequency analyzer focused on human audio perception. It provides precise insights into the spectral properties of a signal and is highly configurable and user-friendly. The great thing about Prism is, that you can compare signal of several plugin instances. This makes it the perfect tool to compare the harmonic response of two or more plugins. 

• Features:
  • Human auditory model including auditory masking.
  • Various spectral weighting/tilting methods and peak/RMS smoothing options.
  • Supports sidechain inputs, external files, and links to other Prism instances.
  • High precision plotting with a visually appealing interface.
  • Freely resizable hardware-accelerated interface, including fullscreen support.
• Pros: Intuitive, flexible, and provides detailed spectral analysis tailored to human hearing.
• Cons: Advanced features may require some learning for beginners. 

The free version of Voxengo Span is a very popular option among professionals. It is a real-time FFT spectrum analyzer plugin for professional audio production. It offers detailed visual analysis of the audio spectrum, making it a great tool to check for harmonic response and aliasing. 

• Features:
  • Real-time spectrum analysis with adjustable block size and overlap.
  • Multi-channel analysis capability.
  • Spectrum smoothing and K-system metering.
  • Various display modes including "real-time maximum" and "average".
  • Resizable interface with different color schemes.
• Pros: Versatile, accurate, and easy to use with a range of customizable options.
• Cons: Lacks the advanced features like instance comparison of the paid version. 

In audio production, the phenomenon known as "louder sounds better" can significantly bias our perception when comparing plugins. Our ears tend to favor louder sounds, making it difficult to objectively assess the true impact of a plugin on the audio signal. This is where gain matching plugins come into play.

Gain matching tools help maintain consistent volume levels when comparing processed and unprocessed audio signals, ensuring that loudness does not influence our judgment. By automatically matching the gain, these plugins allow for a fairer and more objective comparison. 

Our favorite gain matching plugin is LetiMix GainMatch but there other notable tools like HoRNet CLMS or the free MAGC by MeldaProductions. 

LetMix GainMatch is an gain matching plugin that ensures accurate level comparisons before and after processing, eliminating biases caused by volume differences.

Features:

• Auto-Leveling: Automatically maintains consistent volume levels.
• Linked Gain Control: Reverse linked "before" input gain with "after" output.
• Signal Switching: Switch between "before" and "after" signals.
• Auto Delay Compensation: One-click auto delay compensation.
• Delta: Listen to the Delta Signal.
• Channel Modes: Supports Stereo, Left, Right, Mid, and Side modes.
• Neutral Mode: Use as a regular gain knob with ±40 dB range.
• Target Loudness Mode: Match with a reference loudness or peak level.

GainMatch is a versatile tool that provides unbiased and accurate audio comparisons and it only costs you $19. 

Overview: PluginDoctor by DDMF is the comprehensive tool for analyzing VST and AU plugins, filling a crucial gap in the market. Whether you're a developer testing your latest plugin or a power user seeking to understand the precise effects of a plugin, PluginDoctor is the go-to solution for in-depth analysis.

Why PluginDoctor is the Standard: PluginDoctor offers a unique combination of features and flexibility, making it the standard for plugin testing and analysis. Its double precision FFT engine, independent stereo channel analysis, and extensive range of testing modes provide unparalleled insight into plugin behavior.

Key Features:

• High-Quality Analysis: Double precision FFT engine for accurate results.
• Linear Analysis: Magnitude and phase response using delta or random input signals.
• Harmonic Analysis: Precise frequency response to sinusoidal input; includes THD/THD+N calculation and intermodular distortion mode.
• Oscilloscope: Real-time visualization of sinusoidal input signals.
• Dynamics Testing: Ramping and attack-release signals for compression and expansion analysis.
• Performance Monitoring: Measures plugin processing time based on buffer size.
• Parallel Plugin Loading: Load and analyze two plugins simultaneously.
• User-Friendly Interface: Freely resizable with three quality settings and built-in screenshot function.
• Standalone and Plugin Versions: Available as a standalone application and as a plugin in VST/VST3/AU/AAX formats.
• Cross-Platform Compatibility: Supports VST, VST3, and AU (on Mac) effect plugins.

Usage:

• Developers: Test audio quality, analyze phase and frequency response, and monitor performance.
• Power Users: Gain detailed insights into how plugins affect audio, compare different plugins or settings, and ensure optimal plugin performance.

PluginDoctor's comprehensive feature set and user-friendly design make it the ultimate tool for anyone serious about plugin testing and audio analysis. The price is usually around $50 at the DDMF shop.