A compliance program can look strong on paper and still fail in practice.
Policies may be current. Procedures may be documented. Controls may have assigned owners. Compliance technology may generate alerts, surveillance reports, certifications, and exception records exactly as designed.
However, none of those elements answers one of the most important questions a Chief Compliance Officer or Chief Operating Officer may face:
Does the compliance system actually work?
For investment firms, compliance testing provides the evidence needed to answer that question.
Testing helps determine whether policies and controls remain appropriately designed, whether employees follow them in practice, whether systems perform as intended, and whether weaknesses require remediation. Just as importantly, testing gives compliance leadership visibility into the difference between the firm’s documented compliance framework and its actual operating environment.
That distinction matters from a regulatory perspective.
Under SEC Rule 206(4)-7, registered investment advisers must adopt and implement written policies and procedures reasonably designed to prevent violations of the Advisers Act and its rules. In addition, advisers must review those policies and procedures at least annually to determine both their adequacy and the effectiveness of their implementation.
Therefore, compliance testing cannot simply ask whether a policy exists.
It must help firms understand whether the compliance framework operates effectively in practice.
For CCOs and COOs, the challenge becomes deciding what to test, when to test it, who should conduct the testing, and what evidence the organization should retain afterward.
Compliance Testing Is More Than an Annual Exercise
The annual compliance review is an important regulatory requirement, but firms should not confuse an annual review with the entirety of their compliance testing program.
The SEC has stated that advisers should consider significant compliance events, changes in business arrangements, and regulatory developments when determining whether interim reviews are appropriate. In other words, an annual review establishes a minimum cadence, not necessarily the only point at which firms should evaluate their controls.
A mature compliance testing program works throughout the year.
For example, compliance teams may test certain high-risk controls quarterly, review others annually, and reassess specific controls whenever a material event occurs.
Those events could include:
- A new regulatory requirement
- A new investment strategy or product
- Entry into a new jurisdiction
- Changes to trading or compliance technology
- A merger or acquisition
- Significant personnel changes
- Repeated compliance exceptions
- A regulatory examination finding
- A cybersecurity incident
- Changes to a critical vendor
- Material revisions to policies or procedures
As a result, the testing calendar should reflect the firm’s actual risk environment rather than an arbitrary schedule.
What Should Compliance Teams Test?
The short answer is not everything at the same frequency or with the same level of intensity.
A risk-based approach allows firms to concentrate testing resources on the controls, systems, and obligations where failure could create the most significant regulatory, financial, operational, or reputational consequences.
However, firms first need visibility into the compliance environment they are attempting to test.
Ideally, the organization should be able to connect:
Regulatory requirement → policy → procedure → control → owner → test → evidence
Without those relationships, compliance testing can become a collection of isolated exercises rather than an evaluation of the broader compliance framework.
Several areas generally deserve consideration.
1. Test Whether Controls Are Properly Designed
The first question is whether a control is capable of addressing the risk or regulatory requirement it was intended to address.
This is a test of control design.
Suppose, for example, an investment adviser has a policy governing employee personal trading. The firm may require employees to preclear certain securities transactions.
Testing the design of that control could involve asking:
- Does the preclearance process apply to the correct employee population?
- Does it cover the appropriate securities and transactions?
- Does the workflow reflect applicable restricted lists and blackout periods?
- Are approvals routed to the correct personnel?
- Are exceptions documented?
- Does the firm retain the required records?
At this stage, the tester is not necessarily asking whether every employee follows the process.
Instead, the question is whether the process itself is reasonably designed to accomplish its intended purpose.
That distinction becomes important because a perfectly executed control can still fail if the underlying design is inadequate.
2. Test Whether Controls Operate Effectively
Once the firm determines that a control is appropriately designed, compliance teams need to evaluate whether it actually operates as intended.
This is operating effectiveness testing.
Consider the same personal trading example.
Testing could involve selecting a sample of employee transactions and determining whether employees obtained the required approvals, whether compliance completed reviews within the expected timeframe, whether the appropriate restrictions applied, and whether the firm documented the process.
Similarly, if a trading system contains automated investment restrictions, the firm could test whether the system identified transactions that should have triggered those restrictions.
The distinction between design and effectiveness is fundamental.
A control may be designed correctly but inconsistently performed.
Conversely, employees may perform an existing process exactly as documented even though the process no longer addresses the firm’s current risk.
Effective testing needs to identify both situations.
3. Test Compliance Systems and Automated Controls
As investment firms become increasingly dependent on technology, testing should also address the systems that execute or support compliance controls.
Trading systems, surveillance platforms, regulatory reporting tools, employee compliance systems, communications monitoring technology, cybersecurity platforms, and other applications may all play roles in the compliance environment.
Therefore, testing should not assume that automation guarantees effectiveness.
For automated controls, firms may need to evaluate:
- Whether rule logic remains accurate
- Whether data inputs are complete
- Whether integrations function correctly
- Whether system permissions remain appropriate
- Whether alerts reach the right users
- Whether workflows produce the expected outcomes
- Whether configuration changes received proper approval
- Whether exceptions are documented and resolved
- Whether critical system changes affect existing controls
For example, an investment restriction can exist within an OMS for years while the business, regulatory interpretation, portfolio structure, or underlying rule changes around it.
If nobody tests whether the configured logic still reflects the current requirement, the system can continue performing exactly as programmed while producing the wrong compliance outcome.
That is why technology testing should connect back to the underlying compliance obligation.
4. Test Supervisory Procedures
For broker-dealers, supervisory testing carries additional significance.
FINRA Rule 3120 requires member firms to maintain supervisory control policies and procedures that test and verify whether the firm’s supervisory procedures are reasonably designed to achieve compliance with applicable securities laws, regulations, and FINRA rules.
When testing identifies weaknesses, firms must create additional procedures or amend existing ones as appropriate.
Moreover, FINRA requires designated principals to provide senior management with a report at least annually describing the supervisory control system, summarizing testing results and significant exceptions, and identifying procedures created or amended in response.
FINRA also recognizes the use of risk-based methodologies and sampling when determining testing scope.
For CCOs, the broader principle applies beyond any individual rule:
Testing should verify that supervision works, not simply that supervisory procedures exist.
5. Test Data and Evidence
Compliance systems increasingly depend on data.
As a result, testing the control without testing its inputs can create false confidence.
Suppose a surveillance application reviews employee transactions against account data. The surveillance logic may work perfectly, but the control could still fail if the employee account population is incomplete.
Similarly, a regulatory reporting system may perform calculations correctly while relying on incomplete or inaccurate source information.
Therefore, firms should consider testing:
- Data completeness
- Data accuracy
- Data mappings
- Source-system integrity
- Reconciliations
- Exception handling
- Record retention
- Evidence generated by the control
This becomes especially important when compliance processes span multiple systems.
If data moves from a portfolio management system to an OMS, then into a surveillance tool and finally into a compliance reporting environment, the firm needs confidence not only in each system but also in the connections between them.
6. Test Changes, Not Just Existing Controls
One of the highest-risk moments in any compliance environment occurs when something changes.
A new rule may require an updated control. A new product could create risks the existing framework never contemplated. A system migration may alter data flows. An acquisition may introduce hundreds of inherited rules and procedures.
Consequently, change management should become part of the testing process.
When a material change occurs, compliance teams should ask:
- Which requirements are affected?
- Which policies need review?
- Which controls depend on those policies?
- Which systems execute those controls?
- Who owns the affected processes?
- Does testing need to change?
- What evidence demonstrates successful implementation?
This approach prevents the testing program from becoming backward-looking.
Instead, testing becomes part of the mechanism that helps the compliance framework evolve.
When Should Compliance Controls Be Tested?
There is no universal testing cadence appropriate for every investment firm or every control.
A risk-based testing schedule is generally more useful than applying the same annual frequency across the entire organization.
For example, a firm might classify controls as high, medium, or lower risk based on factors such as regulatory significance, potential client impact, transaction volume, reliance on automation, prior testing results, history of exceptions, complexity, and recent changes.
A higher-risk control may warrant quarterly or continuous monitoring combined with periodic formal testing.
By contrast, a stable, lower-risk administrative control may require less frequent review.
The firm’s testing calendar can also combine several approaches:
Scheduled testing occurs at predetermined intervals, such as monthly, quarterly, semiannually, or annually.
Event-driven testing occurs when a material change or incident creates a reason to reassess a control.
Continuous monitoring uses ongoing data or system activity to identify potential exceptions or changes requiring attention.
These approaches are complementary.
For example, continuous monitoring may identify a pattern of exceptions that then triggers deeper control testing.
Ultimately, the question should not be, “Did we test this control this year?”
A better question is, “Are we testing this control often enough given its current risk?”
Who Should Perform Compliance Testing?
This question can become particularly complicated for large investment organizations.
The individual or team that performs a control often possesses the greatest subject matter expertise.
However, asking the control owner to provide the only assessment of that control can limit objectivity.
Therefore, firms need to think carefully about independence.
The appropriate testing model will vary based on firm size, complexity, regulatory status, resources, and risk. Depending on the control, testing might involve:
- Compliance personnel
- Dedicated compliance testing teams
- Risk management
- Internal audit
- Operations
- Technology or information security
- External consultants
- Specialized subject matter experts
The key is to distinguish between performing a control, monitoring a control, testing the control, and independently auditing the broader framework.
Those activities are not necessarily the same.
The Case for Independent Testing
Independence does not always require an outside firm.
However, the tester should have enough separation from the control to evaluate it objectively.
For instance, if a business unit performs a control, compliance may test it. If compliance itself performs the control, another compliance team, risk function, internal audit group, or external reviewer may provide additional independence.
The appropriate degree of separation should generally increase with the significance of the risk.
This approach can also help address confirmation bias.
Control owners naturally understand how a process is supposed to work. As a result, they may unintentionally test the documented process rather than challenge whether the process remains adequate.
An independent tester can ask a different question:
What would cause this control to fail?
That mindset often produces more meaningful testing.
Sampling Should Be Risk-Based Too
Testing every transaction, approval, communication, or control execution is rarely practical.
Therefore, sampling often becomes an important component of compliance testing.
FINRA specifically notes that firms may use risk-based methodologies and sampling to determine the scope of supervisory control testing.
However, a sample should do more than simply select a convenient number of records.
Firms may consider factors such as:
- Transaction volume
- Time period
- Business unit
- Geography
- Employee population
- Product type
- Risk level
- Prior exceptions
- Control changes
- Unusual activity
A combination of random and targeted sampling can often provide a more useful picture.
For example, a firm may randomly select a baseline sample while intentionally including higher-risk transactions, known exceptions, newly onboarded employees, or activity occurring after a system change.
The objective is not statistical perfection in every case.
Instead, firms should design samples that meaningfully test the risk the control is intended to address.
What Happens When Testing Finds a Problem?
A mature testing program assumes that testing will identify weaknesses.
That is not necessarily a sign of failure.
In fact, a testing program that never identifies an issue may deserve scrutiny of its own.
The critical question is what happens after the firm identifies an exception.
A strong process should connect:
Test → finding → affected control → regulatory requirement → remediation → owner → deadline → retest → closure
For minor issues, remediation may involve clarifying a procedure or providing additional employee guidance.
More significant findings may require system changes, control redesign, policy updates, additional supervision, employee training, or escalation to senior management.
Most importantly, firms should verify remediation rather than assuming that completing an action item solved the underlying problem.
A remediation ticket marked “closed” does not necessarily mean the control now operates effectively.
Retesting closes that loop.
Documentation Makes Testing Defensible
For CCOs and COOs, documentation may be just as important as the testing itself.
Consider a regulatory examination several years after a test occurred.
An examiner asks why the firm concluded that a particular control was effective.
Could the organization quickly provide:
- The requirement being tested
- The control being evaluated
- The testing methodology
- The testing period
- The sample selected
- The person who conducted the test
- The evidence reviewed
- The results
- Identified exceptions
- Remediation activity
- Retesting results
- Final disposition
If that information exists across spreadsheets, email threads, ticketing systems, shared drives, and individual recollections, reconstructing the testing history can become difficult.
By contrast, a connected compliance record creates traceability.
This is particularly relevant for investment advisers because Rule 206(4)-7 now requires SEC-registered advisers to document their annual compliance reviews in writing.
The documentation should help tell the story of how the firm evaluated its compliance framework and what it did with the results.
From a Testing Calendar to a Testing Framework
Many firms already maintain some form of compliance testing calendar.
The next step is turning that calendar into a governance framework.
Instead of maintaining a list that simply says:
Personal trading: Q1
Marketing review: Q2
Best execution: Q3
Cybersecurity: Q4
a more mature framework connects each test to the broader compliance environment.
For every test, the firm should be able to understand:
- Why the test exists
- Which requirement it supports
- Which control it evaluates
- Who owns the control
- Who performs the test
- How frequently testing should occur
- What methodology applies
- What evidence the tester should review
- What prior findings exist
- Whether remediation remains outstanding
That creates continuity.
It also helps compliance leadership identify gaps.
For example, the CCO may discover that several high-risk controls have no defined testing process. Conversely, multiple teams may unknowingly test the same control while another important area receives little attention.
Visibility makes those problems easier to address.
What Should CCOs and COOs Be Able to See?
At the leadership level, compliance testing should ultimately provide more than individual workpapers.
CCOs and COOs need an enterprise view.
They should be able to answer questions such as:
- Which high-risk controls have been tested?
- Which tests remain overdue?
- Where are the most significant findings?
- Which business units generate repeated exceptions?
- Which controls have failed more than once?
- What remediation remains outstanding?
- Who owns each open issue?
- Which controls changed since their last test?
- Where does testing coverage appear weak?
- What evidence supports the firm’s annual compliance review?
These questions turn compliance testing into management information.
Instead of viewing testing as a periodic compliance obligation, leadership can use testing results to understand where the firm’s compliance framework is strong, where risk is increasing, and where resources may need to shift.
Better Testing Creates Better Governance
Compliance testing should not exist simply to prove that someone reviewed a control.
Its real purpose is to determine whether the compliance framework remains aligned with the firm’s regulatory obligations, business activities, technology, and risk.
That requires more than an annual checklist.
Firms need a structured approach for deciding what deserves testing, setting testing frequency based on risk, establishing appropriate independence, documenting methodology, tracking findings, managing remediation, and verifying that corrective action actually worked.
Most importantly, testing should connect back to the regulatory requirements and controls it is designed to evaluate.
When those relationships are visible, compliance leaders gain something more valuable than a collection of completed tests.
They gain evidence that their compliance framework works.
Build a More Connected Approach to Compliance Testing
As investment compliance environments become more complex, managing testing through disconnected spreadsheets, workpapers, emails, and operational systems can make it difficult to maintain a complete picture.
TillieStar helps investment compliance teams connect regulatory requirements, policies, controls, ownership, testing, findings, evidence, and remediation within a more structured governance environment.
That traceability can help CCOs and COOs understand not only whether testing occurred, but what was tested, why it mattered, what the firm found, and whether remediation ultimately resolved the underlying issue.
Because effective compliance testing should answer more than “Did we test it?”
It should help the organization confidently answer:
“Does it work?”
Ready to build a more connected and defensible compliance testing framework?
Contact TillieStar at sales@tilliestar.com or (617) 865-3550 to start the conversation.