Evaluating IT Vendors: Why Certifications and Years in Business Tell You Almost Nothing

Most CEOs and COOs go about evaluating IT vendors the same way: years in business, certifications on the wall. Both feel like reasonable shortcuts. Both will lead you to the wrong firm. The vendor that looks best on paper is not always the one that holds together when a server fails at 11 p.m. on a Tuesday. These are the four questions that actually separate high-performing IT partners from well-marketed ones — and why the metrics you’re probably using right now are the wrong ones for evaluating IT vendors effectively.

1. The Problem With Proxy Signals
2. Question 1: Who Will Actually Work on My Account — and Will They Still Be Here in Two Years?
3. Question 2: What Happens to the Knowledge When Someone Leaves?
4. Question 3: How Many Clients Does Each Engineer Support?
5. Question 4: What Does Your Escalation Path Look Like — and How Long Does Each Step Take?
6. What Good Looks Like
7. Red Flags to Watch For
8. How to Decide

The Problem With Proxy Signals When Evaluating IT Vendors

Certifications and longevity are not worthless. A vendor with no certifications and six months of history is probably not who you want managing your network. But once a vendor clears a basic credibility threshold, these signals stop predicting operational quality almost entirely — which is why evaluating IT vendors on credentials alone is a systematic mistake.

Here is why. Certifications are awarded to individuals, not organizations. A firm can employ one certified engineer, put that credential on the front page of their website, and assign you to someone who has never sat for a single exam. The certification is real. The implication that it represents your day-to-day support experience is not.

Years in business is even more treacherous. Longevity tells you a firm survived — it tells you nothing about whether they improved. A company can operate for 15 years by slowly accumulating clients it never loses through inertia or switching costs, while the quality of their operations quietly stagnates. Survival and excellence are two different outcomes. When you are evaluating IT vendors for a long-term partnership, this distinction is critical.

The Cybersecurity and Infrastructure Security Agency (CISA) regularly reminds organizations that vendor risk assessment must go beyond surface credentials. That guidance applies directly to the IT partners you trust with your business infrastructure.

If you are evaluating IT vendors right now, the question is not what credentials they have. The question is what their operations actually look like when the pressure is on. That requires a different line of questioning entirely.

Question 1: Who Will Actually Work on My Account — and Will They Still Be Here in Two Years?

This is the question most buyers never think to ask when evaluating IT vendors. It may be the most important one on this list.

The person who closes your deal is almost never the person who answers your tickets. Ask the vendor directly: who are the specific engineers assigned to accounts like mine, and can I meet them before we sign?

Then ask about turnover. The managed IT industry has a well-documented retention problem. Talented engineers get recruited aggressively, and firms running on thin margins rarely compete on compensation. If a vendor cannot tell you their average engineer tenure, that silence is an answer. If they claim low turnover, ask them to explain what makes that possible.

The reason staff continuity matters so much is institutional memory. An engineer who has worked inside your environment for three years knows where the bodies are buried — your backup quirks, your firewall exceptions, the one legacy application that requires a specific workaround. That knowledge does not live in a ticket system. It lives in a person. When that person leaves and is replaced by someone new, you absorb the cost of their learning curve at exactly the moment you cannot afford it.

A firm that has deliberately built a culture where engineers stay has solved a hard problem. That is not a given. It is a choice. Staff continuity is one of the most important criteria when evaluating IT vendors and is rarely surfaced in standard evaluations.

Question 2: What Happens to the Knowledge When Someone Leaves?

Even the best firms lose engineers occasionally. What separates high-performing IT partners from mediocre ones is what happens next. The answer lies in internal documentation practices — and it is a key factor when evaluating IT vendors for long-term reliability.

Ask the vendor: “If your lead engineer on our account left tomorrow, what would their replacement have access to?” The answer should be immediate and specific. A well-run firm maintains living documentation for every client environment — network diagrams, credential vaults, configuration records, known issues, custom scripts, and change logs. All of it current. All of it accessible to any engineer on the team.

If the vendor pauses, pivots to their ticketing system, or answers in vague generalities, you have learned something important. Ticketing systems record what happened. They do not capture why your environment is built the way it is, or what will break if someone touches the wrong setting.

The documentation question is also a proxy for operational discipline more broadly. Firms that document well tend to operate methodically. Firms that do not tend to run on tribal knowledge and individual heroics — which works fine until it does not.

Ask to see a sample runbook or environment summary, anonymized for a comparable client. A firm confident in their documentation will not hesitate. One that is not will deflect. This is a critical step in thorough IT vendor due diligence that most buyers skip entirely.

Question 3: How Many Clients Does Each Engineer Support?

This number is rarely published, almost never volunteered, and tells you more about your actual support experience than any other metric on a vendor’s website. It is one of the most revealing data points when evaluating IT vendors — and it is the one most vendors hope you never ask.

Client-to-engineer ratios vary enormously across the industry. A firm running a lean, high-touch model might have one engineer responsible for five to eight client organizations. A firm optimizing for margin might have one engineer carrying twenty-five or more. Both can answer your initial inquiry within minutes. Only one can deliver sustained attention when your needs are complex or overlapping with another client’s crisis.

The ratio also tells you how much of the relationship is genuinely managed versus purely reactive. At high ratios, engineers spend most of their time responding to what has already broken. At lower ratios, there is bandwidth to be proactive — to catch the thing that would have broken next month before it becomes your emergency this week.

When you ask this question, watch for the vendor who answers with headcount instead of ratio. “We have 30 engineers” means nothing without knowing how many active client accounts those engineers carry. Push for the ratio. If they will not give it to you, assume the worst.

A well-structured IT relationship is built on the premise that your vendor knows your environment deeply enough to anticipate problems. That is only possible if the engineers on your account have the time to do it. This is why evaluating IT vendors on ratio — not just headcount — is so essential.

Question 4: What Does Your Escalation Path Look Like — and How Long Does Each Step Take?

Every IT vendor will tell you they have an escalation process. Almost none will walk you through it in granular detail unless you press. That granularity is exactly what matters when evaluating IT vendors for mission-critical support.

Ask the vendor to describe, step by step, what happens from the moment something critical fails until it is resolved. Who picks up the initial alert? How long do they work the problem before pulling in a second engineer? Is that second engineer an internal senior resource or a third-party vendor? What happens if the problem requires hardware replacement at midnight? Who calls the vendor? Who follows up on the part?

The goal is not to stress-test their process for the sake of it. The goal is to surface whether the escalation path is real and practiced or theoretical and aspirational. Firms that have genuinely thought through their incident response can describe it specifically. Firms that have not will give you something that sounds like a process but collapses under the first “and then what?” follow-up.

Also ask: who is available outside business hours, and at what level of seniority? A firm that escalates to a junior on-call engineer at 2 a.m. is a very different firm from one that escalates to a senior engineer who has handled this specific type of incident before. Both may advertise “24/7 support.” Only one of them is delivering it in any meaningful sense.

You can benchmark any vendor’s approach against the NIST Computer Security Incident Handling Guide (SP 800-61r2) — a credible reference point for evaluating whether a vendor’s incident process holds up under scrutiny.

What Good Looks Like

A high-performing IT partner answers the four questions above without hesitation, without pivoting to their sales materials, and without asking for more time to prepare. They have thought through these problems because they have lived them. When evaluating IT vendors, this responsiveness itself is a meaningful signal.

Staff continuity is the result of deliberate hiring, competitive compensation, and a culture that makes skilled engineers want to stay. It is not an accident — it is a choice. Ask what percentage of their engineers have been with the firm for more than three years.

Documentation is treated as infrastructure, not overhead. Every client environment is mapped, current, and accessible to any engineer on the team. When someone leaves, the next engineer is not starting from scratch.

Ratios reflect a conscious decision about the kind of firm they want to be. Keeping ratios low means accepting lower revenue per engineer in exchange for deeper client relationships and better outcomes. That is a values decision, not just an operational one.

Escalation paths are practiced, not just described. Senior engineers know the process because they have walked through it. There are no surprises about who calls whom at midnight.

For businesses in South Jersey and the Philadelphia metro area evaluating whether their current IT setup actually meets this bar, our managed IT services page covers how we approach each of these dimensions in practice.

Red Flags to Watch For

• The vendor answers the staffing question by describing company size, not your specific account team.
• Documentation is described as “stored in our ticketing system” with no mention of runbooks or environment-specific records.
• The client-to-engineer ratio is deflected, qualified, or simply unavailable.
• The escalation process is summarized as “we have a team that handles that” — no names, no seniority levels, no timeframes.
• The person selling you the engagement will not commit to introducing you to the engineers who will actually work your account before you sign.
• Turnover is dismissed as a non-issue, with no explanation for what makes retention possible. Any firm that claims to have solved it without explaining how has not solved it at all.
• The conversation keeps returning to certifications or client counts as proof of quality, rather than operational specifics.

If you notice several of these red flags during your evaluation conversations, treat them as disqualifying signals — not minor concerns. When evaluating IT vendors, the friction you feel before signing is a preview of the friction you will feel after.

How to Decide

The vendors who perform best in a crisis are not the ones with the most impressive credential walls. They are the ones who have built operations that do not depend on any single person, any single heroic effort, or any single piece of luck. That takes deliberate choices made over years — choices most buyers never ask about because they do not know to look for them. Evaluating IT vendors on these operational dimensions is how you close that gap.

Run the four questions above with every vendor you are seriously considering. Take notes on how they answer, not just what they answer. Hesitation, deflection, and vague generalities are data. So are specificity, confidence, and the willingness to put you in front of the actual engineers before the contract is signed.

Certifications matter at the margins. Longevity matters at the margins. What actually predicts whether your IT partner will be there when something goes wrong — and will have the depth to fix it — is the operational reality underneath the marketing surface. That reality is entirely visible if you know what to ask. Evaluating IT vendors through these four operational lenses gives you a picture that no credential wall ever will.

The firms with nothing to hide will walk you through those four questions without flinching. The ones who cannot are telling you something important. Believe them. For further guidance on what a well-structured IT partnership looks like from day one, the Xact IT services overview covers how we approach it in practice.