Copied
  • Blog
  • Agency Insights , Brand Insights

5 Questions to Ask Before Choosing a Synthetic Data Vendor

November 15, 2026
Get the freshest insights — straight to your inbox.
5 Questions to Ask Before Choosing a Synthetic Data Vendor

Key Takeaways: 

  • Synthetic data tools can be wrong without telling you. The questions you ask before you buy are the best protection against acting based on ineffective tools. 
  • Start with evidence: what real-world data are the answers built on, and how does that data actually shape the results? 
  • Ask how the vendor handles the known quirks of AI-generated answers, like a tendency to pick the middle option or give inconsistent responses, and whether you can see the reasoning behind each answer. 
  • Find out what you get at the end. A report tells you something. An audience you can activate lets you do something about it. 
  • Be wary of answers that arrive instantly. Custom results tailored to your question will take more than seconds. 

You might sometimes feel like you need to be a data scientist to truly understand synthetic data in marketing. But the good news is, you don’t need to be one to thoroughly evaluate a vendor. In this blog, we’ll discuss five questions you should include in your RFP or ask in your next vendor conversation. 

1. What real-world evidence are your synthetic responses built on?

This is the most important question on the list. After all, synthetic data is only as good as the foundation it’s built on.  

Why it matters: General AI tools, like the chatbots your team may already use, have learned a lot from the internet. If you ask one a question about a specific audience, you’ll get an answer that sounds plausible. Unfortunately, these tools aren’t always right, and they won’t tell you when they’re wrong. As a result, if you’re using synthetic data that’s based only on an AI tool, you’ll either spend a ton of time double-checking its work or start developing strategies based on information that isn’t right.  

What to ask for: Ask the vendor to name the sources behind their answers. Are they built on data collected from real people? How current is it? Is it tied to individuals, or is it a general average? A credible vendor should be able to answer clearly and without hedging.

2. How does that evidence actually shape each answer?

Once you’ve determined a vendor has good data, you need to find out if they’re using it well. 

Why it matters: Having real-world data doesn’t mean every answer is built from it. In some systems, the data defines who the synthetic respondent is, and then the AI model answers the question from its own general knowledge. In others, the model fills in whatever the data doesn’t cover. Either way, you get a confident answer with no indication of whether it came from evidence or from the model’s assumptions. You want to know that the answer to your question traces back to real data about your audience. 

Pitfalls to look out for include: 

  • The data builds the respondent, and the model writes the answer. A vendor might use survey data on 10,000 people to create a synthetic respondent: 42, suburban, two kids, $95K household income. Then the model is asked how that person would answer a question about EV charging. The data decided who is answering. The model’s assumptions about suburban parents decided what they say. 
  • The data doesn’t cover your question. If the vendor’s data includes groceries and streaming but nothing in your product category, the model still produces an answer. It extrapolates from patterns that may not apply to your audience, and nothing in the output separates “the data said this” from “the model guessed this.” 
  • The data gets used inconsistently. Most systems have some process for deciding which pieces of evidence to bring to bear on a given question. When it works, the answer reflects relevant data. When it pulls in something only loosely related, or finds nothing useful, the model answers anyway. That means one question can be well grounded and the next can be mostly the model’s own assumptions, and reading the results won’t tell you which is which.  

What to ask for: Pick one result and have the vendor explain, in plain language, how their data produced it. A vendor who understands their own method can do this in a few minutes. Ask: 

  • Does your data determine who the respondent is, how they answer, or both? 
  • Which data points informed this specific answer? 
  • What happens when your data is thin or missing for my audience? Does the model fill the gap, and do you flag it? 
  • How do you check that the data shaped a given answer, and how often does it fall short? 
  • Can you show me an answer that changed when the underlying data changed? 

3. How do you handle the quirks of AI-generated answers? 

AI is used in some way in all synthetic data. This in itself isn’t an issue, as long as it’s not a chatbot doing all the work or giving answers that nobody is correcting.  

Why it matters: AI models have habits that distort results without looking wrong. Reputable vendors know these habits and account for them. Two matter most: 

  • Picking the middle. Models tend to choose the moderate answer, which makes an audience look less opinionated than it is. Say you ask whether people would pay more for a product made with recycled materials. Among real people, 45% might say “definitely yes,” 40% “definitely not,” and only 15% “maybe.” A synthetic audience could come back with 70% “maybe.” You’d conclude your audience is lukewarm when it’s actually split, and the strategy for a split audience is very different from the strategy for an indifferent one. 
  • Answering differently each time. Ask the same question twice and you can get two different answers. A budget decision built on an answer that wouldn’t survive a second ask rests on luck. 

What to ask for: Ask how the vendor makes sure answers are stable and reflect a genuine preference rather than the model’s habits. Specifically: 

  • If I ask the same question five times, how much do the answers vary? 
  • How do you check that the spread of answers matches how real people split? Do you compare against real survey results? 
  • What do you do when synthetic and real results disagree? 

A credible vendor can show you the comparison and tell you where results diverge. “The model handles that” is not an answer. 

4. What do I get at the end, and can I act on it? 

Many synthetic data vendors stop at a report. Others turn the answers into an audience you can reach. 

Why it matters: A report tells you what your audience thinks, but it doesn’t put a message in front of anyone. If that’s all you get, you need another step to reach people, and sometimes another vendor. That adds time and cost and leaves a gap you have to fill, and you have to trust that whoever fills it translates the insight accurately. 

Reaching people also requires a link between the synthetic findings and real individuals. For some vendors, a synthetic respondent is only a model of a person, not built around actual people, so you can’t advertise to it. For an insight to become an audience, the vendor has to connect it back to real people with matching characteristics, such as a segment you can load into an ad platform or match against your own customer data. And you need to understand that process during your evaluation window.  

What to ask for: Ask what format the results come in and how an insight becomes something you can act on. 

  • Can the results be delivered as an audience I can activate in media, or added to my existing customer data? 
  • How do you connect a synthetic finding to real people, and how do you check that the audience reflects the answer? 
  • Which platforms and channels can I use it in? 

5. How long will this take, and what happens in that time? 

Speed is one of synthetic data’s biggest selling points. Getting results in days instead of weeks changes how often you can test ideas (and how fast you can put the research into the market to drive revenue). The question to ask is what happens between your question and the answer, so you can properly understand the real value of the time you’re saving.  

Why it matters: Some questions a vendor can answer immediately because the evidence already exists in their system, like a broad question about an audience they’ve studied extensively. A new question takes more work: a niche audience, a new product category, or a concept the data hasn’t touched. Someone has to check whether the data covers it (question 2), test for the model’s habits (question 3), and shape the output into something you can use (question 4).  

A timeline is certainly relevant to your evaluation, but it’s most useful when you understand where that timeline is coming from, what it reflects, and why that matters to your business. For example, If a vendor returns an instant answer to a new question, the steps we identified above may not have happened; you may just be getting the same generic answer anyone else would get. 

In other words, you need the context behind the speed, or you may find yourself repeating your work and wasting even more time and money. 

What to ask for: Come in with the questions you need answered and the outcomes you’re working toward, and put them in your RFP’s scope of work. Then ask: 

  • Is my question one you can answer from existing data, or does it need new work? 
  • What happens between my question and your answer, and which steps take the time? 
  • What’s a realistic timeline, and what could make it longer? 

See how we answer these five questions

A good synthetic data vendor can answer every question on this list in plain language and back it up with evidence. Bring them to each conversation, and add them to your RFP scope so every vendor responds to the same ones.

We’d like to take our turn in the hot seat while you put Resonate Digital Twins to the test. Set up some time with our team of experts and we’ll walk you through our answers to these five questions, or any others you might have.

Frequently Asked Questions 

What should a synthetic data RFP include?
At minimum, ask about the evidence behind the answers, how that evidence shapes results, how the vendor handles the quirks of AI responses, whether you can see the reasoning, what the final deliverable looks like, and how the work is scoped and timed. 

Why can’t I just use a general AI chatbot for audience research?
General AI tools can produce confident answers that are wrong, without signaling that anything is off. Without real-world evidence behind the answers, it’s hard to know when to trust them. 

Is a faster synthetic data vendor always better?
Not necessarily. Weeks-long timelines defeat the purpose, but instant answers may be generic rather than built for your specific question. A timeline of a few days often reflects work done specifically for you. 

What’s the difference between a synthetic data report and an activatable audience?
A report tells you what an audience thinks. An activatable audience lets you reach those people with media, without a separate step to turn the insight into a target list.