If you were raised on rom-coms and the idea of “love at first sight,” having a “spark” with a potential love interest seems like a no-brainer. While you’re probably not thinking marriage on a first date, you’re looking for some sign this person could be the one you’re looking for… right?

Actually, many single people today aren’t expecting to feel a connection right away, according to a Match.com survey. Of the participants polled, 59 percent of men and women said they would go on a second date with someone they had no romantic chemistry with on the first date.

And a good chunk of people don’t even count on those butterflies early on: 25 percent of singles don’t expect to feel chemistry until the second date, and 33 percent don’t expect to see that spark until three dates in—or more!

So is the instantaneous spark just a fantasy? And if you don’t feel chemistry at your initial meet-cute, should you give him or her a second chance?

More Than a Feeling

What exactly is that “spark”? It means different things for different people, says Michael McNulty, Ph.D., a master certified Gottman Therapist and Chicago-based couples counselor. It can be purely sexual, or it can be a deeper feeling that someone understands you.

Either way, it leads to something very real happening in your brain, McNulty says: a gradual cascade of neurotransmitters that are released as a person falls in love. A few of the heaviest hitters include dehydroepiandrosterone (DHEA), a natural aphrodisiac; phenylethylamine (PEA), a.k.a. the “love drug;” pheromones, which are produced from DHEA and result in sensuality, a sense of well-being, and comfort; and oxytocin, or the cuddle hormone that’s released when people get physically close.

Someone you don’t have a spark with right away could sweep you off your feet if you meet years later.

So why do we have that heart-fluttering reaction with some people and not others? It depends entirely on what you’re looking for, McNulty explains. “Attraction can involve looks, personality traits, shared experiences, ability to commit, and context—are you in a bad mood? is the other person nervous?—among other factors.”

Plus, who we’re attracted to can change over the course of our life, so someone you don’t have a spark with right away could sweep you off your feet if you meet years later, McNulty adds.

You’re (Not) Really Growing On Me

Psychologists have found that most of the time, our social intuition is like a superpower. You only need a few seconds of exposure to someone to make a reliable, long-term judgment, says Sean Horan, Ph.D., an assistant professor in communication studies at Texas State University who studies initial impressions and attraction.

This instant assessment is called thin slicing, and research suggests that even in a brief encounter—as short as a 30-second round of speed-dating—people can quickly and accurately glean information about someone. What’s more, the feeling you get during those initial 30 seconds predicts how you’ll continue to feel in the future.

The biggest factor for a first impression? Physical attraction (no surprise there). One study showed most people can accurately predict who they’d be interested in just by looking at a photo before they even met the person.Dorsomedial prefrontal cortex mediates rapid evaluations predicting the outcome of romantic interactions. Cooper JC, Dunne S, Furey T. The Journal of neuroscience : the official journal of the Society for Neuroscience, 2013, Jan.;32(45):1529-2401.

But it’s not just looks that shape your snap judgment: That same study shows that even if you’re not physically attracted to someone at first glance, a second region of your brain kicks in to help you decide whether someone’s perceived personality makes them a good catch for you.

What if your first impression falls somewhere in the middle? Then it’s time to trust your gut, McNulty says. If there’s something nagging you to give them a second shot, listen to your instinct. Maybe you went out on a bad night. Maybe you failed to find the contexts or common ground that would help us to connect, McNulty says.

Turns out a lot of people go with their gut, even if their first impression wasn’t great: Another study published in the Journal of Social and Personal Realtionships found that when people formed a negative initial impression of a date, 43 percent still wanted to give it another go.

Finally, and this is important: Keep in mind that chemistry alone is not an accurate read that someone is going to be right for you, says Jennifer Gunsaullus, Ph.D., a San Diego-based sociologist and sexologist. “A sexual charge does not mean that person is kind, will respect you, has common values, or that you’d actually be good dating, let alone creating a life together.”

How to Decide on Date Two

Chemistry is important to move forward in a relationship, McNulty says. But it doesn’t have to happen on the first date. As long as you aren’t feeling negatively toward that person, there are a lot of reasons you may not be feeling giddy just yet.

One reason: if you skipped the good-night kiss. “A lot of people may not feel a spark until the first time they touch the other person,” Gunsaullus says. Still, only 50 percent of singles think a good first date ends with a kiss, according to the Match.com survey.

If locking lips isn’t your style, any sort of physical contact could help. “A touch on the shoulder, brushing hands, placing a hand on the lower back—once there is explicit physical contact, that can ignite that missing chemistry,” she says.

Still no spark or physical attraction? Think about what you did enjoy about the date, she offers. Was there an interest you had in common that you don’t have with anyone else? Did you laugh more than you have in a while? Did they make you feel comfortable and secure?

Finally, flat out ask yourself if you’re looking forward to seeing them again, Gunsaullus suggests. If you are even a little excited about the idea, it’s well worth your while to go out again, even if you weren’t immediately attracted.

But if you’d much rather stay in and watch The Bachelor than grab another drink with the person, there’s your answer. And remember, it’s OK to say no—courtesy dates just lead that person on, which is even worse than rejection.

The Takeaway

If after two or more dates you still don’t feel a spark, move on, McNulty says. But consider staying friends if you enjoyed the time you spent together: “Who you’re attracted to can change over time, and a spark can develop, particularly if you already have that trust and connection built.”