No matter what the weather is, some of us always seem to suffer from chronically cold fingers and toes. And beyond making people jump at your touch, icy extremities can be annoying (thick, scratchy wool socks = not fun) and pretty painful.

Rather than settle for pat answers like “cold hands, warm heart” or the vague idea you might have poor circulation, we dug into the common causes for cold digits and when they might be cause for concern.

Ice, Ice Baby

As it turns out, poor circulation is not the cause. “When you have poor circulation, your skin actually gets red—not white or blue,” says Venita Chandra, M.D., a vascular surgeon at Stanford. “The tiny little blood vessels in the feet and hands are trying to pull as much blood as possible there, so they’re completely vasodialated.” Meaning your blood vessels have widened to their full capacity, allowing them to take in the maximum amount of blood.

Although cold tolerance is different for everyone, for the most part, it’s completely normal for your hands and feet to feel the freeze first when you’re in the cold, Chandra says. It’s part of your natural physiology and method for regulating body temperature: Your body temp is controlled by the hypothalamus, the thermostat in your brain. When you’re exposed to cold, your body pulls blood away from your skin in closer to your core. As the blood—and accompanying heat—rush to warm your vital organs, your extremities are left cold.

But could it be something more? Yes, experts say. Super-cold extremeties could be a symptom of Raynaud’s disease, a medical condition that causes an extra sensitivity to cold.

When people with Raynaud’s—which is around 20 percent of the U.S. population and is more common in women and people living in cold climates—are exposed to cold temps, the blood vessels in the hands and feet close (or vasoconstrict). This causes the extremities to first turn white, then blue. As you warm up, your skin may feel prickly or tingly and turn red. It most commonly occurs in fingers and toes, but may also affect lips, ears, noses, or even nipples.

“Usually, if you see a color change, or if the hands or the feet look pale or are numb, it’s worth seeing a doctor to test for Raynaud’s,” says Catherine Forest, M.D., a family medicine doctor at Stanford. In rare cases, Raynaud’s can be serious and lead to further issues, but for most people, it isn’t debilitating.

In addition to Raynaud’s, occupations that require you to regularly work with vibrating instruments (think power tools) or more commonly perform repetitive motions all day (like typing) could permanently damage your blood vessels and leave you with a feeling of cold or numbness, Chandra says.

Other Reasons to Check With a Doc

If you start to experience numbness, it could indicate another underlying cause—so see a medical professional ASAP. “Numbness is more concerning from a neurological component,” Chandra says. “That could be due to diabetes or peripheral neuropathy, a condition that leaves your extremities less sensitive to touch.

Also: “If you have really cold hands and cold feet in conjunction with other signs like hair loss or changes in your skin, that might signal a real problem with circulation,” Chandra says. And if one particular extremity (think one foot or a few fingers) seems to be suffering from the chills more often than others, that could be a sign of a more serious circulation problem or blood clot.

Your Action Plan

“The best thing one can do is try and avoid the triggers like cold and repetitive motions or vibration,” Chandra says. If you can’t avoid being out in the cold, make sure you’re layering up and not exposing your bare skin to the cold air. (Wearing warm gloves is your best bet—keeping your hands in pockets could lead to a potentially serious injury if you trip on slippery ice!) Also avoid sudden changes in body temperature, like jumping into frigid lake on a hot summer day.

In more drastic cases or when triggers can’t be avoided, there are more extreme solutions. The nerves that cause the blood vessels in your fingers and toes to constrict can be cut to prevent the body’s natural response to the cold, Chandra says—but that’s pretty rare. As for us, we’ll be content with our fuzzy mittens and socks.