Cancer charity Planets described the situation as a "scandal" after exposing cases including a patient being quoted £7,000 for insurance for a two week trip to Canada. Another patient received a quote for £1,000 for a stay in Spain.
Both patients have neuroendocrine tumours, known as NETs, which are slow growing and often stable. They usually grow in the pancreas, bowel or lungs but can develop in other parts of the body too.
The chances of these patients needing emergency care is incredibly rare, the charity said, and so they were being unfairly punished due to lack of understanding.
Neil Pearce, co-founder of the charity Planets said it has been down to him to write letters to insurance companies on behalf of his patients, and said other charities must do the same too. It is usually achieves little difference, leaving patients and their families with limited options and sky-high fees.
He said: "It should not be left to a lottery of finding a company who will accept a letter."
The Southampton-based charity has launched a campaign, with a petition calling for the travel insurance industry to review its procedures and work with cancer organisations and charities to create a fairer way of offering cover.
Mr Pearce added: "The difficulty for cancer patients in obtaining travel insurance at all or dealing with the inflated premiums is a scandal and something the industry must do something about as it is currently sticking the boot in.
"For many people, the respite and morale boost a holiday away with family or friends brings is invaluable to their life, yet we are in the perverse situation where very little effort is even put in to establish the condition a person is in."
Read our research on the average cost of travel insurance with pre-existing conditions here.
Nimblefins has written a guide with tips to securing the best travel insurance deal with a medical condition. Travellers could be best to look at a single trip policy rather than an annual policy, and compare the market. Read more tips here.