While the Senate voted 59-41 in favour of the bill, including all 45 Republicans, this fell short of the 60 votes needed to pass it after the US House of Representatives passed a bill to approve the project last Friday.
Bill 2280 would have authorised TransCanada to "construct, connect, operate and maintain the pipeline and cross-border facilities" as specified in an application the company filed in 2012.
The Bill was introduced by Democratic Sen ator Mary Landrieu, who was reportedly hugged by a pro-pipeline fellow Democrat. One Republican reportedly told Landrieu "you're a fighter" on the Senate floor, shaking her hand.
The Bill narrowly failed because the Democrats currently control the Senate, but that will soon change when the new Congress starts in early January, after the Republicans romped home in the US mid-term elections earlier this month.
Republicans and Landrieu vowed to approve the Bill next year, though the White House has indicated that it would veto any such attempt by Congress to force approval of Keystone XL, a caveat the Sierra Club called "enormous good news".
The 1897km pipeline would carry tar sands oil from Alberta, Canada to Nebraska in the US where it joins pipes running to Texas.
Activist group Sierra Club held an emergency staff meeting yesterday to "fire up our aggressive campaign" to ensure President Barack Obama stopped "holding the line" on the issue.
The group used children, wildlife and the future of the planet, all of which it claimed was at stake, as leverage to pass the hat around for donations via an email campaign. Protesters were arrested in Capitol Hill, chanting slogans as they were escorted out as the vote went ahead.
"Just this past Friday, President Obama said his position on the Keystone XL pipeline had not changed; and earlier in the week, he entered into a historic agreement with China to reduce greenhouse gas emissions," the Sierra Club said.
While Obama is reported to have taken a "dim view" of the legislation, it is understood he has not directly threatened a veto.
Canadian Resources Minister Greg Rickford expressing his disappointment: "This project will create jobs, long-term economic prosperity, energy security and environmental stewardship on both sides of our shared border."
He said the pipeline had "strong public support" and was "environmentally sound," adding that the pipeline would replace insecure sources of crude oil.
Alberta Premier Jim Prentice remained hopeful, calling the setback a "step along the way".
"One might expect once the new members of the US Congress, Senate, are sworn in, that this matter will start anew," he said.
TransCanada president and CEO Russ Girling also saw the glass half full, saying the vote showed a "growing and high level of support" for the pipeline among US policy makers.
"We will continue to push for reason over gridlock, common sense over symbolism and solid science over rhetoric to approve Keystone XL and unlock its benefits for America," Girling said in a statement.