Do you also provide the JS that creates the Ajax call? you could use that to get the client URL and return that value in the the Ajax payload. People may not like it and could consider it 'snooping'.
Personally I'd stick with the referer unless it was proven to be unreliable.