Speculation
Gee, don't get too down on yourself. Just because some guy writes a book called, "Why I Am Not A Christian" does not mean he necessarily believes Jesus did not exist. What if a Muslim wrote a book with the same title? The Koran teaches that Jesus existed. Not all non-Christians doubt the existence of Jesus.
One meaning of the word speculate is to "form opinions on the basis of little or no evidence." Whether there is little or no evidence that Jesus did not exist is purely subjective, a matter of opinion. Lots of books have been written about the subject, so it could be argued that there's lots of evidence. In which case, you've got no hope of ever proving that all non-believers can do is speculate.
It could be argued that since there's little historical evidence Jesus DID exist, those who argue that He did live are speculating. Again, this is just a matter of opinion.