Why We Don’t See Time Travelers
Maybe time travelers don’t come because they fear our diseases, to which they have no immunity. Sudden death by such disease is one way the timeline protects itself from tampering.
You, yourself, should not travel into the past, for the same reason.
You should be safe enough if you observe, don’t meddle, and don’t let yourself be noticed. So maybe we have been visited after all, but only by historians rigorously following the Temporal Prime Directive.