1. One thing that differentiates the past from the future is entropy. Entropy is a flattening and a dispersal of energy. The entropy of the universe increases in the future and decreases in the past. This fundamental difference between the past and future might change our experience of it.
2. There may be other fundamental differences between past and future. If multimnemism and multirespism are to be believed, then it is possible that the ability associated with multimnemism (looking into the past) is stronger than that associated with multirespism (looking into the future). Perhaps this is related to the physiology of the brain, specifically neurons, which have many dendrites (inputs) but only one axon (output).
3. It is also possible that existence is defined by a vector, and humans are oriented out of the past and towards the future. Taking this further, possibly there are other "existences" defined by other vectors, in which we are oriented towards the past. In these scenarios we would "remember" our future and "move towards" the past - essentially time would appear to run backwards.
4. It is also possible that as an observer, we are looking towards the past, via memora. But in reality we do have the ability to orient ourselves in the other direction, we just can't ever remember doing it, as it does not generate memory. This implies exocognism.
See also temporal illusion.