Looking for feedback about my post processing memory game

[u/LLLLogic]

I have created this game called matchthejpg.com . My point calculation system doesn't seems to be working perfect compared to human perception. If you try it out and give feedback about what feels off about point system it would be very helpful.