Loading…
7
TIL my AI model hit 99.7% accuracy on a specific test set
It was a niche medical image dataset, and I honestly thought 95% was the ceiling. Has anyone else seen a jump that big from just tweaking the loss function?
3 comments
Log in to join the discussion
Log In3 Comments
ryan_smith375mo ago
Yeah, I've had that happen before. My model looked great until I realized I was basically just testing it on the training data again. Good point from jordangibson, makes you double-check everything.
7
jordangibson5mo ago
How do you know the test set wasn't just too easy?
4
the_miles5mo ago
That's a solid worry (I saw a talk where someone called it "test set leakage"). Ryan_smith37 is right to flag it, because an easy test set just makes a model look smarter than it is. You really need a tough, separate set of data to know for sure.
5