Well, I did once scrape wikipedia and classify based off simply off string suffix (storing those where longer suffixes disagreed with the shorter rule), and that's quite simple and effective - https://eamonnerbonne.github.io/a-vs-an/AvsAnDemo/ - and since it's based off actual statistics, it correctly distinguishes textually subtle stuff like "a NASA scientist" vs. "an NSA analyst".
There are structural limitations (i.e. this implementation never looks at preceding context and sometimes that matters, nor does it understand other clues like punctuation or multi-word suffixes). Nevertheless, the accuracy is high enough that I'm not sure it'd be easy for a small model to beat it, especially not without considerably more work to make sure your inputs cover more context (which in principle the plain statistics approach could likely deal with too).
If the whole point of multi-layer networks is to deal with weirdly shaped, non-obvious manifolds in a very high-dimensional space, then this problem just doesn't look that difficult and perhaps does not need that mathematical finesse: just store the prototypical examples and you're pretty much there without anything fancier.
There are structural limitations (i.e. this implementation never looks at preceding context and sometimes that matters, nor does it understand other clues like punctuation or multi-word suffixes). Nevertheless, the accuracy is high enough that I'm not sure it'd be easy for a small model to beat it, especially not without considerably more work to make sure your inputs cover more context (which in principle the plain statistics approach could likely deal with too).
If the whole point of multi-layer networks is to deal with weirdly shaped, non-obvious manifolds in a very high-dimensional space, then this problem just doesn't look that difficult and perhaps does not need that mathematical finesse: just store the prototypical examples and you're pretty much there without anything fancier.