JevMade

Sign in
← Back to guides

JevMade field notes / Written guide

How computers learned to sort text

Sebastian Raschka explains the history of sorting text into categories and reports his own Jev test on movie reviews.

Original by Sebastian Raschka / Ahead of AIClassification
Bookmark: How computers learned to sort text Keep this in your collection.
Leave a noteWhat would you try with this? : How computers learned to sort text

Only you can see your notes.

Listen to this guide

JevMade’s plain-English explanation

0:00 /

Our summary

Sebastian Raschka traces how computers learned to put text into categories, such as positive and negative movie reviews. Early methods counted words without considering their order. Later models learned more about context. He explains why using one model across different tasks can avoid training a separate sorter each time.

Jev lets a program ask for a choice, a position on a scale, or an estimate of how likely a yes answer is. Raschka reports 96.47 percent accuracy when using Choice on 25,000 IMDb movie reviews. He also compares specialist models and explains how a similar answer format could be built.

Matching Jev's answer format does not establish equal performance across tasks. Its design and training details remain private, and Raschka labels his proposed explanations as guesses. Whether IMDb test reviews appeared in training is unknown. His results were not reproduced here, and numerical probabilities still need checks against real outcomes.

Key takeaways

  1. Word counts are inexpensive but ignore word order; later models use more context to choose a category.
  2. A general-purpose model can avoid task-specific training, while a specialist may still suit a repeated, high-volume job.
  3. Check probabilities against known outcomes. An attractive answer format and a high test score do not explain a model's private training.

Raschka's explanations of Jev's internal design and training method are guesses, not disclosed facts. Whether the IMDb test reviews appeared in training is unknown. His reported 96.47% Choice result was not independently reproduced.

magazine.sebastianraschka.com · Original published

Read the original guide Opens the author’s site in a new tab.

Your note

Only you can see this. Clear the text and save to remove it.

Keep what you save

Sign in to keep bookmarks and private notes on every device. We email you a link; there is no password, and the same link creates your account.