A fraud detection model finds fraudulent sellers on an e-commerce platform from text. It combines NLP (sentence embeddings) with a standard ML classifier (XGBoost).

Scenario

Fraudulent sellers commit 2 types of fraud, and each type leaves a trace in a different text source:

Fraud typeText data source
Fulfilment fraudProduct reviews written by buyers
Off-platform fraudMessages sent from the seller to the buyer

Fulfilment fraud model

graph TD
    A[Product reviews] --> B[Sentence Transformer]
    B --> C[Sentence embeddings]
    C --> D[Aggregate at seller level]
    D --> E[XGBoost model]
    F[Seller features] --> E
    E --> G[Fulfilment fraud score]

Steps of the pipeline:

  • a Sentence Transformer (a popular free one comes from Hugging Face) turns each review sentence into a high-dimensional embedding
  • the embedding size can be chosen, and popular sizes are 512, 768 and 1024
  • the embeddings are aggregated at seller level, because the model judges sellers, not single reviews
  • XGBoost is the supervised classifier, because it handles embedding data well and gives an efficiency gain when the data is sparse
  • extra seller features can be added to improve the model

Downstream process

The model does not punish anyone by itself. Humans confirm every case:

graph TD
    A[Fraud model] -->|high risk sellers| B[Level 1 reviewer]
    B -->|second review| C[Level 2 reviewer]
    C -->|confirmed fraudsters| D[Seller punishment]
  • 2 levels of reviewers are used to make sure the decision is correct
  • the punishment depends on how severe the fraud is

Remembering tip

Review, Embed, Group, Boost: reviews go in, sentence embeddings come out, they are grouped per seller, then XGBoost scores the seller.

A bank uses a similar mix of embeddings and ML in the Human Trafficking Detection Model.