A fine-tuned 2015 CNN beats GPT-6 Astra and Claude Fable 5.1 at naming skin lesions, a dermatologist reports

Dr Magnus Lynch, a consultant dermatologist at Guy's Hospital and King's College London, published a test of GPT-6 Astra and Claude Fable 5.1 on 200 biopsy-proven lesions drawn at random from the public MILK10k dataset. Each model got a clinical photograph and a dermoscopic image, the same prompt and default settings, and chose among six diagnoses. The comparison model was Inception v3 (published 2015), fine-tuned on 68,011 images from the ISIC Archive on one H100 GPU, with the test lesions removed from its training data.

His figures:

• Correct first choice: the CNN 125 of 200 (62%); GPT-6 Astra 84 and Claude Fable 5.1 85 (42% each).
• Correct in first or second choice: CNN 173; the LLMs 134 and 138.
• Melanoma as first choice (of 30): CNN 13; the LLMs 8 and 11.
• Melanomas judged safe to discharge (of 30): GPT-6 Astra 3, Claude Fable 5.1 4, the CNN 18 at its preset threshold (1 or 2 when its threshold was matched to the LLMs').
• The models failed differently: GPT-6 Astra called 75 lesions basal cell carcinoma when 25 were, and Claude Fable 5.1 called 53 actinic keratosis when 20 were.

Asked as a patient with the clinical photograph alone ("do I need to see a doctor?"), both LLMs advised seeing a doctor for 99 and 98 of 100 further melanomas.

The author's caveats: only 200 lesions and 30 melanomas, each LLM case run once, every lesion biopsied (so harder than typical), and the dataset has been public since 2025, so the LLMs may have seen it. Inception v3 is a baseline, not state of the art; he notes regulated skin-cancer products perform considerably better. He says analysis and first drafts were made with Claude Code. The results are one clinician's test, not a peer-reviewed study. The post had 374 points and 182 comments on Hacker News by October 25.