The stronger view is that semen is pure (ṭāhir): Aisha reported that she would wash it from the Prophet's garment when it was wet and simply scratch it off when dry, after which he would pray in that same garment — and scraping does not purify a truly najis substance the way washing does. Some scholars (Hanafi and Maliki) held that it is najis but that scratching when dry is enough, so there is a recognized ikhtilaf, yet the act of merely rubbing it off points to its purity. Note that this question is separate from ghusl: emitting semen with desire obligates a full ghusl regardless of the ruling on its physical purity.
Evidence — the daleel
On deen2u
Informational, not a personal fatwa. Consult a qualified scholar for rulings on your situation.