These days, it is generally accepted that before a new name can enter general use, it should be clearly established in some form of formal, widely-accessible publication just to what it is that the name is supposed to refer. Back in the day, however, this was not always the case. A century or two ago, the communities of researchers working on a particular group of organisms were often small, and it was not uncommon for names to effectively spread through personal correspondence or word of mouth alone. One naturalist might refer to a new genus he had come to recognise in a letter to another, and the latter naturalist may then assign his own species to that genus without the first naturalist ever publishing a formal description. At the time, this might not be seen as much of an issue: after all, if there was ever any question as to the first naturalist's original intent, surely it could be clarified by simply writing to him personally?
The name Leptothyra seems to have been established in this kind of way in the mid-1800s by the American naturalist James Graham Cooper for a genus of small marine gastropods (belonging to the vetigastropods, related to the top shells and cat's-eyes) found on the coast of California. In 1871, W. H. Dall attributed the name to an unpublished manuscript of Cooper's and cited the type species as Linnaeus' Turbo sanguineus, a Mediterranean species to which Cooper had also attributed specimens from the Pacific. As it happens, Turbo sanguineus was already the type species for an earlier genus name, Homalopoma, so Dall's 'Leptothyra' would be considered invalid and give precedence to Homalopoma. However, in 1869 the name Leptothyra had been used by W. H. Pease for L. costata, a species from Hawaii, without direct reference to any other species (thus making L. costata the effective type species of Leptothyra). Subsequent authors often considered 'Leptothyra Pease' to be a separate genus from Homalopoma/'Leptothyra Dall'. At least one author who did not, Henry A. Pilsbry (1888), nevertheless used the name Leptothyra under the mistaken belief that the name Homalopoma was preoccupied. Over time, numerous species both living and fossil from around the Pacific were assigned to Leptothyra in one way or another.
It was not until over a century later that Coan (1986) pointed out that the name Leptothyra had appeared in print even earlier than Pease's usage. Cooper himself had used the name in a list of Californian molluscs in 1867. Even though Cooper's list lacked any descriptive details, this counts as enough to validate the name because he included species for which descriptions had already been published under other genera. One of these was Turbo sanguineus, which Coan officially designated as type species and fixed Leptothyra's status as an invalid later name for Homalopoma.
Which leaves open the question of what one should call the genus formerly known as 'Leptothyra Pease'. It doesn't help matters that Pease's 'Leptothyra costata' has apparently never been illustrated and its identity has been open to question. Iredale (1918) proposed the name Collonista for use with species previously included in Leptothyra, stating that the latter "proves to have been first published by Pease in connexion with a juvenile shell of a different genus", but gave no further elaboration or explanation how he reached that conclusion. The online resource WoRMS lists L. costata as a junior synonym of the widespread Pacific species Collonista verruca, but I have been unable to find where that synonymy was published. Nevertheless, any sort of replacement name for 'Leptothyra Pease' seems like it would be misguided at best. It is unlikely that L. costata represents any genus otherwise unknown and, even without any explicit statement to the effect, it is quite possible that Pease only intended to assign his species to Cooper's manuscript genus rather than establish a new genus of his own. Any concept of a genus Leptothyra is best left to sink into the annals of history.
Coan, E. 1986. Some additional taxonomic unites that first appear in publications by J. G. Cooper. Nautilus 100 (1): 30–32.
Cooper, J. G. 1867. Geographical catalogue of the Mollusca found west of the Rocky Mountains, between latitudes 33° and 49° north. Geological Survey of California: San Francisco.
Dall, W. H. 1871. Descriptions of sixty new forms of mollusks from the west coast of North America and the North Pacific Ocean, with notes on others already described. American Journal of Conchology 7 (2): 93–160, pls 13–16.
Iredale, T. 1918. Molluscan nomenclatural problems and solutions.—No. 1. Proceedings of the Malacological Society of London 13 (1–2): 28–40.
Pease, W. H. 1869. Descriptions of new species of marine Gasteropodae inhabiting Polynesia. American Journal of Conchology 5 (2): 64–79.