Are underlines used in e-Sword kept in markup.ovlx?
Yep.
Not only that, so too are Highlights stored in the same file. That said, you can also create new markup files and give them a unique filename (pretty much the same way one can with topic, study, and journal Note files).