The preview pane shows PDFs with aliasing. Some platforms (e.g., OSX) have APIs/widgets for rendering PDFs with anti-aliasing, which look far better. This ticket requests that feature.
Since TXS is cross-platform, we cannot use a Platform-dependent PDF-API (at least not with reasonable effort). The PDF-Rendering in TXS is performed using the poppler library, which supports antialiasing and should have it enabled, though I currently cannot check that on OSX. In Windows it works reasonably well. Could you post a screenshot (ideally two, comparing TXS on OSX and some other pdf viewer with antialiazing). I'd like to see how bad it is.