Menu ▾ ▴

#81 Check %sp too in dispatcher for stack overflow

1.0
open
nobody
None
False
531:fdb5a06b0b37
2016-05-25
2016-05-23
Brian Ruthven
No

Whilst messing around with SHA512 code, I blew out the stack, but it was only detected when the redzone byte happened to be overwritten. The kernel should check %sp as well, ensuring it is bounded between t_stkbase and t_stklim. At least then we get a panic closer to the point where we left the stack area for that thread.

Discussion

  • Brian Ruthven

    Brian Ruthven - 2016-05-23
    • Description has changed:

    Diff:

    --- old
+++ new
@@ -1 +1 @@
-Whilst messing around with SHA512 code, I blew out the stack, but only when the redzone byte happened to be overwritten. The kernel should check %sp as well, ensuring it is bounded between t_stkbase and t_stklim.
+Whilst messing around with SHA512 code, I blew out the stack, but it was only detected when the redzone byte happened to be overwritten. The kernel should check %sp as well, ensuring it is bounded between t_stkbase and t_stklim. At least then we get a panic closer to the point where we left the stack area for that thread.
    • Found in: --> 531:fdb5a06b0b37
     

  • Brian Ruthven

    Brian Ruthven - 2016-05-25

    Actually, it turned out that I was overflowing the results buffer and therefore corrupting lower down the stack, particularly the function return address, and corruption ensued from there onwards. I'm going to shelve this as a "nice to have" for now, as it adds more to the dispatcher codepath during context switch.

     

Log in to post a comment.

MongoDB Logo MongoDB