A soldering iron or woodburning pen will help to smooth out the whittling marks from the box cutter.
Really, the phone's not tooooo difficult to take apart. I'm sure your warranty's already void by drilling there, can't hurt. If you disassemble it you can pull the headphone jack itself out and then properly drill into the hole of the case without worry of damaging the jack itself.
Good luck and thanks for your efforts and videos!