Leggy Sage and Other Seedlings

I am new to growing culinary herbs this year, and actually started sage, thyme, and oregano indoors around the 3rd this month (zone 5/6). Since I don't have any special lights the seedlings are getting quite leggy, but I am wondering, after frost is it possible to bury the leggy stems in the soil right up to the first set of leaves? I know this trick works for tomatoes but as for herbs I am not so sure.

No, most plants cannot be planted deeper. Tomatoes are an exception because they have the ability to develop adventitious roots along the stems. But this is a rare ability. The stems of most other plants planted too deeply would eventially rot.
You have to find a way to increase the light intensity. Grow lights can help supplement the natural light coming through a window and I recommend them. Lowering the temperature to 10-15 degrees can help slow growth and better match it to the available light. But increasing the light intensity is the best route.