Finishing the Business Side
In addition to registering with the state, you may need to get a federal ID number (unless you decided to operate as a sole proprietorship, in which case, your social security number is adequate), a local business license, and a sales tax number.
If you intend to have employees, you'll have to get a lot more rigmarole. I advise you to stay away from employees, at least at first, and just use contract labor.
You'll need to set up a bank account for the business and open a merchant account so you can accept credit cards. Although you probably can set up the merchant account with your bank, it is likely that you can find better rates on the Internet.
Finally, you need to set up your workspace, and purchase or upgrade your equipment and software. This doesn't mean you have to rent an office somewhere. You can work out of your home and meet clients at their locations or in public places, such as restaurants or coffee houses. It's done all the time and is perfectly acceptable to clients.