Training, Ongoing Support
Most new sites require some amount of training and ongoing support, especially if your new site will be self-updateable. Define your expectations for training sessions, materials, and ongoing support after launch.
Key Staff, Company History
Have them provide a brief general history of their firm, as well as bios on the key personnel at the company. Have your prospect describe their previous experience in providing services to other companies similar to yours. Again, if you're close enough, a company visit would be a great idea. In our local area there are several companies that like to leave the impression through their website and marketing materials that they have a large staff but the reality is most of their "employees" are freelancers leaving them with limited control over production timelines, quality, etc.
Just like a car, websites need maintenance. New browsers and devices come out all the time and a website that worked when you launched it often develops issues over time if not upgraded and maintained. Be sure to define with your potential provider on how long after launch they plan to ensure the website functions and what any ongoing costs for maintenance might be, as well as, how they monitor your site for critical security and version updates that are common in today's CMS systems.
Before you launch a new website you will want to ensure that the site has been fully tested across multiple devices and screen sizes. If you have any special functionality, like a shopping cart or store locator, all of that needs to be tested by multiple users. In addition, all contact forms should be extensively tested to ensure they arrive in the correct mailbox at your company.
Are you looking for a time and material bid on your project or a fixed fee for the entire scope of the project? Define your expectations in the pricing section of the RFP. Ask the prospective contractor to break out or line item the project pricing as much as possible. This will help you ensure that they have given your project detailed consideration as to what it will take to actually complete the build.