Once you download them, you can install them as an extension: click Tools > Extension Manger. Then click Add and select the OXT files. They'll magically appear when you browse the templates: click File > New > Templates and Documents.
OOo now also provides an online gallery of templates. A link is also provided on the Templates and Documents dialog. You download these templates one by one. This is a great place to browse or search for very specific ones.
When you find one you want, right-click the Use it! button and select Save Target As. Save the template file directory to the template directory. The exact location might vary, but here's the general path:
In Windows Vista and 7: C:\Users\username\AppData\Roaming\OpenOffice.org\user\template\
In Windows XP: C:\Documents and Settings\username\ApplicationData\OpenOffice.org\user\template\
To see the AppData or ApplicationData directories, Windows must be set to show hidden files. Otherwise you could manually enter the path into Computer or My Computer.
For the exact path to the templates directory, click Tools > Options in OOo. Expand the OOo category, select Paths, and then see the Templates path.
There's still another option. You might be able to use the Microsoft templates as well. If you still have MS Office 2003 or earlier installed, you can probably find them at C:\Program Files\Microsoft Office\Templates\1033.
Later versions of MS Office use the new XML template format, which doesn't open well in OOo. If you have created your own templates in MS Office 2003 or earlier, you might also be able to use them.
You can still probably download and use the Microsoft online templates. Look for ones that support Word 2003. You can open these up directly in OOo.
You can optionally convert them into the OOo format by doing a Save As. To convert a batch of them in OOo, click File > Wizards > Document Converter.