Hi,
You need to have at least one Price list created in Settings --> Product Catalog --> Price lists. Open any of the Price list record. From the left navigation, open the list of Price list Items. Click "Add New Price list item" from the ribbon. In the
new Price list Item window, select each of those 4000 products one by one. Unfortunately, this would be a manual process.
You need to use Import\Export feature if you are looking towards bulk updates. You need to follow below steps.
1. Create a Price list record.
2. Export 4000 record having Pricelist column so that you can import it again.
3. Open the exported file in Excel, set name of the Pricelist record in the Pricelist column. Save and Close
4. Delete all those 4000 products from the system. (Do not delete if these products are used by Orders/Opportunity records)
5. Import the file again.
Above steps are not recommended If you have other Sales related data in the system.
Please be careful to ensure that No records are linked to existing products before deleting them. If you are not sure then it would be good to go with the manual process otherwise you may land up in disturbing the data integrity.
Hope this helps.
Ajay Chawla