If you have a friend with a costco membership, you can ask them to buy you some $25 Costco Shopcards, which you can use at the entrance to get into the store and at checkout to verify you can make a purchase. Anything you have over $25, they’ll allow you to use an alternative method of payment to pay for it.
They make you turn in the card at checkout, so it’s certainly not worth it if you go more than a handful of times a year, but for those who go less, it’s a good way of avoiding the membership fee.
You should look for ethnic groceries. They often have bulk spice real cheap.
I have an Indian grocer who sells basically all spices (and some common blends) in 1lb bags for a couple dollars.
Amish groceries, if you live near any Amish communities, are really good for that as well.