One thing most certainly taking the fun out of your travel is if your luggage breaks during your trip. I myself had problems with luggage. One of my first “big” trips was a 4 week backpacking vacation on the west coast of the US. Two weeks into the trip zipper stopped working and by the 3rd week one of the carrying straps ripped off. Needless to say that lugging around a heavy backpack with one strap (which might also rip off any minute because of the additional weight) isn't very comfortable on the back (actually I should say on the front since I had to carry it on the front to support it with my hands). After this trip I went to an sports and outdoors store to get a backpack. Not just had they much higher quality backpacks but the sales person was actually very knowledgeable providing recommendations what pack works better for certain body sizes and how to adjust it properly so it actually doesn't hurt wearing a heavy backpack. I ended up buying a backpack from Deuter which was twice the price of my backpack that failed me. About 12 years and 37 countries later the Deuter backpack is still holding up well.
Over the years I have met a fair share of people where their luggage broke while traveling with them. Usually this meant spending time (aka wasting travel time) looking for a place to buy new luggage (most of the time only being ably to replace the broken item with another superior quality item that's doomed to fail). None of the luggage failures I observed have been with any of the reputable luggage brands.
Good luggage comes with good warranty
My Deuter backpack came with a three year warranty during which one of the small buckles broke. The company fixed it without any questions asked (even though they could have easily called it airline damage). There are many reputable companies that stand behind their products and offer several years of warranty that you don't have to pay for extra. The place I bought the first cheaper backpack from on the other hand initially wasn't even going to take the defective item back after just 5 weeks.
Durability and Brilliant Design
Besides quality luggage usually lasting much longer and therefore probably saving you money in the long run I also started to appreciate how well designed and thought through some travel luggage is. Sometimes it's the small things that make your trip so much more enjoyable. Your standard trekking backpack might have one big compartment that you can access from the top which might seem to be just fine. That's what a backpack is all about isn't it? Soon you'll realized though that you might have repack your pack every day since you can't access the content at the bottom though a different entry or you don't have any smaller compartments allowing you to organize your belongings (e. g. separate clean and worn clothing, have your smelly shoes in it's own compartment...).
Even if you think you are careful with your luggage and don't need to spend additional money as soon as it leaves your hands at the airport it's abuse is going to start. After every flight when you get your items at the baggage claim you'll most likely see additional scuffs, cuts and dents on your sturdy luggage which give you some idea what it went through.
Save Money with last seasons model
It doesn't always have to be so much more expensive. Often you can find a heavily reduced discontinued model.
Buy from a reputable place
One thing to keep in mind is that there is even counterfeit luggage out there which most likely isn't the same quality. Make sure you are buying from a reputable dealer. This might also make it easier if you have a warranty claim.
All this doesn't just apply for backpacks, but also holds up for your regular suitcase or even your carry-on. Usually for my business travels I can't show up with a big trekking backpack so I also have some business travel gear from Samsonite and High Sierra which is withstanding all the abuse when flying.