Fly fishing doesn't have to be a lot more expensive, really. That Eagle Claw glass rod for $24.99 is comparable in price to the cheaper spin rods out there, you can get something like a Shakespeare click pawl single action reel for $15, and 3M/Scientific Anglers makes a short fly line that Bi Mart sells for about $20. You'll need some dacron backing, but you can get the big spool of the 20lb dacron, that will fill MANY fly reels, or do double duty for sturgeon fishing - for about $7. That's about a $67 investment, which, while it's not a $19.99 el-cheapo Shakespeare special, is very reasonable. Part of the cost is the air that fly fishing is somehow superior - it's also a bit of a niche (a BIG niche, but still) type of fishing, compared to lure or bait fishing, and thus the lower demand for fly gear = higher costs. But you can get good fly gear inexpensively, if you don't HAVE to have big high dollar names on your rods. Sage and G.Loomis rods are great - but they're not needed to catch a fish. Kind of like the salmon/steelhead guys that get a hard on for Lamiglass and Loomis rods - you don't NEED such a rod to drift corkies and yarn, or eggs - even though some guys will swear you do. Fly fishing, like any other type of fishing, can be done as cheaply or expensively as you want.
I check the Goodwill and Salvation Army stores every time I go int there (probably once a week or so) because occasionally you'll find a real gem mixed in with the junk. There's also craigslist, forums like this one, and eBay. Cabela's Bargain Cave occasionally has some nifty fly gear in it too.
You really can get started in fly fishing for under a hundred bucks. Spend $70 on that Eagle Claw, or similarly priced outfit, buy a couple knotless tapered leaders (about $3-5 each), couple spools of Maxima leader material (again - about $3 a pop) and spend the rest on flies from someplace like StreamFlies.Com (fifty cents a fly, free shipping) and you'll be all set to get on the water and get some fish. You need not have exact anatomical matches for insects - you can get by with 3 fly patterns if you really, really want to pare down. Something like the Adams dry fly, in sizes 10, 12, and 14, then a Gold Ribbed Hare's Ear nymph, sizes 8, 10, and 12, and the ever neato Woolly Bugger (black or olive) in sizes 6, 8, 10 and you're set. If you don't care about dry fly fishing - you could nix the Adams, even ditch the Hare's Ear, and just buy a bunch of Woolly Buggers and get everything from trout and panfish to large and smallmouth bass, or salmon and steelhead.
One thing that helps, fly or gear fishing, is some waders - but they're not necessary. You don't need a vest either - a small fly box, forceps, nail clippers, and two spools of leader/tippet material will fit in a cargo pants pocket. The forceps can clamp onto your collar, and the nail clippers can be put on one of the little zipper retractable pins (about $3).
Like I said - you can spend as little money as you can get away with, or spend boat loads, you can go light, or carry every piece of tackle known to man. You'll catch fish either way. The biggest learning curve in fly fishing, is learning the basic casting strokes, but that just takes a little time.