Blog Rss FeedBy changing the signature of your agent, ie: forcing the Referer: and by changing the signature of the User-Agent: you can have access to this protected content.
How to access this protected content with wget
You need to create a .wgetrc file in your home directory with setting the Referer and the User-Agent variable set with the signature of a well known search engineb
Exemple:
# .wgetrc file # Last change: 11-24-2006 header = Accept-Language: en header = Accept-Encoding: gzip, deflate header = Keep-Alive: 300 user_agent = Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html referer = http://www.google.com
Then when you get the page with wget you'll get this output:
$ wget http://www.example.com/protected_content.html
--14:30:23-- http://www.example.com/
=> `protected_content.html'
Resolving www.example.com... 172.16.1.66
Connecting to www.example.com|172.16.1.66|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 37,356 (36K) [text/html]
100%[====================================>] 37,356 100.31K/s
14:30:25 (100.05 KB/s) - `protected_content.html' saved [37356/37356]
I can't show you a real example here, it will be considered as stilling.
For more detail on HTTP protocol check the RFC-1954 and RFC-2068
For more information about what is the User-Agent, search for User-Agent on WikiPedia
Comments
Posted by: chat Nov 23, 2009 @ 03:26
After half an hour of Google-ing and applying a little horse-sense, I discovered (from counting the lines in a Windows GUI text editor) that the blank line was at the very end, right after the last perc ("%") delimiter. Pico, as has been my experience, loves adding those blanks. (Anticipating comments on the order of "Why not use [insert favorite shell editor here]": I can say that after the third time of not being able to quit out of vim or emacs in OS X's Terminal app, pico seemed like a breath of fresh air; for all it's attendant quirks and foibles, I still prefer it to the others.)
Posted by: sohbet Nov 23, 2009 @ 03:33
thanks you canım
Posted by: games Nov 27, 2009 @ 00:39
Works flawless.
Posted by: gmt master ii Dec 08, 2009 @ 00:32
Great article. Thanks!
Posted by: ugg boots Dec 11, 2009 @ 00:32
ok
Posted by: sell ugg ugg boots online ugg boots Dec 16, 2009 @ 18:54
1 Chestnut UGG Bailey Button Boots
2 Rose UGG Classic Cardy Boots 3 Sand UGG Classic Mini Boots 4 UGG Classic Argyle Knit whit 5879 5 UGG 5822 Stripe Cable Knit Chocolate White 6 Chestunt Genuine Australia classic Tall 7 Sand UGG Ultra Shotr Boots 8 Chestnut UGG Sundance Boots
Posted by: japanese flower delivery Dec 29, 2009 @ 04:14
hmm great idea!
Posted by: fake rolex watches Jan 04, 2010 @ 23:40
Welcome to our website,and you will enjoy the sunshine service. Let's
abandon all the worry,and enjoy the happy shopping travel.All the
replica watches paypal and replica watches are most in fashion
boutique this year.rolex replica and rolex replica paypal and the
replica rolex paypal show your chaming temperament. In addtin, the
fake watches paypal and the other fake rolex paypal make you mold
your unique glamour this winter.The swiss rolex replica have the high
quality but low price. replica watch paypal and replica rolex have
all the style and cheapest price replica rolex watches. Enjoying the
fun of net purchase! replica rolex We will supply you the perfect
service and after sale support system. Our honest will bring you
absolute satisfaction.
Posted by: film izle Jan 17, 2010 @ 04:42
Thanks for your good website and for sharing your experiences.
Posted by: divx film izle Jan 17, 2010 @ 04:44
Thank you for sharing your friends. Hope to see you another day.
Posted by: links of london Jan 19, 2010 @ 00:04
outique this year.rolex replica and rolex replica paypal and the
replica rolex paypal show your chaming temperament. In addtin, the
fake watches paypal and the other fake rolex paypal make you mold
your unique glamour this winter.The swiss rolex replica have th
Posted by: Ed Hardy Clothing Jan 19, 2010 @ 17:43
Exquisite design, superior material, exquisite craft. [HTML_REMOVED]Ed Hardy Clothing[HTML_REMOVED] Close to her heart, the most perfect expression of your love.
Posted by: kamilcan Jan 23, 2010 @ 13:25
very nice informations i have found you from google for my questions and i take the answers of my questions thanks for it have a nice day Regards.
[HTML_REMOVED]acı cehre[HTML_REMOVED] [HTML_REMOVED]vajinismus[HTML_REMOVED] [HTML_REMOVED]saç ekimi[HTML_REMOVED] [HTML_REMOVED]saç ekimi[HTML_REMOVED] [HTML_REMOVED]notebook[HTML_REMOVED]
Posted by: uvip33 Feb 05, 2010 @ 21:58
That’s something,That's what I was thinking.Brilliant idea.
Posted by: evening dresses Feb 19, 2010 @ 06:09
Thanks for your good website
Posted by: tag heuer watch Mar 04, 2010 @ 23:17
can show a man not only his wealthy but also his high taste.tag heuer watch As the society developed, many people are pay more attention to one's wearing than before, especially a watch. tag heuer watch If you are wearing a famous brand watch, people will look up you, also will love to do business with you. You will be a successful man with a such luxury fashion watch.
Posted by: omega moon watch Mar 11, 2010 @ 20:40
I recently came across your blog and have been reading along. I thought I would leave my first comment. I don't know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.
Posted by: MBT Chapa Mar 12, 2010 @ 23:03
can show a man not only his wealthy but also his high taste.tag heuer watch As the society developed, many people are pay more attention to one's wearing than before
Posted by: Sex Games Mar 28, 2010 @ 23:44
yes I do agree with all....... this is very interesting article for every one....
Thanks
Posted by: Data Recovery Mar 29, 2010 @ 06:52
Would this work within a Drupal interface?
Posted by: Data Recovery Mar 29, 2010 @ 06:53
Or...better still...how about cascade?
Posted by: Download Apr 03, 2010 @ 02:30
Thanks for sharing the information dude. I found the information very helpful. That's a awesome article you posted. I will come back to read some more.
Posted by: replica handbags Apr 04, 2010 @ 09:06
So, some tip about this issue are welcome and really sorry if my question is very simple. Thanks in advance Thanks again for your help. Your site contain a many useful information.
Posted by: installment loans for bad credit Apr 04, 2010 @ 12:23
I am so glad that I have found this your post because I have been searching for almost an hour on how to download protected web pages using wget. You helped me a lot indeed and reading this your entry I have found many new and useful information about this question. Well, I will definitely bookmark your website and wait for other useful posts in the future. Regards, Frank.
Posted by: Penny Stocks Apr 07, 2010 @ 05:54
Pretty good post. I just stumbled upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I'll be subscribing to your feed and I hope you post again soon.
Posted by: payday loans canada Apr 09, 2010 @ 12:37
This is some very valueable information, thank you very much. [HTML_REMOVED]online payday loans[HTML_REMOVED]
Posted by: cheap nfl jerseys Apr 15, 2010 @ 01:01
How to Buy NFL Throwback Jerseys.Travel to sports clothing stores in your local mall or shopping center to find throwback jerseys.Examine the different jerseys available through the National Football League's online shop before you buy.Locate autograph sessions and team reunions for your favorite franchise to increase the value of your throwback jersey. Contact sports collectors in your community who may possess rare throwback jerseys from the past.Customize your throwback jersey with your own last name and favorite number.Purchase a throwback jersey one size larger than normal if you plan on attending games or wearing the jersey around town. Coordinate an entire throwback uniform for your favorite team by purchasing matching hats and shirts. http://www.nflfansjersey.com nfl football jerseys
Posted by: chopard watch Apr 18, 2010 @ 23:12
chopard watches and replica chopard tend to be coated in a simulation silver or tag heuer gold and replica rolex filled with breitling watches a much cheaper rolex watche smetal to save on cost.
Posted by: wholesale electronics Apr 21, 2010 @ 22:06
Here have a variety of advanced technology to produce
electronic products, http://www.trademic.com Most wholesale electronics today use semiconductor
apparatus to accomplish electron control. The abstraction
of semiconductor wholesale electronics and accompanying
technology is advised a annex of physics, admitting the
architecture and architecture of cyberbanking circuits to
break applied problems appear beneath go to wholesale
electronics,engineering. This commodity focuses on
engineering aspects of electronics.wholesale electronics
is a able manufacturer, specializes in development,
manufacture. We accept broad Cheap Consumer Electronics,
China wholesale Electronics go!: Like Cheap Unlocked Cell
Phones, Hiphone, MP3 Players, MP4 Players, MP5 Players,
Computer Accessories, Car Electronics, Bluetooth Products,
USB Flash Drive, Memory Cards, Health & Beauty, iPod &
iPhone Accessories, Digital Photo Frames,Car Audios, Car
DVD Players, Digital Cameras, Digital Camcorders, Video
Game Accessories, Spy Cameras, Security Products and
Electronic Gadgets from China. We accept over 2500 accomplished and accomplished advisers
and our accessories of 7,300 aboveboard meters are able
with avant-garde productions and testing machines. We
accept the absolute superior ascendancy arrangement and
strive to accommodated and beat our customer's
expectations,wholesale electronics. Do you find mbt lami shoes for a long time. Here is the
richest in the world and you can find your needs bring you
harvest. yes,http://www.mbtsonsale.com/mbt-lami.html [HTML_REMOVED]mbt lami
shoes[HTML_REMOVED] Purple Shoes is the first physiological
footwear that has a positive effect on the whole body.
[HTML_REMOVED]buy mbt lami shoes[HTML_REMOVED] 's sole is curved, not
flat. Anti-shoe is unstable, not stable. Swissmasai
activates your muscles instead of undermining them. The
[HTML_REMOVED]mbt lami shoes[HTML_REMOVED] Birch Shoes Chapa feature
feature rich nubuck leather with single-layer mesh upper
bodies for strong but breathable wear,[HTML_REMOVED]mbt
lami[HTML_REMOVED].