anil.org.in » Technology

Netcat (nc): “TCP/IP Swiss Army Knife”

anil — Tue, 26 May 2009 06:31:28 +0000

Many of us use telnet as a network debugging tool quite often. Just with its hostname and port options we find it very useful. But, I’m sure that many would recall following situations where we find the telnet client inadequate:

To test a UDP service
Want to send a pre-prepared set of request/responses to a server (to test a service)
Specify a Gateway for the TCP or UDP connection (checking routing and firewalling)
Receive and store dump of the traffic (network and application troubleshooting)
Send a UDP broadcast over the subnet (Network and application testing)
To set the ToS (type of service) flag in IP packet (to test QoS settings)
To copy a file over network without any regular services running
Do a port scan (without opening I/O)
Mimic the request/response sequence of a service/client you want to test

There is a less-known tool called nc–acronym for Netcat–which was ranked 4th in the latest Top 100 Network Security Tools 2006 conducted by Insecure.org and always in the top 5 in previous years as well.

nc can perform all the above and more. As it is rightly assigned with the caption of “TCP/IP swiss army knife.” Here is a quick set of commands to use nc for the use of a network administrator/troubleshooter.

Please note that Linux and BSD ports of nc differs in behaviour and syntax. This post is mostly based on the Linux port. I have tried to mention the difference where ever possible.

Test a TCP Service:
nc
e.g.: nc www.google.com 80

If you need to run a shell command after the TCP connection is established, use the “-c” option.

nc -c wwwclient www.google.com 80

There is an important behaviour you need to notice while using -c and -e (invokes a script) options: the called command is supposed to handle both input and output of nc. This means that while passing an echo command, for example, with -c option, you cannot expect nc to print the response to standard out. For example, the following command writes nothing to standard out:

nc -c “echo GET / HTTP/1.0″ www.google.com 80

This is because nc pipes the HTTP response from www.google.com to echo. The -e option has the same behaviour. In the BSD port of nc there is no -c option available. And the -e has a very different meaning.

If you want to make a request to a server in two or more lines (as in the case of, say, HTTP protocol), write them down in a file and cat-pipe that file to NC. For example if you want to find the public IP address of your Internet gateway, create a text file http.txt with the following content:
GET /ip HTTP/1.0 Host: www.linuxense.com

Now issue this command:

cat http.txt|nc www.linuxense.com 80

In certain protocols, such as SMTP, it needs to wait for the server to make a response before nc sends the next request. In such situation the -i option comes handy; it waits for the specified number of seconds before it sends the next line.

Now try this SMTP transaction. Save the following in to a file named, say, smtp.txt
helo greetings mail from: rcpt to: data Subject: test from Netcat
Hello, This is a test from Netcat See ya! ~me .

Now play out this transaction:

cat smtp.txt | nc -i 2 your_smtp_server 25

See the -i in action.

There is a -n option that disables any DNS look up. This is useful if you are providing the IP address of the host and to explicitly say so to nc.

Use -p option to specify a source port. It is even possible to specify a range of port as in 80-1024 (both inclusive). If you omit the -p option, it will use a random port.
Emulate TCP Service
Use the -l flag to instruct nc to stay in listen mode. Use the following to bind nc to port 8000:

nc -l 8000

Normally nc quits when the remote connection closes. To make it stay listening for another connection, use -k (found to work only in BSD port).

Here -p option to specify the port it is listening to is allowed in Linux. In the BSD port, this is illegal.
Emulate a UDP Service
Use the -u option, nc will turn the protocol to UDP. Almost all options which are valid for TCP mode (default mode) are valid with this option too.

Emulate a DNS service:

nc -u -l -p 53 (note that all parameters are similar to those in TCP mode. The option -u makes all the difference).

Here, the -p is a required option in Linux where us in BSD it is not permitted.

Now, run a dig command at it:

dig @localhost www.linuxense.com

You will see a partially readable line being printed on the console and this is how a DNS query looks like. Capture this into a file:

nc -u -i -p 53 > dns-query.txt

And playback it at a functional DNS server and see the output:

cat dns-query.txt |nc -u mydnsserver 53
Specifying Source IP Address:

If you are on your gateway and you want to specify the source IP address of packets leaving the gateway, use the -s option. For example:

nc -s 192.168.1.1 remotehost 80
Run a Zero I/O Port Scan
See this example:
nc -z -v remotehost 80
The -z runs nc in zero I/O scan mode. -v option is to turn on the verbose mode. It is also possible to specify a range of IP addresses as follows (both IP addresses are inclusive): nc -zv remotehost 80-1024


Set "Type of Service" (ToS) Flag
The flag -x  allows to set the IP ToS flag. Possible values are "Minimize-Delay", "Maximize-Throughput", Maximize-Reliability", "Minimize-Cost".

Set Up a Service Gateway/Proxy
This is a quick and trivial service gateway you can try out in a few seconds. You device more sophisticated gateways with nc. BSD port of nc provides built-in support for this functionality with -X and connect verb.


If you enjoy reading this post and liked nc, please post here the nc tricks and hacks you invent.
--

Related post:

Learn IP Networking
Further reading:

BSD nc man page

GNU Netcat official homepage



Best Reference Site for Each Programming Language
anil — Sun, 21 Sep 2008 18:37:28 +0000
Inspired from a Slashdot post, I thought of preparing a list of best reference web sites for each programming languages that we use at Linuxense. Send in if your suggestions.


 C

http://www.acm.uiuc.edu/webmonkeys/book/c_guide/


C++

http://www.research.att.com/~bs/C++.html



Java

http://java.sun.com


Javascript

http://developer.mozilla.org/en/JavaScript


Lua

http://www.lua.org/


Pascal

http://www.freepascal.org/docs-html/ref/ref.html


Perl

http://perldoc.perl.org/
http://www.perl.org/
http://cpan.org/
http://www.perlmonks.org/


PHP

http://www.php.net


Python

http://python.org/


Ruby

http://www.ruby-lang.org/en/documentation/


Scheme

http://groups.csail.mit.edu/mac/projects/scheme/


TCL

http://en.wikibooks.org/wiki/Programming:Tcl



–

General Programming

Structure and Interpretation of Computer Programs


Regular Expression

http://www.regular-expressions.info/






IPTV: The Next Big Thing?
anil — Thu, 12 Apr 2007 07:31:50 +0000
As a concept IPTV has been around for a while. Everybody talks about it. Everybody wants to subscribe to IPTV service. But companies are yet to wet their hands. Those who are new this new method of television content delivery can check out the Wiki page.
Players in India
All those companies who extended a piece of wire (“last mile” connectivity) to homes can be a potential provider of IPTV service in its broadest sense. So your cable provider, wired/land phone service provider or even the state electricity board can enter into IPTV business.
Telecom companies such as Reliance which doesn’t have last mile connectivity to homes are fast lying fiber connections to homes these days. They are preparing for the IPTV market.
Actually these companies are looking at a broader spectrum of services called “Triple Play” (combining voice, data and video) or even “Quadruple Play” (where wireless communications is introduced as yet another media to deliver video). This way one provider can cater all communication needs of a home all by themselves (and get all possible revenues).
In India, predominantly land phone service companies (BSNL/MTNL, Reliance, etc.) and cable service providers (Hathway, Asianet, Sify, etc.) are the potential players. MTNL in Pune and BSNL in Calcutta have already launched IPTV services on experimental basis. Reliance, with their newly laid FTTP (Fiber to the Premises), seems to be ready to follow.
What IPTV Provides
With IPTV, subscribers can expect to get what they are currently getting with their normal cable TV/CAS system. In addition to that it can provide Video on Demand (VOD). Those who subscribe to Triple Play providers can enjoy broadband and VoIP as well–all through just one connection and one IP device. All from a single provider.
Which Player has the Advantage?
BSNL is far ahead of the rest in terms of number of homes and coverage/geographical area. But coper pair has its own limitation in terms of the bandwidth it can carry.
One channel at SDTV resolution will take up to 1.4 mbps with fairly good compression. Simultaneous delivery of channels is necessary to keep user demands. For example, this is required if a subscriber is using a DVR which can record one channel while another channel is being viewed on the TV. Sometimes subscriber might prefer to use picture-in-picture which, again, needs multiple channels to be delivered simultaneously.
BSNL employs ADSL. ADSL  can carry 2mbps which is too inadequate to support this. I’m not sure what technology they are using at Pune and Calcutta. One technology I can think of is ADSL2+ which can deliver up to 25mbps. But this bandwidth reduces substantially as the subscriber distance increases from the DSLAM.
Cable providers has an advantage here. HFC, theoretically, can carry up to 4.5gbps of data. It is a lot more than required to deliver multiple channels simultaneously even at HDTV resolution. So cable operators has a an edge over BSNL which uses coper as last mile.
Who Can Ride the Wave
Though people are waiting for the rollout of the service to sign up, business viability of IPTV service is yet to be proved. But if people receive it, IPTV can be a lucrative business. (It is possible to build several business models based on the viewer demography information IPTV system can provide. No such thing is possible with conventional cable.)
Those operators with HFC has the advantage of bandwidth capability. But HFC installations which are originally laid for analogue cable TV may fail to carry digital signals without quality degradation. Reliance with FTTP closely follows the above lot. Though their home count is less compared to that of cable TV operators, Reliance can provide better quality service due to obvious reasons.
Local cable operators are another category who can enter into this business. And thus they can move higher up in the value chain. Local cable operators can provide more localised content. They can stream targeted ads. But one thing which may prevent them from doing so; entry barrier of IPTV business.
High entry cost of IPTV business is constituted by IPTV/networking equipment costs and media rights. But local cable operators can buy media rights jointly. Equipment costs will continue to remain as a real entry barrier.
One good thing that can happen is the commoditisation of IPTV equipments including CMTS. If chip manufacturers can come up with solutions which are based on PC architecture, for example, rather than using specialised electronics (which drive up cost), this would become possible.
TRAI may consider unbundling BSNL’s last mile in future. If this happens local players can lease BSNL lines to run their business on.
Let’s wait and see.



Google Knows
anil — Sat, 17 Mar 2007 13:00:46 +0000
Now everyone is known to the world through an Email ID. Or if you know an Email address, you can trace down the person who owns it. That became magnificently possible with the social networking revolution on the Internet. But there is a limit to the extend of information you can dig out this way; to the extend the owner of the Email ID decides to expose about himself. We cannot go further down but Google does.
Roughly, one in every two people who searches the Internet uses Google and Google handles more than 200 million searches every day.
When you run a search in Google for the first time in your browser, Google will set a Cookie with a unique ID in your browser. There after, Google knows that all the searches you are running are coming from “you” and presumably they keep record of them all.
So, based on the searches you run on Google, they can build a “picture” about you–what you want, what you do, what’s bothering you, your interests, hobbies, and probably what are you planning to do this weekend. May be, after God (“Everything is uncovered and laid bare before the eyes of Him to whom we must give account” –Hebrews 4:13), Google knows you best. Now closest friend of man is not his wife or even a human being any more; it is Google (or Internet search engines in general). People share the most private thoughts with Google.
This is harmless till they link that “picture” with the real you. That happened with the introduction of Gmail. You might have noticed that even if you type gmail.com, you login to mail.google.com. The moment you successfully log in, they can associate the above-mentioned Cookie with your Email ID.
An Email ID in Gmail is not like yet another ID in Hotmail or Yahoo. Each Email ID has the “genealogy” information attached to it. Each Gmail ID is created out of an invite sent out by an existing Gmail ID. So it is almost impossible to disguise yourself before Google. Besides knowing who you are, Google also knows whom you are related to. For example Google knows that you are Ram, husband of Meena and friend of John.
Google even knows what you do out side Google to a certain extent. For example when you sign up for a new third-party service (with your Gmail ID), Google knows that you did so. They even keep track of the links you click in Emails and chat texts.
They are the best in the world for text parsing and analysing. And if we closely examine we can see that that makes the core of all their technologies. Google can even analyse your chat live. So think twice next time you chat with your friend on Google to discuss a new business idea or when you use Google Documents to type a new business plan. The point is that Google is a privacy bomb which can blow up any time either though a security breach or by the data mining bureaucrats of Washington.
Google, after handling trillions of queries from around the world by now, might have an interesting picture about the people in this world, I’m sure. Nobody has ever got a chance to see the world at such a macro level yet.