X-Recipient: archive-cygwin AT delorie DOT com
X-SWARE-Spam-Status: No, hits=-1.6 required=5.0	tests=AWL,BAYES_00,MAY_BE_FORGED,RCVD_IN_DNSWL_NONE,TW_NV,T_RP_MATCHES_RCVD
X-Spam-Check-By: sourceware.org
Message-ID: <4D81CED1.9050601@xs4all.nl>
Date: Thu, 17 Mar 2011 10:05:21 +0100
From: Erwin Waterlander <waterlan AT xs4all DOT nl>
User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.2.15) Gecko/20110303 Thunderbird/3.1.9
MIME-Version: 1.0
To: cygwin-apps AT cygwin DOT com, cygwin AT cygwin DOT com
Subject: Re: ITP dos2unix 5.2.1-1
References: <4D7FE2A7 DOT 8080409 AT xs4all DOT nl> <4D7FE57A DOT 4020903 AT redhat DOT com> <4D806DCF DOT 5090803 AT xs4all DOT nl> <4D80C0B8 DOT 8090603 AT cwilson DOT fastmail DOT fm> <4D80CEB4 DOT 7090005 AT xs4all DOT nl> <20110316154913 DOT GA18995 AT ednor DOT casa DOT cgf DOT cx> <4D80DEAF DOT 6020307 AT xs4all DOT nl> <4D810FAF DOT 6040609 AT t-online DOT de> <20110316193249 DOT GA15365 AT ednor DOT casa DOT cgf DOT cx> <4D8117FD DOT 40507 AT cwilson DOT fastmail DOT fm> <20110316201821 DOT GA773 AT ednor DOT casa DOT cgf DOT cx> <4D812DBE DOT 1090607 AT cwilson DOT fastmail DOT fm>
In-Reply-To: <4D812DBE.1090607@cwilson.fastmail.fm>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-IsSubscribed: yes
Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm
List-Id: <cygwin.cygwin.com>
List-Subscribe: <mailto:cygwin-subscribe AT cygwin DOT com>
List-Archive: <http://sourceware.org/ml/cygwin/>
List-Post: <mailto:cygwin AT cygwin DOT com>
List-Help: <mailto:cygwin-help AT cygwin DOT com>, <http://sourceware.org/ml/#faqs>
Sender: cygwin-owner AT cygwin DOT com
Mail-Followup-To: cygwin AT cygwin DOT com
Delivered-To: mailing list cygwin AT cygwin DOT com

On 03/16/2011 10:38 PM, Charles Wilson wrote:
> Moved to main cygwin list for more feedback.
>
> Background: currently the following utilities
> 	unix2dos
> 	dos2unix
> 	u2d
> 	d2u
> are all provided by the cygutils package. They are, in fact, all
> hardlinks/copies of the same 'conv.exe' program, developed specifically
> for cygwin.
>
> We have a proposal to provide the "dos2unix" package instead, and
> removing these utilities from cygutils.  For more info, see:
> http://cygwin.com/ml/cygwin-apps/2011-03/msg00067.html
> and following.
>
>
> On 3/16/2011 4:18 PM, Christopher Faylor wrote:
>> Ah, right.
>>
>> I didn't get that Eric was referring to this scenario but, regardless,
>> I should have remembered that alternatives means using symlinks.  Given
>> the use that d2u is put, I don't think we want to make it a symlink.
>>
>> So, I guess that means that we poll the community if we want to go ahead
>> with this.  Either that or the binaries in the new package are given a
>> different name to avoid conflicts.
> Given that the unix2dos package supports pipe operation, the only real
> difference is the command line options (and following symlinks on
> converted files).
>
>
> So, compare:
>
> ================================================================
> unix2dos 5.1.1 (2010-08-18)
> Usage: unix2dos [-fhkLlqV] [-c convmode] [-o file ...] [-n infile
> outfile ...]
>   -c --convmode    conversion mode
>     convmode       ascii, 7bit, iso, mac, default to ascii
>   -f --force       force conversion of all files
>   -h --help        give this help
>   -k --keepdate    keep output file date
>   -L --license     display software license
>   -l --newline     add additional newline
>   -n --newfile     write to new file
>     infile         original file in new file mode
>     outfile        output file in new file mode
>   -o --oldfile     write to old file
>     file ...       files to convert in old file mode
>   -q --quiet       quiet mode, suppress all warnings
>                    always on in stdio mode
>   -V --version     display version number

That is an old version. Here is the latest:

dos2unix 5.2.1 (2011-03-04)
Usage: dos2unix [options] [file ...] [-n infile outfile ...]
  -ascii           convert only line breaks (default)
  -iso             conversion between DOS and ISO-8859-1 character set
    -1252          Use Windows code page 1252 (Western European)
    -437           Use DOS code page 437 (US) (default)
    -850           Use DOS code page 850 (Western European)
    -860           Use DOS code page 860 (Portuguese)
    -863           Use DOS code page 863 (French Canadian)
    -865           Use DOS code page 865 (Nordic)
  -7               Convert 8 bit characters to 7 bit space
  -c --convmode    conversion mode
    convmode       ascii, 7bit, iso, mac, default to ascii
  -f --force       force conversion of all files
  -h --help        give this help
  -k --keepdate    keep output file date
  -L --license     display software license
  -l --newline     add additional newline
  -n --newfile     write to new file
    infile         original file in new file mode
    outfile        output file in new file mode
  -o --oldfile     write to old file
    file ...       files to convert in old file mode
  -q --quiet       quiet mode, suppress all warnings
                   always on in stdio mode
  -V --version     display version number



> ================================================================
>
> vs.
>
> ================================================================
> unix2dos is part of cygutils version 1.4.4
>    converts the line endings of text files from
>    UNIX style (0x0a) to DOS style (0x0d 0x0a)
>
> Usage: unix2dos [OPTION...] [input file list...]
>
> Main options (not all may apply)
>    -A, --auto     Output format will be the opposite of the autodetected
> source
>                   format
>    -D, --u2d      Output will be in DOS format
>    --unix2dos     Output will be in DOS format
>    -U, --d2u      Output will be in UNIX format
>    --dos2unix     Output will be in UNIX format
>    --force        Ignore binary file detection
>    --safe         Do not modify binary files
>
> Help options
>    -?, --help     Show this help message
>    --usage        Display brief usage message
>    --version      Display version information
>    --license      Display licensing information
>
> Other arguments
>    [input file list...]       for each file listed, convert in place.
>                               If none specified, then use stdin/stdout
> ================================================================
>
>
> Now, of the cygutils options, we don't need
>    -D, --u2d, --unix2dos
>    -U, --d2u, --dos2unix
> because...those really only apply to the generic 'conv.exe' application.
> cygutils' unix2dos app is really only supposed to be used in, well,
> unix2dos mode!  So, unix2dos explicitly disallows the use of the
> "opposite" options (--dos2unix etc) and --auto; similarly dos2unix:
>
>    if (progtype == CT_UNIX2DOS) {
>      if (opts.ConvType != CT_UNIX2DOS) {
>        fprintf(stderr,
>                "%s: cannot accept any conversion type argument other\n" \
>
>
> Similarly, we don't need
>    -A, --auto
> because, again, it really only makes sense for the generic conv.exe app
> (and u2d/d2u/etc explicitly disallow its use).
>
> All that leaves are the informational options (--help, -?, --usage,
> --version, --license) which, if they go missing or renamed, I doubt that
> will break any scripts.
>
> Finally, we have
>    --force        Ignore binary file detection
>    --safe         Do not modify binary files
> All of cygutils' conversion progs operate in safe mode by default, so
> the --safe option is a bit redundant (it was added so that if somebody
> aliased unix2dos as 'unix2dos --force', they could override on the cmd
> line using --safe if necessary).
>
> The proposed package provides a similar --force option, but no --safe.
>
>
> So, the only real issues are
>
> 	1) somebody is (redundantly) specifying --safe

In my opinion --safe is redundant. If an administrator forced an alias  
with --force, you can always run /usr/bin/unix2dos, or create your own 
alias.

> 	2) how does the unix2dos "binary detection algorithm" differ
> 	   from cygutils'?

The new package considers a file binary if there are characters < 0x20, 
with the exception of line breaks, tabs, and form feeds.

> 	3) Does the unix2dos package handle setmode() on stdio properly?
I think so.
> 	4) Ditto for O_BINARY on physical files?
Ditto.
> 	5) cygutils always follows symlinks. This new package does not,
> 	   unless --force, according to the man page (which is
> 	   unfortunate: the same option means "follow symlinks" AND
> 	   "convert everything even if you think it is binary"
The new package does not follow symlinks, so you don't damage files on 
other locations. If you force conversion a copy is created. The symlink 
target remains unmodified. There is not a separate option to force 
conversion of symlinks. Perhaps the creation of copies should be default.

> 	6) dos2unix foo.h foo.c (cygutils), but according to the manpage
> 	     dos2unix -o foo.h foo.c (new package)
> 	   However, experimentation shows the man page is wrong.
> 	     dos2unix foo.h foo.c (new package)
> 	   works fine.

The option -o (old file mode) works fine. It is the default mode, so you 
can skip it. You only need it if you used 'paired' conversion (option 
-n) on the same command-line.

dos2unix -n in.txt out.txt -o inplace.txt


> 	7) Output messages:
> 	   cygutils: $ dos2unix bob.c bob.h
> 		       bob.c: done.
> 		       bob.h: done.
> 		on stderr
>
> 	   newpkg:   $ dos2unix bob.c bob.h
> 		     dos2unix: converting file bob.c to UNIX format ...
> 		     dos2unix: converting file bob.h to UNIX format ...
> 		on stderr
> 	8) output status -- both packages report '0' on success, where
> 	   success means "nothing went wrong".  That is, if you convert
> 	   a file with unix LE using dos2unix, no changes are made to
> 	   the file AND 0 status is returned.  Again, this behavior is
> 	   the same on both pkgs, I just thought it was interesting.
>
> 		
> I think 2-4 are manageable (but need investigation), and *I* am willing
> to break users who are relying on --safe.  Or cygwin's version of the
> new package could be patched to add a --safe option.
>
> 5: if the new package is used, I think we should patch it to always
> follow symlinks (or add a new option, and make it default to follow).

I would propose a new option to follow symlinks. By default not follow, 
but copy (don't damage files on other locations).

Erwin

> 7: This might cause a problem if some script is relying on the output
> messages of the utility.  I think this is bad practice, and am willing
> to break people that are doing this.
>
>
>
> So: I don't see any insurmountable problems with entirely replacing
> cygutils' unix2dos/dos2unix/u2d/d2u programs with these other versions.
>   (I would hope that cygwin's package would provide a u2d.exe hardlinked
> to unix2dos.exe, etc)
>
> --
> Chuck


--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple