X-Recipient: archive-cygwin AT delorie DOT com X-SWARE-Spam-Status: No, hits=0.1 required=5.0 tests=AWL,BAYES_00,RCVD_IN_DNSWL_NONE,SARE_SPEC_LEO_DOLLARSa,TW_AV,T_FRT_BELOW2 X-Spam-Check-By: sourceware.org X-Yahoo-SMTP: ycweUreswBCK.d0cygTP5tXwHncbOU7YVeVfIxOQoyRMI2IuIKLmUqE- Message-ID: <4F68B020.8090404@molconn.com> Date: Tue, 20 Mar 2012 12:28:16 -0400 From: LMH User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:10.0.2) Gecko/20120216 Firefox/10.0.2 SeaMonkey/2.7.2 MIME-Version: 1.0 To: cygwin AT cygwin DOT com Subject: Re: rsync ERROR: out of memory in flist_expand [sender] References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit X-IsSubscribed: yes Mailing-List: contact cygwin-help AT cygwin DOT com; run by ezmlm List-Id: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: cygwin-owner AT cygwin DOT com Mail-Followup-To: cygwin AT cygwin DOT com Delivered-To: mailing list cygwin AT cygwin DOT com If you are continually copying large numbers of files with rsync, maybe you would be better off just using cp. Do you need to keep older versions with the same file name, meaning an incremental backup? I generally start by just doing a cp of the src directory to the backup location. When the number of files is large, I just do, cp -Rfp /src_loc_path/ /backup_loc_path/ &> copylog This gives me a record of anything that might not have been copied due to permission issues, etc. It's best to do this from a windows admin account. Then I run rsync on subsequent days so that rsync is just dealing with changes made on any one given day and not trying to copy very large directories. I always use a log file so I can check to make sure it has finished and got everything it was supposed to. #!/bin/bash function backup { # write some stuff to the log file echo " " >> $1 echo " " >> $1 echo "***** new log entry *****" >> $1 date >> $logfile_path # run the backup rsync -v -a -r -p -b --suffix=.back -E -t --log-file=$1 $2 $3 # convert the log file format to windows EOL unix2dos $1 } # give a location for the log file, src, and backup loc logfile_path="/logfile_path/log.txt" backup_src_path="/backup_src_path/" backup_destination_path="/backup_destination_path/" backup $logfile_path $backup_src_path $backup_destination_path I just have this script attached to a windows shortcut. That way I can run it with a double click, or add the shortcut to the task manager if I want to schedule backups. I guess I should be using cron to get the full quantity of linux points, but I haven't got around to it. If you wanted to, you could make a script that would check if the backup destination directory exists, and if it doesn't, just do cp. If it does exist, then run rsync. I don't know how much volume it rsync designed to manage, especially if it needs to keep a connection over a net, but if you are running out of resources you may want to look at a different tool. There are other options, like rdiff and unison. I have used unison, but it is more a mirroring tool and I found the incremental backup options gave me errors. I you are just trying to mirror directories or machines, then it is a good tool. LMH Richard Ivarson wrote: > Bruno, is this rsync problem maybe related to my rsync problem which I > mentioned recently in the thread "Rsync stops inmid of synchronisation" ? > > (My latest post there was: http://article.gmane.org/gmane.os.cygwin/132270 > ... but I don't know how to link to the entire thread.) > > -Richard > > > > Am 19.03.2012 11:51, schrieb Bruno Galindro da Costa: >> Hi! >> >> I'm try to copy some files from windows to Linux using rsync but, >> after some short of time, an error was showed. Here is the log: >> >> ERROR: out of memory in flist_expand [sender] >> rsync error: error allocating core memory buffers (code 22) at >> /home/lapo/package/rsync-3.0.9-1/src/rsync-3.0.9/util.c(117) >> [sender=3.0.9] >> rsync: connection unexpectedly closed (35795723020 bytes received so >> far) [receiver] >> rsync error: error in rsync protocol data stream (code 12) at >> io.c(601) [receiver=3.0.7] >> rsync: connection unexpectedly closed (102 bytes received so far) >> [generator] >> rsync error: error allocating core memory buffers (code 22) at >> io.c(601) [generator=3.0.7] >> >> Bellow is the command that I used: >> >> /usr/bin/rsync >> --exclude-from=/opt/backup_ti/tmp/FILESERVER_exclude_from.tmp >> -avzR --delete-excluded --rsh="ssh -o UserKnownHostsFile=/dev/null -o >> StrictHostKeyChecking=no -p 31060" >> root AT 127 DOT 0 DOT 0 DOT 1:"/cygdrive/c/fileserver" >> /backup/fileserver/bkp_files_20120318225522 >> >> Bellow is the /opt/backup_ti/tmp/FILESERVER_exclude_from.tmp content: >> >> - TEMP >> - RECYCLE.BIN >> - System >> >> Is there any workaround that I can use to "solve" this problem? >> Maybe using an option or change a configuration file on cygwin... >> >> Search in google, I've found this solution >> (http://blog.rudnyi.ru/2011/04/problem-with-rsync.html ), but, before >> apply that, I like to see If there is another option to use. >> >> -- >> Att. >> Bruno Galindro da Costa > > > -- > Problem reports: http://cygwin.com/problems.html > FAQ: http://cygwin.com/faq/ > Documentation: http://cygwin.com/docs.html > Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple > > -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple