X-Authentication-Warning: delorie.com: mail set sender to geda-user-bounces using -f
X-Recipient: geda-user AT delorie DOT com
X-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        bh=Zi1CFTJMdzf10m/vH8GGaZYPJ+jPunZ3eBgadvR9AO8=;
        b=SqZEBguLFfPpoKnFQhg6q/LNxuDP9/MsUbTdP1OHH2tcnskNUJBZstYrAUTGxjZB2R
         DIjFlWRAi7WyDoHqtfeat7dqUu637igSOQmDdUo3/3FmKDNKdP9dMOetPKiHBnyiILhj
         pV4q+JTDzP7KozPTEJ6XUVro29ggPrXz81TZ+aDXv4sFtAP1neWA3DX5tQ77Ml5lGo9n
         14aWGVN0otrCK4lHkPb0t7BI172zROh8QqQKwAjvm9L+6JNwiK15EHVXqr1C5BcYL9ab
         JRvX9QisJ6qc1UHoWZ41dyeaBmEBqUverVyoQXMvECTLaWEZ3G6JGdllfhNdfpPYT6WY
         4kSg==
MIME-Version: 1.0
X-Received: by 10.194.173.233 with SMTP id bn9mr103911871wjc.1.1452036503571;
 Tue, 05 Jan 2016 15:28:23 -0800 (PST)
In-Reply-To: <CAJXU7q81_G+1ndudoWKBTnbJwEkMhpD42e2b5FoF7gJ8WwwozQ@mail.gmail.com>
References: <1512221837 DOT AA25291 AT ivan DOT Harhan DOT ORG>
	<CAJXU7q_qxdvJaejF-VcY=u7VHZ-zrfrc+Z7-qSwfFyPdy-umxw AT mail DOT gmail DOT com>
	<B02363CD-469D-493A-AC15-1D5DC7836982 AT noqsi DOT com>
	<20151222232230 DOT 12633 DOT qmail AT stuge DOT se>
	<0F6F1D0F-4F07-48EA-90FE-836EAD4E2354 AT noqsi DOT com>
	<CAM2RGhTficnys3a4xs=UBFvk8aPwpzYWUADFLP_pUQ+R1iKs0g AT mail DOT gmail DOT com>
	<0FCF3774-F93C-4BFF-BB61-636F75DCCACB AT noqsi DOT com>
	<CAC4O8c_UAiFE-vGfoE2tXppHLhaa0dSYz9o_rkdCBo7_SRRtxw AT mail DOT gmail DOT com>
	<FFBE7623-E240-4798-96B0-2BECF56C8E29 AT noqsi DOT com>
	<CAC4O8c980g1gj15=5njstC_BT-WYDgKQx9BRycdFKA8OvgtiOg AT mail DOT gmail DOT com>
	<B54C0E1F-1986-4C79-9F70-7F1919B8B26D AT noqsi DOT com>
	<CAC4O8c9bxJP1eMG4yz3YwKkQJRmsDGmLQ0aMd5pJRyu0WpdCtQ AT mail DOT gmail DOT com>
	<C1CFCCEE-C64A-4E49-AA64-446C061656D6 AT noqsi DOT com>
	<CAC4O8c-zt8B=joDd+ws77D2jt6aZf3MWfR_dAvpzGcNuBrTURQ AT mail DOT gmail DOT com>
	<alpine DOT DEB DOT 2 DOT 11 DOT 1601030040320 DOT 2176 AT newt>
	<D9825C8C-B6FD-4C7F-A8D5-B8AF06253B72 AT noqsi DOT com>
	<CAC4O8c_R5xWLmzj_cz0g0mPWNs6mR4efjXKGBoup8YO6nwnPTA AT mail DOT gmail DOT com>
	<A942261D-7C25-4F2D-9CB1-FFC60FA1C160 AT noqsi DOT com>
	<CAC4O8c8zk8=Py1yX6fVqF+35SYe39Li=y4jZ8bCeZ1Ev8WccAg AT mail DOT gmail DOT com>
	<20160105182120 DOT 3237F809D79B AT turkos DOT aspodata DOT se>
	<201601051829 DOT u05IT7TI021027 AT envy DOT delorie DOT com>
	<CAJXU7q81_G+1ndudoWKBTnbJwEkMhpD42e2b5FoF7gJ8WwwozQ AT mail DOT gmail DOT com>
Date: Tue, 5 Jan 2016 14:28:23 -0900
Message-ID: <CAC4O8c8r_J5udUy5EbuEm+UpgutFTFu3HhyeLaH4ZYxLt0iNMw@mail.gmail.com>
Subject: Re: [geda-user] A fileformat library
From: "Britton Kerin (britton DOT kerin AT gmail DOT com) [via geda-user AT delorie DOT com]" <geda-user AT delorie DOT com>
To: geda-user AT delorie DOT com
Content-Type: multipart/alternative; boundary=089e0122f088ff170a05289e975e
Reply-To: geda-user AT delorie DOT com
Errors-To: nobody AT delorie DOT com
X-Mailing-List: geda-user AT delorie DOT com
X-Unsubscribes-To: listserv AT delorie DOT com
Precedence: bulk

--089e0122f088ff170a05289e975e
Content-Type: text/plain; charset=UTF-8

[snip]

> Hypothetically, I think the best compromise is a format which has a
> lossless translation between text and binary representations.
>
The last time we discussed this, there was some semblance of a consensus
that we could use YAML or JSON as a text format, with possible translation
to sqlite for a binary format.

> In reality, the speed issue is for the most part irrelevant to us. We
> simply don't have the quantity of floating point numerical data in our
> files to cause enough slow down to warrant
>
> For processing 3D step files - two approaches... 1. Don't perform the
> conversion unless the number is needed (shunt strings in and out of the
> system). 2. Test out the idea of hashing and caching conversions.... I've a
> suspicion that many coordinates and vectors get repeated a lot.... (The
> Autodesk dwg format special cases 0.0 and 1.0 with a very short bit pattern
> (3 bits I recall), which gives them enough reduction in file size to make
> it worth while for them.
>
> (Btw... Anyone else react with a "wtf" to realise that the DWG binary
> format operates on a literal BIT stream? - ie. Not even byte alignment!)
>
Fun.

Britton

--089e0122f088ff170a05289e975e
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><div class=3D"gmail_extra">[snip]<br><div class=3D"gma=
il_quote"><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;bord=
er-left:1px #ccc solid;padding-left:1ex"><p dir=3D"ltr">Hypothetically, I t=
hink the best compromise is a format which has a lossless translation betwe=
en text and binary representations.</p></blockquote><div style=3D"">The las=
t time we discussed this, there was some semblance of a consensus that we c=
ould use YAML or JSON as a text format, with possible translation to sqlite=
 for a binary format.</div><blockquote class=3D"gmail_quote" style=3D"margi=
n:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<p dir=3D"ltr">In reality, the speed issue is for the most part irrelevant =
to us. We simply don&#39;t have the quantity of floating point numerical da=
ta in our files to cause enough slow down to warrant<br></p>
<p dir=3D"ltr">For processing 3D step files - two approaches... 1. Don&#39;=
t perform the conversion unless the number is needed (shunt strings in and =
out of the system). 2. Test out the idea of hashing and caching conversions=
.... I&#39;ve a suspicion that many coordinates and vectors get repeated a =
lot.... (The Autodesk dwg format special cases 0.0 and 1.0 with a very shor=
t bit pattern (3 bits I recall), which gives them enough reduction in file =
size to make it worth while for them.</p>
<p dir=3D"ltr">(Btw... Anyone else react with a &quot;wtf&quot; to realise =
that the DWG binary format operates on a literal BIT stream? - ie. Not even=
 byte alignment!)</p></blockquote><div style=3D"">Fun.</div><div>=C2=A0</di=
v><div style=3D"">Britton</div></div></div></div>

--089e0122f088ff170a05289e975e--