X-Authentication-Warning:	delorie.com: mail set sender to geda-user-bounces using -f
X-Recipient:	geda-user AT delorie DOT com
X-Original-DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=gmail.com; s=20161025;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to;
	bh=P0mtPmNoiH4nPp9ADKkRtfk/cs4YvInuDGxWFITOSAo=;
	b=cWBsGCPIFctTX/BNyE0a6AdgX3zcqBoa69JTHG4xRXDh8oE6MvK8R5Fnc4VGEs7g1E
	1+BzvcZIPz1n+1yUy9u3pn75zlb9CpiwCCnslXLPrM9MEy8pJDdbAHy5NTpjG9v6JGDX
	UH7HeQZWXSdKOPpjhFBtfuXX5QkJYR56nE2wOnMYkXujLSKM8ZEhUkksjYrGaDkuFRTz
	Gj2hPRGKfCNEXlGNcZeYQq7Czx46sOdx7FPzo4Cy3i3WpIr5en+ug6xZuI/cFs7PKAV2
	HV0BRUpYpis7Aq4Bm/iGUBptXgtyHRQ0ue+mHf8FpvWFXQgsgcsvtheI093wvRdmQNF/
	HYWA==
X-Google-DKIM-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:in-reply-to:references:from:date
	:message-id:subject:to;
	bh=P0mtPmNoiH4nPp9ADKkRtfk/cs4YvInuDGxWFITOSAo=;
	b=OgP3Wv3vs4EH3b+8RO6d1d8i6zXsrpof1J8xvClaeI1WALHmXsC0evRsvOfv6JfR/1
	bJQT/8Zjs9l/JwU4+0gzesqkmuOtw0zcUpGejOOH/M918PI9yBP9tUIngM15WS57Q2cx
	wlO7P4lIWLPzQpZdBpNvRZYcoOSeFb8Spe9Bna+ZlF/OezO448ZSDcD6caDyyx4jmNYP
	uUgew87RY5inTyxPPhakLPgNOE2mgLNsDs76Odco7sCY8MGnhi6FWF+GOVz7tX9159uF
	f4e8ktq9ElxJlxJAA+Mn0Dhg/+sk5iexB+OGxOmkH3sgxVISNTqFXnXEq04dzcK9FPxw
	29kw==
X-Gm-Message-State:	AIVw112BBb2zK/ll6Z2EygS1GYbD6ebKTQEM792E+hjpy/2iRz1ysjC1
	x9NXeCiLDzbnWG1hAsXap/Wv9GW8zO0T
X-Received:	by 10.237.51.167 with SMTP id v36mr2039709qtd.72.1499801235977;
	Tue, 11 Jul 2017 12:27:15 -0700 (PDT)
MIME-Version:	1.0
In-Reply-To:	<alpine.DEB.2.00.1707110447510.27212@igor2priv>
References:	<20170711005040 DOT d96eccaffe490027849789c3 AT gmail DOT com>
	<20170711020955 DOT 0108aaea AT akka> <alpine DOT DEB DOT 2 DOT 00 DOT 1707110447510 DOT 27212 AT igor2priv>
From:	"Britton Kerin (britton DOT kerin AT gmail DOT com) [via geda-user AT delorie DOT com]" <geda-user AT delorie DOT com>
Date:	Tue, 11 Jul 2017 11:27:15 -0800
Message-ID:	<CAC4O8c9vqX9R0QWPzMz4-dimsqrX-GrcXhhno6YgR+WRjRD_aw@mail.gmail.com>
Subject:	Re: [geda-user] element restrictions: is it really about the file
	format? (was: Re: PCB, load element bug)
To:	geda-user AT delorie DOT com
Reply-To:	geda-user AT delorie DOT com
Errors-To:	nobody AT delorie DOT com
X-Mailing-List:	geda-user AT delorie DOT com
X-Unsubscribes-To:	listserv AT delorie DOT com

It's been obvious for a long time that the file format is not the
major problem, anyone who has spent time time in e.g. search.c or
find.c knows this.  The file format is itself a problem for other
reasons, however.

If you're going to replace the entire data model, why not use SQLite?
This would give you an almost free replacement for the convoluted and
buggy rtree and undo implementations as well as providing file format
accessible to almost every language anyone might care to use.

Britton

On Mon, Jul 10, 2017 at 8:03 PM,  <gedau AT igor2 DOT repo DOT hu> wrote:
> Hi all,
>
> because this topic comes up from time to time, and by now I have actual data
> (implementation costs) on it, I decided to write a short analysis on:
>
> - why we have the "element restrictions" problem in the first place
>
> - different ways to resolve the "element restrictions" problem
>
> - measured (or partly estimated) cost of some of these
>
> - conclusion: why simply changing the element file format won't help
>
> My intention is to give some insight (backed up with actual data) to both
> users and developers, before the thread escalates into the usual xml vs.
> json vs. my-favorite-file-format festival.
>
>
>> Maybe you composed the footprint entirely of atoms that are not elegible
>> to be included in a pcb footprint. Due to restrictions in the file
>> format,
>
>
> Sorry to say, but the main problem is not the file format.
>
>
> THE PROBLEM
>
>
> Elements contain element-specific drawing primitives and the code has
> special casing for these in every little corner. Extending/replacing the
> file format is a relatively small effort, affecting a small part of the code
> base. Adding support for polygons, copper lines, vias, arbitrary text, etc.
> in elements is a huge effort because it really touches everything from the
> famous find.c through all drawing pimitive holding data structures to
> draw.c.
>
> When you see a silk line or silk arc on screen, produced by an element, it
> is not really just a line or arc. It is really an element-line and an
> element-arc. Yes, these are separate, special drawing primitives that work
> only in elements; elements can't consists of plain line or arcs. Majority of
> the code simply has an "if" or a separate "case" branch or a separate
> callback function for handling each of these element-* object. The data
> structs don't hold a list of drawing primitives; they hold a list and an
> rtree of lines, another list and rtree of arcs, etc. And yes, this means a
> specific element-line and element-arc list within elements. Plus these
> element-objects ignore layers: instead of being on a layer, they are either
> on the top or on the bottom side. For silk and pads it's fine, but if you
> want outline, paste, inner copper layers, that's 1000 other special casing.
>
> For example if you wanted to have non-pad copper lines in a pcb footprint,
> extending the existing code base, you'd need to:
>
>  - decide to introduce an element-copper-line drawing primitive (estimated
> cost: at most an hour to add it in all the central lists of magic numbers
> and names)
>
>  - go and grep the code for handling plain old lines in any way, and copy
> all code and introduce an element-copper-line drawing primitive (estimated
> cost: days or weeks!!!)
>
>  - reimplement the which-side-is-this-on hack for your new object (estimated
> cost: many hours or days)
>
>  - add a printf in file.c and a yacc/lex section for the new drawing
> primitive (estimated cost: at most 1..2 hours)
>
> (Or you can chose to rather implement a generic object grouping feature of
> the existing primitives: line, arc, text, poly, via, and say element-*
> special casing should be just removed as groups can replace elements. This
> is what I am doing in pcb-rnd with subcircuits, see below)
>
> This is why, contrary to the popular belief, simply changing the file format
> to xml or json or whatever-else won't magically fix the problem. Instead
> rewriting 1/3 of the code base would. Then whether you keep the .fp/.pcb
> format (and upgrade it to handle the new situation) or introduce a new file
> format is a tiny implementation detail.
>
> Note: I'm saying this after implementing a new native file format (lihata),
> working on half dozen alien board/footprint file formats _and_ being halfway
> through resolving the element restrictions by replacing elements with
> subcircuits in pcb-rnd.
>
>
>
> EVIDENCES: DATA
>
>
> Below I'm sharing objective data on this topic. Talking in pcb-rnd net
> development times, implementing support for a new footprint file format is
> ranging from a few hours to 1..2 days, if the format is reasonable. For
> example implementing a working plugin for tEDAx that loads netlist, loads
> and saves footprints took less than 14 hours total. Note: this is the time
> spent on a new, non-native format _and_ a netlist load. Extending the
> .fp/.pcb format would be way cheaper, so you can take this as a very
> permissive upper estimation.
>
> Implementing half of the subcircuits took me 85 hours so far: we have
> generic grouping, with dynamic layer binding; we can save and load the
> result; we can have any object on any layer; we keep track of
> rotation/mirroring/scaling; we have and unique ID to keep track on what
> subcircuit is cloned from what other subcircuit. What's still missing for
> replacing elements, and we will have in the next development cycle:
> "terminals" (turning any copper object into a pin or pad).
>
> By the time subcircuits will be able to fully replace footprints/elements, I
> estimate total expense between 180..200 hours. And these "low" numbers are
> possible only because I had already invested more than 130 hours in the
> layer rewrite before I started on subcircuits. If we account the the
> element-restriction-resolving parts of the layer rewrite in this effort, the
> final "element rewrite" cost would be over 250 hours (from which more than
> half are already spent and only the other part is estimation).
>
>
> CONCLUSION
>
> 250 hours of subcircuits+layers (half spent, half estimated) vs. 14 hours of
> a file format (spent).
>
> So a [new footprint file format + new netlist format] was less than 6% of
> the half-measured-half-estimated cost of solving the element restriction
> problem in case of pcb-rnd.
>
> Of course different projects and different developers would make different
> choices, and with subcircuits I am really doing the most flexible thing
> possible: i just decided to really clean up the data model, not just hack in
> polygons and copper lines.
>
> But I still say that with even a cheaper, more hackish element upgrade the
> file format issue is only 5..15% of the whole problem.
>
> Best regards,
>
> Igor2
>
> P.S. I am also converting this into a pcb-rnd devlog entry for later
> reference.

- Raw text -