X-Authentication-Warning: delorie.com: mail set sender to geda-user-bounces using -f X-Recipient: geda-user AT delorie DOT com X-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; bh=qTMyfM5dU3wBtsSeQIpHe7541o0bF9tFztGnG0LLAM0=; b=QzHTiFRAL8msXrVpq/NBrExrgEnX+39mVu9mMkPjA/ISH9WwdHJAp044bn4343tKHB HH4/efE17Zfy+nyq2kPZGKXpAEpxUmP3G23VDtI3hdL+zwadSlOWkwnQ/6aR/770T/gG 9FcDbT2CN8Q3J2K8Uuprsfmrnq9xfYnDmq0o3L8R5nydJgIN05m5y0k67iHOhTRVLyZR ln/KSFe5vKHb+n/pzA9E9Hmxc4bgrGDia18fhL7UIsFIdsWhINlgt9bGFIgpqYviRe4k 0+uqm56MJ+EOTdloJXICafjByBCFdkVOPRRVu/xEZN8IqwzXuxed6EYyz4Pkwr0yKKGU Tvgw== X-Received: by 10.180.187.227 with SMTP id fv3mr5145311wic.57.1441983153336; Fri, 11 Sep 2015 07:52:33 -0700 (PDT) Date: Fri, 11 Sep 2015 16:52:28 +0200 From: "Nicklas Karlsson (nicklas DOT karlsson17 AT gmail DOT com) [via geda-user AT delorie DOT com]" To: geda-user AT delorie DOT com Subject: Re: [geda-user] About reinventing the wheel, and how to avoid it Message-Id: <20150911165228.da52124d5eccae94d093bd30@gmail.com> In-Reply-To: References: X-Mailer: Sylpheed 3.5.0beta1 (GTK+ 2.24.25; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Reply-To: geda-user AT delorie DOT com Errors-To: nobody AT delorie DOT com X-Mailing-List: geda-user AT delorie DOT com X-Unsubscribes-To: listserv AT delorie DOT com Precedence: bulk I did not have time to read source code before I engaged in the discussion but it certainly seems we are heading in the same direction. Nicklas Karlsson On Fri, 11 Sep 2015 16:06:36 +0200 (CEST) Roland Lutz wrote: > While reading the past discussions here on the list, I had the impression > that some ideas are developed over and over again, while few people took > notice of the infrastructure which I already created. > > An in-memory database? Look at libxornstorage. An interchange mechanism > between programs written in different languages? Look at libxornstorage > and write a wrapper for your favorite language if it doesn't already exist. > > I'm not "working on the Python branch", and I haven't "created a complete > parallel to gnetlist, but based on Python rather than Scheme". > xorn-netlist *is* gnetlist. I'm gradually shifting the gEDA > infrastructure to a more modular design, using C where I have to (for > interoperability with any language) and Python where I can (for those > implementation parts which don't have to be hand-optimized in C). In the > process, I converted the API for netlist backends to Python, but that was > partly because there are no Guile bindings in Python, and the code turned > out to be much easier this way. > > > On Tue, 25 Aug 2015, Nicklas Karlsson (nicklas DOT karlsson17 AT gmail DOT com) [via > geda-user AT delorie DOT com] wrote: > > It might work better you postpone the memory representation and figure > > which kind of functions you need to access the objects. Then the access > > functions are decided it will be easier to think about the > > representation. > > The thing is, I've already done exactly that. xornstorage.h is a > carefully designed interface that provides the kinds of queries which are > to be expected in both a GUI aplication and in scripts (excluding > rectangular selection, which is an entirely different problem on its own). > The implementation is independent from that and can easily be exchanged > for another one. > > > On Tue, 25 Aug 2015, Evan Foss (evanfoss AT gmail DOT com) [via > geda-user AT delorie DOT com] wrote: > > And now imagine the biggest difference of all. Instead of one giant > > program with poorly defined boundaries between layers you had > > > > * libpcbfile for moving between a memory representation of a layout > > and the file that stores it > > * libpcbgeo for processing geometric properties > > * libpcbexport for translation to pdf, ps, png, jpeg > > * libpcbnetlist for forward and backward annotation > > * libpcbui for providing the HID interface > > > > and etc. > > This is just what I did with Xorn for schematic files: > > xorn.storage - in-memory representation of schematic objects > xorn.geda.read - reads .sch/.sym files > xorn.geda.write - writes .sch/.sym files > xorn.geda.netlist - creates netlists from schematic files > xorn.geda.render - renders schematics to Cairo for display and PDF export (work in progress) > > The obvious thing would be to continue that scheme for PCB, too: > > xorn.storage - in-memory representation of PCB objects > xorn.pcb.read - reads .pcb files > xorn.pcb.write - writes .pcb files > xorn.pcb.netlist - creates netlists for back annotation > xorn.pcb.render - renders PCBs to Cairo for display and PDF export > > Right now, there is some code duplication between libgeda and Xorn since I > didn't update libgeda to use the new infrastructure yet. This is on my > to-do list, though. > > > On Wed, 26 Aug 2015, Nicklas Karlsson (nicklas DOT karlsson17 AT gmail DOT com) [via > geda-user AT delorie DOT com] wrote: > > I guess it would be possible to get different views. What is needed is a > > set of functions to access the data. I guess should be a need for: > > iterate thru the objects on one layer, iterate thru the nets, iterate > > thru the descriptors, extract symbols with a particular value, iterate > > thru symbol coordinates to renumber them left rigt, ... > > It shouldn't be necessary to iterate over the objects in the first place. > Of course, there are situations where you can't avoid it (like, for > example, when drawing them to the screen), but usually the storage library > should know best which algorithm(s) to use to handle the request most > efficiently. Therefore, it should be provided with a "high-level" > description of the query so it is able to optimize the request. > > Imagine the following SQL query: > > DELETE FROM objects WHERE radius = 0.; > > Using Xorn as an example, you could either iterate over all objects, > looking for circles and arcs wth a matching radius, and delete these > objects: > > xorn_object_t *objects; > size_t count; > unsigned int i; > > xorn_get_objects(rev, &objects, &count); > > for (i = 0; i < count; i++) { > xorn_obtype_t type = xorn_get_object_type(rev, objects[i]); > > if (type == xornsch_obtype_circle && > xornsch_get_circle_data(rev, objects[i])->radius == 0.) > xorn_delete_object(rev, objects[i]); > > if (type == xornsch_obtype_arc && > xornsch_get_arc_data(rev, objects[i])->radius == 0.) > xorn_delete_object(rev, objects[i]); > } > > free(objects); > > Or, you could provide the library with all information necessary to > optimize the request and let it figure out the details: > > xorn_selection_t sel = xornsch_select_by_radius(rev, 0.); > xorn_delete_selected_objects(rev, sel); > xorn_free_selection(sel); > > Maybe the list of objects is implemented in a way that is optimized for > item lookup, but makes deleting an object a really slow operation? In > this case, the library could implement xorn_delete_selected_objects as a > bulk deletion, which would result in a much faster response. > > The current implementation of libxornstorage isn't very sophisticated, but > it has a well-documented API and an exhaustive test suite, so anyone who > is interested is welcome to provide a better alternative. > > > On Thu, 27 Aug 2015, Stephen R. van den Berg (srb AT cuci DOT nl) [via > geda-user AT delorie DOT com] wrote: > > It should easily be possible to have the applications display realtime > > from the db without creating complicated extra internal datastructures. > > This is already how -lxornstorage works. With SQL, you have to create at > least the result array. With Xorn, you get raw pointers to the storage > structs. For example, the struct for a schematic text is defined as: > > struct xornsch_text { > struct xorn_double2d pos; > int color; > int text_size; > bool visibility; > int show_name_value; > int angle; > int alignment; > struct xorn_string text; > }; > > where xorn_double2d is defined as: > > struct xorn_double2d { > double x, y; > }; > > and xorn_string is defined as: > > struct xorn_string { > const char *s; > size_t len; > }; > > This struct and the referenced string are copied exactly once--when > setting the data, so the library doesn't interfere with the calling > application's memory management. This concept obviously only works if all > calling applications are well-behaved and respect the "const" qualifier, > but that's the price of accessing the data structures directly. > > Roland >