X-Authentication-Warning: delorie.com: mail set sender to geda-user-bounces using -f
X-Recipient: geda-user AT delorie DOT com
X-Original-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:from:date:message-id:subject:to;
        bh=2zwhFOW9Mm3X1R3e5l+2mSGPrKT/gBeSFrizIMcCyQY=;
        b=b5Tc+ibeUlZl9BQlgWtY1hOP49PGn6j5JEEg5JAHMMU85LhvPUqIvwpPpHlMc5+EwH
         Hp9FPMTvO6bHNKII5wuv6+LeJq2yymtWWv4VYl01PAfoJTldY9jSJTI26Zh26V+sTp2K
         TI2vBHMgko+v9GfNNb4qHqxhlqXAuRsglX/z0jI952CsmeNvOIACHhbHsbSkom65VFY0
         3Cl96R/Wj18BMPEEQMkjh+EhQwTyBtbRQNO3vz4KryerbbgY1oT39PCpX3X/lA6/D86y
         DCnn2rrP6A9+XLVnd/vCZZeE/LDyVfY5glrEFw7FTJaIHQPOO4UCo2y1yRQi9sZkjeuN
         3GJg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=1e100.net; s=20130820;
        h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
        bh=2zwhFOW9Mm3X1R3e5l+2mSGPrKT/gBeSFrizIMcCyQY=;
        b=kbZJem2K6Td/ZkPGKh9+Fz2nac0oaXVLU1KzpMgf9nqI8kBUoSm5QOSxEhW0T3oOtK
         BgFpFVpTKaGXIQ9YdnSQ6DJnWckqhCw1n7j2tgPORLETIqQosHIaPLjvN5CcDtoM2UEO
         wDSP5itlHo7ZKsIZhQmgabFcopM7vXOpYwJji8HzdV/Lv2fhqyVCXUbV2EBKFZTZ39ku
         qgzbLCXqNMxczedBiE8fhJpuck8OXD4Lp/LW4hr+FI8tV5HGqvEfUZ3tFfht1K0HOqFW
         my7+5swysXO6XA2gVaMo3kQDXm3uO7IPGobOYAsbPmQsUfEoaQD0ODIyphhKr1Zu7FF6
         nldA==
X-Gm-Message-State: ALyK8tJITYTWwIYo3D7JXvLlWQcKzqzbpYJZk7wzUSmtr2YN9cXNf7ZoKsWwa8EJtoHMorsZ5Cv4Ar3x7LAzSA==
X-Received: by 10.31.166.20 with SMTP id p20mr19762723vke.2.1468915199475;
 Tue, 19 Jul 2016 00:59:59 -0700 (PDT)
MIME-Version: 1.0
From: "Vladimir Zhbanov (vzhbanov AT gmail DOT com) [via geda-user AT delorie DOT com]" <geda-user AT delorie DOT com>
Date: Tue, 19 Jul 2016 10:59:58 +0300
Message-ID: <CAMvDHVAFAZieeveBynAkYRgnfyTYM+27cXVzfd9277aSOASNZA@mail.gmail.com>
Subject: [geda-user] Easy sophisticated partlist creation
To: geda-user <geda-user AT delorie DOT com>
Content-Type: multipart/mixed; boundary=001a11416d80ab95460537f8787e
Note-from-DJ: This may be spam
Reply-To: geda-user AT delorie DOT com

--001a11416d80ab95460537f8787e
Content-Type: text/plain; charset=UTF-8

Hi dear gEDA users,

Inspired by discussion on the list about half a year ago I decided
to write a Scheme function for creating sophisticated
partlists. The discussion started due to a bug in the backend
partslist3 reported by Karl Hammar (thank you, Karl!). It took a
while to understand what I want and how to do this, but now you
can try to write a small custom partlist backend yourself.

Now, a little introduction.

I have started rewriting things as reusable Scheme modules
accompanied with unit testing provided in the standard Scheme unit
testing framework module (srfi srfi-64). Look at the last couple
of patches in the geda-gaf repository, if you want to know more.

-------------------------------->8--------------------------------
$ gnetlist -i test.sch
...

Enter `,help' for help.
gnetlist> ,use (gnetlist partlist)
gnetlist> (help partlist->string)
`partlist->string' is a procedure in the (gnetlist partlist) module.

Takes PART-LIST  and outputs a string representing a
table of attribute values of the list. The following optional
arguments may be used:

  GROUP-BY is a name of the attribute by which the part list is
  grouped.

  SORT-ORDER may be an alist consisting of attribute names with
  corresponding sorting functions, or just a list of attribute names
  in which case the parts are sorted using default functions defined
  for corresponding attributes. If no sorting function is defined
  for an attribute name, the case insenstive string sorting function
  string<? is used. No sorting is carried out for attribute names
  missing in the list. The priority of sorting lowers from the first
  element to the last, that is, last attributes are taken into
  account only if all previous attributes are equal.

  OUTPUT-ORDER is a list of attribute names specifying the order in
  which the part attributes are output.

  HEADER is a text which is output without any change before
  the attribute table contents.

  FOOTER is a text which is output without any change after the
  attribute table contents.

  PREPEND-NAMES? is predicate specifying if the first line
  of the table must be the list of output attribute names.

  QUANTITY-HEADER is a text which is output for quantity column if
  PREPEND-NAMES? is not #f.

  REDUCE? is a predicate specifying if lists of grouped values must
  be reduced so that attribute values in series contain only their
  first and last values, e.g. the list of strings "R1", "R2",
  and "R3" can be transformed to "R1-R3".

  ROW-SEPARATOR is a separator string for table rows. Default
  value is "\n" (newline).

  COLUMN-SEPARATOR is a separator string for table
  columns. Default value is "\t" (tabulation).

  GROUP-SEPARATOR is a separator string for group members. Default
  value is ", ".

  SUBGROUP-SEPARATOR is like GROUP-SEPARATOR, but it is used
  between first and last elements of each subgroup (if any) of
  reduced groups if REDUCE? is not #f. Default value is "-".

  TRANSPOSE? is a predicate used to specify that the resulting
  table must be transposed, that is, part attributes are output in
  columns rather than rows.

  LETTER-CASE is an attribute value case transformation alist of
  the form
    '((NAME . VALUE) ...)
  where NAME is an attribute name and VALUE may be one of 'upper,
  'lower, or 'smart. The first two values mean that letters are to
  be transformed to upper or lower case, accordingly. The last one
  is used for special case transformations where an appropriate
  transformation function is seached for by attribute name in an
  internal function table.

  REMOVE is an alist of attributes of the form
    '((NAME . VALUE) ...)
  where NAME is a symbol and VALUE is a string, and both define
  the attribute which, being found on a part, tells that all
  attributes of the part are to be removed from the output. Such
  filtering is carried out after letter case modification.
--------------------------------8<--------------------------------

As you can see, the new function's name is partlist->string.
There are also two comparison functions in the module (gnetlist
attrib compare): refdes<? and value<?. Use gnetlist as above to
learn more of them.

Now look at the code of the backends partslist1, partslist2, and
partslist3 rewritten using these new functions. Simple, huh?
Unit tests can give you a hint on how to create your own custom
backend. Say, you want to output part list using LaTeX longtable
environment. Let's try to write it.

0) Invoke needed modules:
--------------------------------8<--------------------------------
(use-modules (gnetlist partlist)
             (gnetlist partlist common))
-------------------------------->8--------------------------------

1) Define header (later you can read it from a file):
--------------------------------8<--------------------------------
(define document-header "\\documentclass[a4paper,11pt]{letter}
\\usepackage[utf8x]{inputenc}
\\usepackage{lscape}
\\usepackage{longtable}

\\begin{document}
\\pagestyle{empty}
\\noindent\\begin{longtable}{p{.3\\linewidth}|p{.3\\linewidth}|p{.4\\linewidth}}
\\hline
")
-------------------------------->8--------------------------------

2) Define footer:
--------------------------------8<--------------------------------
(define document-footer "
\\end{longtable}

\\end{document}
")
-------------------------------->8--------------------------------

3) Define the function itself:
--------------------------------8<--------------------------------
(define (custom-partlist output-filename)
  (display
   (partlist->string
    (make-partlist '(device value footprint refdes))
    #:group-by 'refdes
    #:sort-order '(refdes footprint device)
    #:output-order '(device footprint refdes)
    #:header document-header
    #:footer document-footer
    #:prepend-names? #f
    #:reduce? #t
    #:row-separator " \\\\\n"
    #:column-separator " & "
    #:group-separator "; "
    #:subgroup-separator "--"
    #:transpose? #f
    )
   (gnetlist:output-port output-filename)))
-------------------------------->8--------------------------------

Now create a directory for your gnetlist backends, say ~/custom,
save the file as ~/custom/gnet-custom-partlist.scm and give it a try:

--------------------------------8<--------------------------------
$ cd my-project-directory
$ gnetlist -L ~/custom/  -g custom-partlist -o "-" test.sch
\documentclass[a4paper,11pt]{letter}
\usepackage[utf8x]{inputenc}
\usepackage{lscape}
\usepackage{longtable}

\begin{document}
\pagestyle{empty}
\noindent\begin{longtable}{p{.3\linewidth}|p{.3\linewidth}|p{.4\linewidth}}
\hline
resistor & m1608_a.fp & R7 \\
Resistor & m1608_a.fp & R8; Rb1 \\
resistor & m1608_a.fp & R13
\end{longtable}

\end{document}
-------------------------------->8--------------------------------

Use `-o partlist.tex' to redirect output to a file, then pdflatex,
and that's all. Your PDF is ready.

Ah, stop. Why resistors are not properly grouped?! Obviously,
their devices have different cases, though I don't understand yet,
why two lowercase 'resistor's aren't grouped?

Let's add the keyword letter-case:
--------------------------------8<--------------------------------
    ...
    #:transpose? #f
      #:letter-case '((device . smart))
    )
    ...
-------------------------------->8--------------------------------

Now we have (I keep only important part removing header and footer):
--------------------------------8<--------------------------------
RESISTOR & m1608_a.fp & R7 \\
RESISTOR & m1608_a.fp & R8; Rb1 \\
RESISTOR & m1608_a.fp & R13
-------------------------------->8--------------------------------

Hey, they aren't grouped as I wanted?!?

Ah, I see, I use 'value' here
--------------------------------8<--------------------------------
    (make-partlist '(device value footprint refdes))
-------------------------------->8--------------------------------
but don't output it.  OK, let's add it

--------------------------------8<--------------------------------
    #:sort-order '(footprint device refdes value)
    #:output-order '(device footprint refdes value)
-------------------------------->8--------------------------------
If you don't add it to #:sort-order, it will throw an error.

Now we have:
--------------------------------8<--------------------------------
RESISTOR & m1608_a.fp & R7 & 330 \\
RESISTOR & m1608_a.fp & R8; Rb1 & 10k \\
RESISTOR & m1608_a.fp & R13 & 1k
-------------------------------->8--------------------------------

OK, now I see why they are different. Let's remove 'value' all
over and try again:
--------------------------------8<--------------------------------
RESISTOR & m1608_a.fp & R7; R8; R13; Rb1
-------------------------------->8--------------------------------

Hurray! This is what I wanted!

OK, output it to a file and make a pdf file:
--------------------------------8<--------------------------------
$ gnetlist -L ~/custom/ -g custom-partlist -o partlist.tex test.sch
$ pdflatex partlist.tex
...
! Missing $ inserted.
<inserted text>
                $
l.10 RESISTOR & m1608_
                      a.fp & R7; R8; R13; Rb1
?
-------------------------------->8--------------------------------

Brr, what's wrong?
Ah, I see, underscore is a special symbol in LaTeX and must be
escaped. Then we need the module (ice-9 regex) and
substitute its function 'regexp-substitute/global' for
'display'. It has another order of arguments, and their quantity,
so the main function slightly changes.
--------------------------------8<--------------------------------
v(define (custom-partlist output-filename)
  (regexp-substitute/global
   (gnetlist:output-port output-filename) "_"
   (partlist->string
    ... skip keywords here for clarity
    )
   'pre "\\_" 'post))
-------------------------------->8--------------------------------

Yes, I just escaped underscores for LaTeX, and escaped the
backslash itself for Scheme.

Now, do some cosmetic changes (add \hline at the end of the output
table) and run gnetlist and pdflatex once again.

The resulting backend is attached.
Enjoy!

-- 
  Vladimir

--001a11416d80ab95460537f8787e
Content-Type: application/octet-stream; name="gnet-custom-partlist.scm"
Content-Disposition: attachment; filename="gnet-custom-partlist.scm"
Content-Transfer-Encoding: base64
X-Attachment-Id: file0

Cih1c2UtbW9kdWxlcyAoZ25ldGxpc3QgcGFydGxpc3QpCiAgICAgICAgICAgICAoZ25ldGxpc3Qg
cGFydGxpc3QgY29tbW9uKQogICAgICAgICAgICAgKGljZS05IHJlZ2V4KSkKCihkZWZpbmUgZG9j
dW1lbnQtaGVhZGVyICJcXGRvY3VtZW50Y2xhc3NbYTRwYXBlciwxMXB0XXtsZXR0ZXJ9ClxcdXNl
cGFja2FnZVt1dGY4eF17aW5wdXRlbmN9ClxcdXNlcGFja2FnZXtsc2NhcGV9ClxcdXNlcGFja2Fn
ZXtsb25ndGFibGV9CgpcXGJlZ2lue2RvY3VtZW50fQpcXHBhZ2VzdHlsZXtlbXB0eX0KXFxub2lu
ZGVudFxcYmVnaW57bG9uZ3RhYmxlfXtwey4zXFxsaW5ld2lkdGh9fHB7LjNcXGxpbmV3aWR0aH18
cHsuNFxcbGluZXdpZHRofX0KXFxobGluZQoiKQoKKGRlZmluZSBkb2N1bWVudC1mb290ZXIgIiBc
XFxcXG4KXFxobGluZQpcXGVuZHtsb25ndGFibGV9CgpcXGVuZHtkb2N1bWVudH0KIikKCihkZWZp
bmUgKGN1c3RvbS1wYXJ0bGlzdCBvdXRwdXQtZmlsZW5hbWUpCiAgKHJlZ2V4cC1zdWJzdGl0dXRl
L2dsb2JhbAogICAoZ25ldGxpc3Q6b3V0cHV0LXBvcnQgb3V0cHV0LWZpbGVuYW1lKSAiXyIKICAg
KHBhcnRsaXN0LT5zdHJpbmcKICAgIChtYWtlLXBhcnRsaXN0ICcoZGV2aWNlIGZvb3RwcmludCBy
ZWZkZXMpKQogICAgIzpncm91cC1ieSAncmVmZGVzCiAgICAjOnNvcnQtb3JkZXIgJyhmb290cHJp
bnQgZGV2aWNlIHJlZmRlcykKICAgICM6b3V0cHV0LW9yZGVyICcoZGV2aWNlIGZvb3RwcmludCBy
ZWZkZXMpCiAgICAjOmhlYWRlciBkb2N1bWVudC1oZWFkZXIKICAgICM6Zm9vdGVyIGRvY3VtZW50
LWZvb3RlcgogICAgIzpwcmVwZW5kLW5hbWVzPyAjZgogICAgIzpyZWR1Y2U/ICN0CiAgICAjOnJv
dy1zZXBhcmF0b3IgIiBcXFxcXG4iCiAgICAjOmNvbHVtbi1zZXBhcmF0b3IgIiAmICIKICAgICM6
Z3JvdXAtc2VwYXJhdG9yICI7ICIKICAgICM6c3ViZ3JvdXAtc2VwYXJhdG9yICItLSIKICAgICM6
dHJhbnNwb3NlPyAjZgogICAgIzpsZXR0ZXItY2FzZSAnKChkZXZpY2UgLiBzbWFydCkpCiAgICAp
CiAgICdwcmUgIlxcXyIgJ3Bvc3QpKQo=
--001a11416d80ab95460537f8787e--