the digital librarian

this means that this div has it's own page (because it has nzetc:has-text="true" and that the id of that page is tei-gorlaws-t -g -t -front -tp (because of the nzetc:id="tei-gorlaws-t -g -t -front -tp "). the id can be plugged into: http://www.nzetc.org/tm/scholarly/.html to get a url for the html. thus the url for this div is http://www.nzetc.org/tm/scholarly/tei-gorlaws-t -g -t -front -tp .html this process should work for both text and figures. happy remixing everyone! posted by stuart yeates at : comment: sunday, november epubs and quality you may have heard news about the release of "bookserver" by the good folks at the internet archive. this is a drm-free epub ecosystem, initially stocked with the prodigious output of google's book scanning project and the internet archive's own book scanning project. to see how the nzetc stacked up against the much larger (and better funded) collection i picked one of our maori language dictionaries. our maori and pacifica dictionaries month-after-month make up the bulk of our top five must used resources, so they're in-demand resources. they're also an appropriate choice because when they were encoded by the nzetc into tei, the decision was made not to use full dictionary encoding, but a cheaper/easier tradeoff which didn't capture the linguistic semantics of the underlying entries, but treated them as typeset text. i was interested in how well this tradeoff was wearing. i did my comparison using the new firefox epub plugin, things will be slightly different if you're reading these epubs on an iphone or kindle. the epub i looked at was a dictionary of the maori language by herbert w. williams. the nzetc has the sixth edition. there are two versions of the work on bookserver. a second edition scanned by google books (original at the new york public library) and a third edition scanned by the internet archive in association with microsoft (original in the university of california library system). all the processing of both works appear to be been done in the u.s. the original print used macrons (nzetc), acutes (google) and breves (internet archive) to mark long vowels. find them here. lets take a look at some entries from each, starting at 'kapukapu': nzetc: kapukapu. . n. sole of the foot. . apparently a synonym for kaunoti, the firestick which was kept steady with the foot. tena ka riro, i runga i nga hanga a taikomako, i te kapukapu, i te kaunoti (m. ). . v.i. curl (as a wave). ka kapukapu mai te ngaru. . gush. . gleam, glisten. katahi ki te huka o huiarau, kapukapu ana tera. kapua, n. . cloud, bank of clouds. e tutakitaki ana nga kapua o te rangi, kei runga te mangoroa e kopae pu ana (p.). . a flinty stone. = kapuarangi. . polyprion oxygeneios, a fish. = hapuku. . an edible species of fungus. . part of the titi pattern of tattooing. kapuarangi, n. a variety of matā, or cutting stone, of inferior quality. = kapua, . kāpuhi, kāpuhipuhi, n. cluster of branches at the top of a tree. kāpui, v.t. . gather up in a bunch. ka kapuitia nga rau o te kiekie, ka herea. . lace up or draw in the mouth of a bag. . earth up crops, or cover up embers with ashes to keep them alight. kāpuipui, v.t. gather up litter, etc. kāpuka, n. griselinia littoralis, a tree. = papauma. kapukiore, n. coprosma australis, a shrub. = kanono. kāpuku = kōpuku, n. gunwale. google books: kapukapu, s. sole of the foot, eldpukdpu, v. to curl* as a wave. ka kapukapu mai te ngaru; the wave curls over. kapunga, v. to take up with both hands held together, kapungatia he kai i te omu; take up food from the oven. (b. c, kapura, s. fire, -' tahuna he kapura ; kindle a fire. kapurangi, s. rubbish; weeds, kara, s. an old man, tena korua ko kara ? how are you and the old man ? kara, s> basaltic stone. he kara te kamaka nei; this stone is kara. karaha, s. a calabash. ♦kardhi, *. glass, internet archive: kapukapu, n. sole of the foot. kapukapu, v. i. . curl (as a wave). ka kapukapu mai te ngaru. . gush. kakapii, small basket for cooked food. kapua, n. cloud; hank of clouds, kapunga, n. palm of the hand. kapunga, \. t. take up in both hands together. kapiira, n. fire. kapiiranga, n. handful. kapuranga, v. t. take up by hand-fuls. kapurangatia nga otaota na e ia. v. i. dawn. ka kapuranga te ata. kapur&ngi, n. rubbish; uveds. i. k&r&, n. old man. tena korua ko kara. ii. k&r&, n. secret plan; conspiracy. kei te whakatakoto kara mo te horo kia patua. k&k&r&, d. scent; smell. k&k&r&, a. savoury; odoriferous. k^ar&, n. a shell-iish. unlike the other two, the nzetc version has accents, bold and italics in the right place. it' the only one with a workable and useful table of contents. it is also edition which has been extensively revised and expanded. google's second edition has many character errors, while the internet archive's third edition has many 'á' mis-recognised as '&.' the google and internet achive versions are also available as pdfs, but of course, without fancy tables of contents these pdfs are pretty challenging to navigate and because they're built from page images, they're huge. it's tempting to say that the nzetc version is better than either of the others, and from a naïve point of it is, but it's more accurate to say that it's different. it's a digitised version of a book revised more than a hundred years after the second edition scanned by google books. people who're interested in the history of the language are likely to pick the edition over the edition nine times out of ten. technical work is currently underway to enable third parties like the internet archive's bookserver to more easily redistribute our epubs. for some semi-arcane reasons it's linked to upcoming new search functionality. posted by stuart yeates at : no comments: labels: library, macrons, maori, nzetc what librarything metadata can the nzetc reasonable stuff inside it's cc'd epubs? this is the second blog following on from an excellent talk about librarything by librarything's tim given the vuw in wellington after his trip to lianza. the nzetc publishes all of it's works as epubs (a file format primarily aimed at mobile devices), which are literally processed crawls of it's website bundled with some metadata. for some of the nzetc works (such as erewhon and the life of captain james cook), librarything has a lot more metadata than the nzetc, becuase many librarything users have the works and have entered metadata for them. bundling as much metadata into the epubs makes sense, because these are commonly designed for offline use---call-back hooks are unlikely to be avaliable. so what kinds of data am i interested in? ) traditional bibliographic metadata. both lt and nzetc have this down really well. ) images. lt has many many cover images, nzetc has images of plates from inside many works too. ) unique identification (isbns, issns, work ids, etc). lt does very well at this, nzetc very poorly ) genre and style information. lt has tags to do fancy statistical analysis on, and does. nzetc has full text to do fancy statistical analysis on, but doesn't. ) intra-document links. lt has work as the smallest unit. nzetc reproduces original document tables of contents and indexes, cross references and annotations. ) inter-document links. lt has none. nzetc captures both 'mentions' and 'cites' relationships between documents. while most current-generation ebook readers, of course, can do nothing with most of this metadata, but i'm looking forward to the day when we have full-fledged openurl resolvers which can do interesting things, primarily picking the best copy (most local / highest quality / most appropiate format / cheapest) of a work to display to a user; and browsing works by genre (librarything does genre very well, via tags). posted by stuart yeates at : comment: labels: epubs, library, librarything, nzetc thursday, october interlinking of collections: the quest continues after an excellent talk today about librarything by librarything's tim, i got enthused to see how librarything stacks up against other libraries for having matches in it's authority control system for entities we (the nzetc) care about. the answer is averagely. for copies of printed books less than a hundred years old (or reprinted in the last hundred years), and their authors, librarything seems to do every well. these are the books likely to be in active circulation in personal libraries, so it stands to reason that these would be well covered. i tried half a dozen books from our nineteenth-century novels collection, and most were missing, erewhon, of course, was well represented. librarything doesn't have the "treaty of waitangi" (a set of manuscripts) but it does have "facsimiles of the treaty of waitangi." it's not clear to me whether these would be merged under their cataloguing rules. coverage of non-core bibliographic entities was lacking. places get a little odd. sydney is "http://www.librarything.com/place/sydney,% new% south% wales,% australia" but wellington is "http://www.librarything.com/place/wellington" and anzac cove appears to be is missing altogether. this doesn't seem like a sane authority control system for places, as far as i can see. people who are the subjects rather than the authors of books didn't come out so well. i couldn't find abel janszoon tasman, pōtatau te wherowhero or charles frederick goldie, all of which are near and dear to our hearts. here is the spreadsheet of how different web-enabled systems map entities we care about. correction: it seems that the correct url for wellington is http://www.librarything.com/place/wellington,% new% zealand which brings sanity back. posted by stuart yeates at : no comments: labels: authority, community building, metadata, semantic web, social network, taxonomy saturday, september ebook readers need openurl resolvers everyone's talking about the next generation of ebook readers having larger reading area, more battery life and more readable screen. i'd give up all of those, however, for an ebook reader that had an internal openurl resolver. openurl is the nifty protocol that libraries use to find the closest copy of a electronic resources and direct patrons to copies that the library might have already licensed from commercial parties. it's all about finding the version of a resource that is most accessible to the user, dynamically. say i've loaded ebooks into my ebook reader: a couple of encyclopedias and dictionaries; a stack of books i was meant to read in school but only skimmed and have been meaning to get back to; current block-busters; guidebooks to the half-dozen countries i'm planning on visiting over the next couple of years; classics i've always meant to read (tolstoy, chaucer, cervantes, plato, descartes, nietzsche); and local writers (baxter, duff, ihimaera, hulme, ...). my ebooks by nietzsche are going to refer to books by descartes and plato; my ebooks by descartes are going to refer to books by plato; my encyclopaedias are going to refer to pretty much everything; most of the works in translation are going to contain terms which i'm going to need help with (help which theencyclopedias and dictionaries can provide). ask yourself, though, whether you'd want to flick between works on the current generation of readers---very painful, since these devices are not designed for efficient navigation between ebooks, but linear reading of them. you can't follow links between them, of course, because on current systems links must point either with the same ebook or out on to the internet---pointing to other ebooks on the same device is verboten. openurl can solve this by catching those urls and making them point to local copies of works (and thus available for free even when the internet is unavailable) where possible while still retaining their until ebook readers have a mechanism like this ebooks will be at most a replacement only for paperback novels---not personal libraries. posted by stuart yeates at : comment: older posts home subscribe to: posts (atom) about me stuart yeates view my complete profile blog archive ▼ ( ) ▼ march ( ) #christchurchmosqueshootings ► ( ) ► october ( ) ► ( ) ► october ( ) ► ( ) ► july ( ) ► ( ) ► june ( ) ► ( ) ► december ( ) ► november ( ) ► august ( ) ► june ( ) ► march ( ) ► ( ) ► november ( ) ► ( ) ► november ( ) ► october ( ) ► september ( ) ► july ( ) ► june ( ) ► may ( ) ► february ( ) ► january ( ) ► ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) shared google reader items simple theme. powered by blogger. risk data library the risk data library schema database data resources project documentation the risk data library making disaster risk assessment more effective by making risk data easier to store, find, edit and share. making risk data easier to work with disaster risk assessments are more important and more effective than ever, but common issues with risk data are beginning to limit their application. the rdl was launched to address those issues by developing tools and resources designed specifically for the needs of disaster risk experts and the assessments they perform. learn more about the project a system for better managing risk data we seek to create a collection of components to make working with risk data easier, each one addressing a different underlying problem. working together, these components create an effective library system designed explicitly for storing, finding, editing and exchanging data for disaster risk assessments. the first of these two components are now available. the rdl schema a new language for describing hazard, exposure, vulnerability and loss data the rdl schema gives risk experts a single language to describe hazard, exposure, vulnerability and modelled loss datasets. it gives datasets an underlying consistency that makes them highly interoperable and easily read by both people and machines. the schema also contains labels for key metadata fields, making it easier to identify datasets without relying on external files or descriptions. learn more the database storing hazard, exposure, vulnerability and loss datasets in the same environment the architecture of the rdl database is capable of storing all four common types of risk data in the same environment, where they can be indexed and retrieved by a single search function. this reduces the number of separate resources needed to store risk data, reducing costs and search times. learn more developed by risk data experts the rdl project is the result of the collective effort and ongoing support of internationally-recognised research insitutions and established global partnerships with combined expertise across multiple hazards and all aspects of risk assessment. the partners have a complete understanding of the challenges risk analysts face in finding, editing and exchanging data for disaster risk assessments, for various scales and contexts. learn more about the partners project contact and license to learn more or contribute to the risk data library project, contact us at: sfraser@worldbank.org and pchrzanowski@worldbank.org the content of this website, unless otherwise noted, is licensed under creative commons attribution-sharealike . international (cc-by-sa . ) github: where the world builds software · github skip to content github no longer supports this web browser. learn more about the browsers we support. sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories → security → team enterprise explore explore github → learn & contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → nonprofit → education → search all github ↵ jump to ↵ no suggested jump to results search all github ↵ jump to ↵ search all github ↵ jump to ↵ search all github ↵ jump to ↵ sign in sign up {{ message }} where the world builds software millions of developers and companies build, ship, and maintain their software on github—the largest and most advanced development platform in the world. email address sign up for github + million developers + million organizations + million repositories % fortune build like the best with github enterprise take collaboration to the next level with security and administrative features built for teams. contact sales code collaborate develop automate secure community sign up give your code a home in the cloud record or rewind any change to your code to keep you and your team in sync. host it all for free with unlimited public and private repositories. sign up for github jasonetco added some commits minutes ago updated readme.md octocat-classifier assets tests license readme.md index.js package.json readme.md app.js index.html # octocat classifier :octopus: :cat: :mag: ![](https://img.shields.io/badge/build-passing-brightgreen) ![](https://img.shields.io/badge/coverage- % -green) ![](https://img.shields.io/badge/dependencies-up% to% date-brightgreen) as the name suggests, octocat classifier is used to determine whether a given image contains an octocat. it is trained with images from the [octodex]( ), images shared with [#myoctocat on twitter]( ), and [photographs of laptops with :octocat: stickers on them](). ## installation ``` git clone https://github.com/jasonetco/octocat-classifier ``` → ~/octocat-classifier gh repo create octocat-classifier ✓ created repository jasonetco/octocat-classifier on github ✓ added remote https://github.com/jasonetco/octocat-classifier.git → ~/octocat-classifier git push origin main support octocats shared on twitter created index.js build on what's been built write less code thanks to the world's largest software package registry. find the best community-approved projects to accelerate your work, then share it with the world with npm and github packages. → ~/octocat-classifier npm install eslint + eslint@ . . added packages from contributors and audited packages in . s packages are looking for funding run `npm fund` for details found vulnerabilities → ~/octocat-classifier added package for javascript linting git checkout -b origin add-status-screens origin/add-status-screens + collaborators ready player two. scale your team to any size in the cloud. create a new organization for free better code starts with pull requests—conversations around your code where you can experiment, squash bugs, and build new features. code review is built in. pull requests cover the entire review flow: propose changes, browse code, ask for input, make a suggestion, and sign off in one place. know when your pull request is ready to merge when everything’s green. reviews approved? check. tests passing? check check. no conflicts? ship it already. git merge add-status-screens keep work moving. review or merge code, manage notifications, browse repositories, and more with github for mobile. available for ios and android ••• bash ➜ ~ gh pr status relevant pull requests in cli/cli current branch there is no pull request associated with [main] created by you you have no open pull requests requesting a code review from you # correctly handle and set empty fields... [octocat:emptybody] ✓ checks passing # added confirmation steps for risk... [octocat:confirmations] x / checks failing ➜ ~ work however you want. put a gui on it with github desktop or stay in the command line with github cli. available for macos, windows, and linux* * github cli is available on macos, windows, and linux * github desktop is available on macos and windows instant dev environments with codespaces learn more about github codespaces the future of code is in the cloud, not your local copy. codespaces gives you a complete, configurable dev environment on top of a powerful vm in minutes. visual studio code, in your browser, full stop. codespaces brings the world’s most popular desktop editor to every repo. code, build, test, use the terminal, and open pull requests from anywhere. customize to your heart’s desire. add your favorite vs code extensions, create a devcontainer config file, install new themes, and tweak your settings. automate anything with github actions learn more about github actions setup ci/cd, enhance your devops, and script your entire workflow with github actions. kick off automated workflows with github events like push, issue creation, merge, and release. , + actions write your own, or import actions from the open source community, all within our world-class editor. feeling stuck? browse the actions developer docs as you code. explore the actions marketplace you can have it all. run actions in any language or operating system, on linux, macos, windows, arm, and containers. or all at once with matrix builds. with million jobs run per month you’re in good company with actions, the number one ci service on the world’s largest developer platform. speaking of automation, dependabot keeps your projects up to date with automated pull requests that update all your dependencies. just review and merge to keep your software secure. learn more about dependabot find and fix vulnerabilities before you merge learn more about advanced security secure your code as you write it. codeql’s code scanning automatically reviews every change to your codebase and identifies known vulnerabilities before they ever reach production. vulnerabilities found fixed deserialized data security vulnerability keep your secrets. we automatically scan repositories for oauth tokens, api keys, personal tokens, and more. if we find one, we’ll notify you and the partner that issued it to invalidate the secret. oauth token found replaced oauth token with a key vault value found a vulnerability? our security advisory remediation tools help developers identify and disclose them responsibly so maintainers can patch them in dedicated, private workspaces. the home for all developers— including you beta github discussions is dedicated space for your community to come together, ask and answer questions, and have open-ended conversations. learn how to get started with discussions amplify your voice in your own personal readme on your profile. tell the story of your work through your repositories, contributions, and technologies of choice. sophshep started sponsoring you for $ /month x days ago joshaber started sponsoring you for $ /month x days ago pmarsceill started sponsoring you for $ /month days ago that open source code you wrote for yourself might just help someone else. get paid for building what matters to you with github sponsors. support the projects you depend on, too. % to developers, zero fees learn more about sponsors make your contribution small experiments, inspired inventions, and the software everyone depends on—the code you write on github can reach one codebase or millions. sign up for github contact sales tensorflow/tensorflow an open source machine learning framework for everyone gatsbyjs/gatsby build blazing fast, modern apps and websites with react home-assistant/core 🏡 open source home automation that puts local control and privacy first rust-lang/rust empowering everyone to build reliable and efficient software. flutter/flutter flutter makes it easy and fast to build beautiful apps for mobile and beyond. kubernetes/kubernetes production-grade container scheduling and management apple/swift the swift programming language ansible/ansible ansible is a radically simple it automation platform. hashicorp/terraform terraform enables you to safely and predictably create, change, and improve infrastructure. ohmyzsh/ohmyzsh 🙃 a delightful community-driven framework for managing your zsh configuration. facebook/react a declarative, efficient, and flexible javascript library for building user interfaces. npm/cli the package manager for javascript product features security team enterprise customer stories the readme project pricing resources roadmap platform developer api partners atom electron github desktop support docs community forum professional services learning lab status contact github company about blog careers press social impact shop github on twitter github on facebook github on youtube github on linkedin github’s organization on github © github, inc. terms privacy site map what is git? you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. none none none ranti. centuries.org ranti. centuries.org eternally yours on centuries keeping the dream alive - freiheit i did not recall when the first time i heard it, but i remembered it was introduced by my cousin. this song from münchener freiheit became one of the songs i listen a lot. the lyrics (see below) resonate stronger nowadays. keeping the dream alive (single version) cover by david groeneveld: cover by kim wilde: lyrics:freiheit - keeping the dream alive tonight the rain is fallingfull of memories of people and placesand while the past is callingin my fantasy i remember their faces the hopes we had were much too highway out of reach but we had to trythe game will never be overbecause we're keeping the dream alive i hear myself recallingthings you said to methe night it all startedand still the rain is fallingmakes me feel the wayi felt when we parted the hopes we had were much too highway out of reach but we have to tryno need to hide no need to run'cause all the answers come one by onethe game will never be overbecause we're keeping the dream alive i need youi love you the game will never be overbecause we're keeping the dream alive the hopes we had were much too highway out of reach but we had to tryno need to hide no need to run'cause all the answers come one by one the hopes we had were much too highway out of reach but we had to tryno need to hide no need to run'cause all the answers come one by one the game will never be overbecause we're keeping the dream alivethe game will never be overbecause we're keeping the dream alive the game will never be over… lou reed's walk on the wild side if my memory serves me right, i heard about this walk on the wild side song (wikipedia) sometime during my college year in the s. of course, the bass and guitar reef were the one that captured my attention right away. at that time, being an international student here in the us, i was totally oblivious with the lyrics and the references on it. when i finally understood what the lyrics are about, listening to the song makes more sense. here's the footage of the walk on the wild side song (youtube) but what prompted me to write this was started by the version that amanda palmer sang for neil gaiman. i was listening to her cd "several attempts to cover songs by the velvet underground & lou reed for neil gaiman as his birthday approaches" and one of the songs was walk on the wild side. i like her rendition of the songs, which prompted me to find it on youtube. welp, that platform does not disappoint; it's a quite a nice piano rendition. of course, like any other platform that wants you to stay there, youtube also listed various walk on the wild side cover songs. one of them is from alice phoebe lou a singer-songwriter. her rendition using a guitar is also quite enjoyable (youtube) and now i have a new singer-songwriter to keep an eye on. among other videos that were listed on youtube is the one that kinda blew my mind, walk on the wild side - the story behind the classic bass intro featuring herbie flowers which explained that those are two basses layered on top of each other. man, what a nice thing to learn something new about this song. :-) tao read it from the lazy yogi on climate change read the whole poem tv news archive from the internet archive i just learned about the existence of the tv news archive (covering news from until the day before today's date) containing news shows from us tv such as pbs, cbs, abc, foxnews, cnn, etc. you can search by the captions. they also have several curated collections like news clips regarding nsa or snippets or tv around the world i think some of you might find this useful. quite a nice collection, imo. public domain day (january , ): what could have entered it in and what did get released copyright law is messy, yo. we won't see a lot of notable and important works entering public domain here in the us until . other countries, however, got to enjoy many of them first. public domain reviews put a list of creators whose work are entering the public domain for canada, european union (eu), and many other countries (https://publicdomainreview.org/collections/class-of- /.) for those in eu, nice to see h.g. wells name there (if uk do withdraw, this might end up not applicable to them. but, my knowledge about uk copyright law is zero, so, who knows.) as usual, center of study for the public domain from duke university put a list of some quite well-known works that are still under the extended copyright restriction: http://web.law.duke.edu/cspd/publicdomainday/ /pre- . those works would have been entered the public domain if we use the law that was applicable when they were published. i'm still baffled how current copyright hinders research done and published in to be made available freely. greedy publishers… so, thanks to that, usa doesn't get to enjoy many published works yet. "yet" is the operative word here because we don't know what the incoming administration would do on this topic. considering the next potus is a businessman, i fear the worst. i know: gloomy first of the year thought, but it is what it is. on a cheerful side, check the list from john mark ockerbloom on his online books project. it's quite an amazing project he's been working on. of course, there are also writings made available from hathitrust and gutenberg project, among other things. here's to the next days. xoxo for read the full poem light "light thinks it travels faster than anything but it is wrong. no matter how fast light travels, it finds the darkness has always got there first, and is waiting for it."― terry pratchett, reaper man dot-dot-dot more about bertolt brecht poem assistive technology many people would probably think assistive technology (at) are computer software, applications, or tools that are designed to help blind or deaf people. typically, the first thing that one might have in mind was screen readers, braille display, screen magnifier app for desktop reading, or physical objects like hearing aid, wheel chair, or crutches, a lot of people probably won't think glasses as an at. perhaps because glasses can be highly personalized to fit one's fashion style. woodchuck there's a question how much wood would a woodchuck chuck if a woodchuck could chuck wood. obviously, a woodchuck would chuck wood as much wood as a woodchuck could. shrugs droplets the story of the chinese farmer "you'll never know what would be the consequences of misfortune. or, you'll never know what would be the consequences of good fortune." — alan watts persistent bat is persistent for the last couple weeks or so, there's a bat that somehow managed to sneak in and hid somewhere in the house and then flew frantically in the living room every evening around this time of the day, causing the cats to run and jump around trying to catch it. we caught this bat every time and delivered it outside, hoping it would never return again. but it kept coming back. now i am sort of giving up trying to catch it. even the cats are no longer paying attention to the bat and just give this "meh" face when they spotted it. old window #garage none none none none none none none none none none none none none dshr's blog: isp monopolies dshr's blog i'm david rosenthal, and this is a place to discuss the work i'm doing in digital preservation. tuesday, january , isp monopolies for at least the last three years (it isn't about the technology) i've been blogging about the malign effects of the way the faangs dominate the web and the need for anti-trust action to mitigate them. finally, with the recent lawsuits against facebook and google, some action may be in prospect. i'm planning a post on this topic. but when it comes to malign effects of monopoly i've been ignoring the other monopolists of the internet, the telcos. an insightful recent post by john gilmore to dave farber's ip list sparked a response from thomas leavitt and some interesting follow-up e-mail. gilmore was involved in pioneering consumer isps, and leavitt in pioneering web hosting. both attribute the current sorry state of internet connectivity in the us to the lack of effective competition. they and i differ somewhat on how the problem could be fixed. below the fold i go into the details. i've known gilmore since the early days of sun microsystems, and i'm grateful for good advice he gave me at a critical stage of my career. he has a remarkable clarity of vision and a list of achievements that includes pioneering paid support for free software (the "red hat" model) at cygnus support, and pioneering consumer internet service providers (isps) with the little garden. because in those dial-up days internet connectivity was a commercial product layered on regulated infrastructure (the analog telephone system) there was no lock-in. consumers could change providers simply by changing the number their modem dialed. this experience is key to gilmore's argument. he writes: the usa never had "network neutrality" before it was "suspended". what the usa had was , isps. so if an isp did something unfriendly to its customers, they could just stop paying the bad one, and sign up with a different isp that wouldn't screw them. that effectively prevented bad behavior among isps. and if the customer couldn't find an isp that wouldn't screw them, they could start one themselves. i know, because we did exactly that in the s. anyone could start an isp because by law, everyone had tariffed access to the same telco infrastructure (dialup phone lines, and leased lines at kbit/sec or . mbit/sec or mbit/sec). you just called up the telco and ordered it, and they sent out techs and installed it. we did exactly that, plugged it into our modems and routers and bam, we were an isp: "the little garden". i was an early customer of the little garden. a sparcstation, a scsi disk and a modem sat on my window-ledge. the system dialed a local, and thus free, number and kept the call up / , enabling me to register a domain and start running my own mail server. years later i upgraded to dsl with stanford as my isp. as gilmore points out, stanford could do this under the same law: later, dsl lines required installing equipment in telco central offices, at the far end of the wire that leads to your house. but the telcos were required by the fcc to allow competing companies to do that. their central office buildings were / th empty anyway, after they had replaced racks of mechanical relays with digital computers. gilmore explains how this competitive market was killed: the telcos figured this out, and decided they'd rather be gatekeepers, instead of being the regulated monopoly that gets a fixed profit margin. looking ahead, they formally asked the fcc to change its rule that telcos had to share their infrastructure with everybody -- but only for futuristic optical fibers. they whined that "fcc wants us to deploy fiber everywhere, but we won't, unless we get to own it and not share it with our competitors." as usual, the regulated monopoly was great at manipulating the public interest regulators. the fcc said, "sure, keep your fibers unshared." this ruling never even mentioned the internet, it is all about the physical infrastructure. if the physical stuff is wires, regulated telcos have to share it; if it's glass, they don't. the speed of dialup maxed out at kbit/sec. dsl maxed out at a couple of megabits. leased lines worked to mbit/sec but cost thousands of dollars per month. anything over that speed required fiber, not wire, at typical distances. as demand for higher internet speeds arose, any isp who wanted to offer a faster connection couldn't just order one from the telco, because the telco fibers were now private and unshared. if you want a fiber-based internet connection now, you can't buy it from anybody except the guys who own the fibers -- mostly the telcos. most of the , isps could only offer slow internet access, so everybody stopped paying them. the industry consolidated down to just one or a few businesses per region -- mostly the telcos themselves, plus the cable companies that had build their own local monopoly via city government contracts. especially lucky regions had maybe one other competitor, like a wireless isp, or an electrical co-op that ran fibers on its infrastructure. leavitt makes a bigger point than glimore's: the only reason the internet exists as we know it (mass consumer access) was the regulatory loophole which permitted the isp industry to flourish in the s. the telcos realized their mistake, as john said, and made sure that there wasn't going to be a repeat of that, so with each generation (dsl, fiber), they made it more and more difficult to access their networks, with the result that john mention--almost no consumer choice, for consumers or business. last office i rented, there was one choice of internet provider: the local cable monopoly, which arbitrarily wanted to charge me much more ($ /mo) to connect my office than it did the apartments upstairs in the same building ($ ). as is the case in most of that county; the only alternatives were a few buildings and complexes wired up by the two surviving local isps, and a relatively expensive wisp. gilmore concludes: the telcos' elimination of fiber based competition, and nothing else, was the end of so-called "network neutrality". the rest was just activists, regulators and legislators blathering. there never was an /enforceable federal regulatory policy of network neutrality, so the fcc could hardly suspend it. if the fcc actually wanted us customers to have a choice of isps, they would rescind the fiber rule. and if advocates actually understood how only competition, not regulation, restrains predatory behavior, they would ask fcc for the fiber rule to be rescinded, so a small isp company could rent the actual glass fiber that runs from the telco to (near or inside) your house, for the actual cost plus a regulated profit. then customers could get high speed internet from a variety of vendors at a variety of prices and terms. so far neither has happened. leavitt shows the insane lengths we are resorting to in order to deliver a modicum of competition in the isp market: it's ridiculous that it is going to take sending s of thousands of satellites into orbit to restore any semblance of competitiveness to the isp market, when we've had a simple regulatory fix all along. it's not like the telco/cable monopolies suffered as a result of competition... in fact, it created the market they now monopolize. imagine all the other opportunities for new markets that have been stifled by the lack of competition in the isp market over the last two decades! i have been, and still am, an exception to gilmore's and leavitt's experiences. palo alto owns its own utilities, a great reason to live there. in september palo alto's fiber to the home trial went live, and i was one of citizens who got a mbit/s bidirectional connection, with the city utilities as our isp. we all loved the price, the speed and the excellent customer service. the telcos got worried and threatened to sue the utilities if it expanded the service. the city was on safe legal ground, but that is what they had thought previously when they lost a $ . m lawsuit as part of the fallout from the enron scandal. enron's creditors claimed that the utilities had violated their contract because they stopped paying enron. the utilities did so because enron became unable to deliver them electricity. so, when the trial ended after i think five years, we loved it so much that we negotiated with motorola to take over the equipment and found an upstream isp. but the utilities were gun-shy and spent iirc $ k physically ripping out the fiber and trashing the equipment. since then, palo alto's approach to municipal fiber has been a sorry story of ineffective dithering. shortly after we lost our fiber, stanford decided to stop providing staff with dsl, but we again avoided doing business with the telcos. we got dsl and phone service from sonic, a local isp that was legally enabled to rent access to at&t's copper. it was much slower than comcast or at&t, but the upside was sonic's stellar customer service and four static ip addresses. that kept us going quite happily until covid- struck and we had to host our grandchildren for their virtual schools. dsl was not up to the job. fortunately, it turned out that sonic had recently been able to offer gigabit fiber in palo alto. sonic in its north bay homeland has been deploying its own fiber, as has cruzio in its santa cruz homeland. here they rent access to at&t's fiber in the same way that they rented access to the copper. so, after a long series of delays caused by at&t's inability to get fiber through the conduit under the street that held their copper, we have gigabit speed, home phone and sonic's unmatched customer service all for $ /month. as a long-time sonic customer, i agree with what the internet advisor website writes: sonic has maintained a reputation as not only a company that delivers a reliable high-speed connection to its customers but also a company that stands by its ethics. both dane jasper and scott doty have spoken up on numerous occasions to combat the ever-growing lack of privacy on the web. they have implemented policies that reflect this. in , they reduced the amount of time that they store user data to just two weeks in the face of an ever-growing tide of legal requests for its users’ data. that same year, sonic alongside google fought a court order to hand over email addresses who had contacted and had a correspondence with tor developer and wikileaks contributor jacob applebaum. when asked why, ceo dane jasper responded that it was “rather expensive, but the right thing to do.” sonic has made a habit of doing the right thing, both for its customers and the larger world. it’s a conscientious company that delivers on what is promised and goes the extra mile for its subscribers. leavitt explained in e-mail how sonic's exception to gilmore's argument came about: sonic is one of the few independent isps that's managed to survive the regulatory clampdown via stellar customer service and customers willing to go out of their way to support alternative providers, much like cruzio in my home town of santa cruz. they cut some kind of reseller deal with at&t back in that enabled them to offer fiber to a limited number of residents, and again, like cruzio, are building out their own fiber network, but according to [this site], fiber through them is potentially available to only about , customers (in a state with about million households and million businesses); it also reports that they are the th largest isp in the nation, despite being a highly regional provider with access available to only about million households. this says everything about how monopolistic and consolidated the isp market is, given the number of independent cable and telco companies that existed in previous decades, the remaining survivors of which are all undoubtedly offering isp services. i doubt sonic's deal with at&t was much more lucrative than the dsl deals santa cruz area isps were able to cut. gilmore attempted to build a fiber isp in his hometown, san francisco: our model was to run a fiber to about one person per block (what cruzio calls a "champion") and teach them how to run and debug g ethernet cables down the back fences to their neighbors, splitting down the monthly costs. this would avoid most of the cost of city right-of-way crud at every house, which would let us and our champions fiber the city much more broadly and quickly. and would train a small army of citizens to own and manage their own infrastructure. for unrelated reasons it didn't work out, but it left gilmore with the conviction that, absent repeal of the fiber rule, isp-owned fiber is the way to go. especially in rural areas this approach has been successful; a recent example was described by jon brodkin in jared mauch didn't have good broadband—so he built his own fiber isp. leavitt argues: i'd like to see multiple infrastructure providers, both private for profit, and municipally sponsored non-profit public service agencies, all with open access networks; ideally, connecting would be as simple as it was back in the dial up days. i think we need multiple players to keep each other "honest". i do agree that a lot of the barriers to building out local fiber networks are regulatory and process, as john mentions. the big incumbent players have a tremendous advantage navigating this process, and the scale to absorb the overhead of dealing with them in conjunction with the capital outlays (which municipalities also have). i think we all agree that "ideally, connecting would be as simple as it was back in the dial up days". how to make this happen? as gilmore says, there are regulatory and process costs as well as the cost of pulling the fiber. so if switching away from a misbehaving isp involves these costs there is going to a significant barrier. it isn't going to be "as simple as it was back in the dial up days" when the customer could simply re-program their modem. my experience of municipal fiber leads me to disagree with both gilmore and leavitt. for me, the key is to separate the provision of fiber from the provision of internet services. why would you want to switch providers? pretty much the only reason why you'd want to switch fiber providers is unreliability. but, absent back-hoes, fiber is extremely reliable. there are many reasons why you'd want to switch isps, among them privacy, bandwidth caps, price increases. municipal fiber provision is typically cheap, because they are the regulator and control the permitting process themselves, and because they are accountable to their voters. and if they simply provide the equivalent of an ethernet cable to a marketplace of isps, each of them will be paying the same for their connectivity. so differences in the price of isp service will reflect the features and quality of their service offerings. the cost of switching isps would be low, simply reprogramming the routers at each end of the fiber. the reason the telcos want to own the fiber isn't because owning fiber as such is a good business, it is to impose switching costs and thus lock in their customers. we don't want that. but equally we don't want switching isps to involve redundantly pulling fiber, because that imposes switching costs too. the only way to make connecting "as simple as it was back in the dial up days" is to separate fiber provision from internet service provision, so that fiber pets pulled once and rented to competing isps. if we are going to have a monopoly at the fiber level, i'd rather have a large number of small monopolies than the duopoly of at&t and comcast. and i'd rather have the monopoly accountable to voters. posted by david. at : am labels: platform monopolies no comments: post a comment newer post older post home subscribe to: post comments (atom) blog rules posts and comments are copyright of their respective authors who, by posting or commenting, license their work under a creative commons attribution-share alike . united states license. off-topic or unsuitable comments will be deleted. dshr dshr in anwr recent comments full comments blog archive ▼ ( ) ► february ( ) ▼ january ( ) effort balancing and rate limits isp monopolies the bitcoin "price" two million page views! the new oldweb.today ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► july ( ) ► june ( ) ► february ( ) ► ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) lockss system has permission to collect, preserve, and serve this archival unit. simple theme. powered by blogger. dshr's blog: certificate transparency dshr's blog i'm david rosenthal, and this is a place to discuss the work i'm doing in digital preservation. thursday, november , certificate transparency today is 's world digital preservation day. it might appear that this post has little to do with digital preservation. however, i hope that the long post below the fold is the start of a series asking the simple question underlying not just digital preservation, but many areas of the digital world, "how do i know this digital content is real?" it is another of the many problems for which blockchain is touted as a solution by people lacking real-world understanding of either the problem or the technology, or both. in this post i'm going to look in some detail at certificate transparency, an important initiative aimed at improving security and authenticity on the web, and relate the techniques it uses to those underlying the lockss system. in the next post i plan to ask how these techniques could be applied to other areas in which authenticity is important, such as in the software supply chain. certificate transparency how do i know that i'm talking to the right web site? because there's a closed padlock icon in the url bar, right? mozilla says: a green padlock (with or without an organization name) indicates that: you are definitely connected to the website whose address is shown in the address bar; the connection has not been intercepted. the connection between firefox and the website is encrypted to prevent eavesdropping. the padlock icon appears when the browser has validated that the connection to the url in the url bar supplied a certificate for the site in question carrying a signature chain ending in one of the root certificates the browser trusts. browsers come with a default list of root certificates from certificate authorities (cas). two years ago in the curious case of the outsourced ca i wrote about this list: the list of cas that my firefox trusts is here, all of them. note that because geotrust is in the list, it can certify its own website. i wrote about the issues around cas back in , notably that: the browser trusts all of them equally. the browser trusts cas that the cas on the list delegate trust to. back in , the eff found more than organizations that internet explorer and firefox trusted. commercial cas on the list, and cas they delegate to, have regularly been found to be issuing false or insecure certificates. among the cas on the list are agencies of many governments, such as the dutch, chinese, hong kong, and japanese governments. my current firefox browser trusts root certificates from unique organizations. one of these organizations is the internet security research group, a not-for-profit organization hosted by the linux foundation and sponsored by many organizations including mozilla and the eff, which has greatly improved the information hygiene of the web through a program called let's encrypt. this has provided over million web sites with free certificates carrying a signature chain rooted in a certificate that almost all browsers trust. this blog's certificate is one of them, as you can see by clicking on the padlock icon. some of the organizations whose root certificates my browser trusts are known to have abused this trust. for example, dan goodin writes in one-stop counterfeit certificate shops for all your malware-signing needs: barysevich identified four such sellers of counterfeit certificates since . two of them remain in business today. the sellers offered a variety of options. in , one provider calling himself c@t advertised certificates that used a microsoft technology known as authenticode for signing executable files and programming scripts that can install software. c@t offered code-signing certificates for macos apps as well. ... "in his advertisement, c@t explained that the certificates are registered under legitimate corporations and issued by comodo, thawte, and symantec—the largest and most respected issuers," abuse of the trust users place in cas is routine: over the past few years there have been numerous instances of misissued certificates being used to spoof legitimate sites, and, in some case, install malicious software or spy on unsuspecting users. in one case, a prominent dutch ca (diginotar) was compromised and the hackers were able to use the ca’s system to issue fake ssl certificates. the certificates were used to impersonate numerous sites in iran, such as gmail and facebook, which enabled the operators of the fake sites to spy on unsuspecting site users. in another case, a malaysian subordinate certificate authority (digicert sdn. bhd.), mistakenly issued weak ssl certificates, which could be used to impersonate websites and sign malicious software. as a result, major browsers had to revoke their trust in all certificates issued by digicert sdn. bhd. (note: digicert sdn. bhd. is not affiliated with the u.s.-based corporation digicert, inc.) more recently, a large u.s.-based ca (trustwave) admitted that it issued subordinate root certificates to one of its customers so the customer could monitor traffic on their internal network. subordinate root certificates can be used to create ssl certificates for nearly any domain on the internet. although trustwave has revoked the certificate and stated that it will no longer issue subordinate root certificates to customers, it illustrates just how easy it is for cas to make missteps and just how severe the consequences of those missteps might be. just yesterday, there was another example: the issue with the two headsetup apps came to light earlier this year when german cyber-security firm secorvo found that versions . , . , and . installed two root certification authority (ca) certificates into the windows trusted root certificate store of users' computers but also included the private keys for all in the senncomcckey.pem file. in a report published today, secorvo researchers published proof-of-concept code showing how trivial would be for an attacker to analyze the installers for both apps and extract the private keys. making matters worse, the certificates are also installed for mac users, via headsetup macos app versions, and they aren't removed from the operating system's trusted root certificate store during current headsetup updates or uninstall operations. ... sennheiser's snafu, tracked as cve- - , is not the first of its kind. in , lenovo shipped laptops with a certificate that exposed its private key in a scandal that became known as superfish. dell did the exact same thing in in a similarly bad security incident that became known as edellroot. this type of “mistake” allows attackers to impersonate any web site to affected devices. cas are supposed to issue three grades of certificate based on increasingly rigorous validation: domain validated (dv) certificates verify control over the dns entries, email and web content of the specified domain. they can be issued via automated processes, as with let's encrypt. organization validated (ov) certificates are supposed to verify the legal entity behind the dv-level control of the domain, but in practice are treated the same as dv certificates. extended validation (ev) certificates require "verification of the requesting entity's identity by a certificate authority (ca)". verification is supposed to be an intrusive, human process. source but, as can be seen from the advert, the extended verification process is far from fool-proof. this lack of trustworthiness of cas should not be a surprise. four years ago security collapse in the https market, a fascinating analysis of the (lack of) security on the web from an economic rather than a technical perspective by axel arnbak et al from amsterdam and delft universities showed that cas lack incentives to be trustworthy. arnbak et al write: a crucial technical property of the https authentication model is that any ca can sign certificates for any domain name. in other words, literally anyone can request a certificate for a google domain at any ca anywhere in the world, even when google itself has contracted one particular ca to sign its certificate. this "technical property" is actually important, it is what enables a competitive market of cas. but symantec in particular has exploited it wholesale: google's investigation revealed that over a span of years, symantec cas have improperly issued more than , certificates. such mis-issued certificates represent a potentially critical threat to virtually the entire internet population because they make it possible for the holders to cryptographically impersonate the affected sites and monitor communications sent to and from the legitimate servers. they are a major violation of the so-called baseline requirements that major browser makers impose of cas as a condition of being trusted by major browsers. but symantec has suffered no effective sanctions because they are too big to fail: symantec's repeated violations underscore one of the problems google and others have in enforcing terms of the baseline requirements. when violations are carried out by issuers with a big enough market share they're considered too big to fail. if google were to nullify all of the symantec-issued certificates overnight, it might cause widespread outages. my firefox still trusts symantec root certificates. because google, mozilla and others prioritize keeping the web working over keeping it secure, deleting misbehaving big cas from trust lists won't happen. when mozilla writes: you are definitely connected to the website whose address is shown in the address bar; the connection has not been intercepted. they are assuming a world of honest cas that isn't this world. if you have the locked padlock icon in your url bar, you are probably talking to the right web site, but there is a chance you aren't. even if you are talking to the domain in the url, brian krebs reports that: recent data from anti-phishing company phishlabs shows that percent of all phishing sites in the third quarter of bore the padlock security icon next to the phishing site domain name as displayed in a browser address bar. that’s up from percent just one year ago, and from percent in the second quarter of . efforts to reduce the chance that you aren't have been under way for a long time. in wendlandt et al published perspectives: improving ssh-style host authentication with multi-path probing, describing a system that: thwarts many of these attacks by using a collection of “notary” hosts that observes a server’s public key via multiple network vantage points (detecting localized attacks) and keeps a record of the server’s key over time (recognizing short-lived attacks). clients can download these records on-demand and compare them against an unauthenticated key, this is the basic idea that underlay the early efforts. except for rare cases, such as a certificate being replaced after compromise or expiration, every time a service is accessed the client should receive the same certificate. clients can consult a service that collects the (hash of) certificates as they are received at locations all over the internet to detect the use of fraudulent certificates. five years ago in trust in computer systems i reviewed the state of the art in implementing this concept, including moxie marlinspike's convergence, a distributed approach, and the eff's ssl observatory, a centralized approach. note that both approaches are implemented without participation by certificate authorities or owners, and neither has achieved widespread adoption. in google started work on an approach that requires participation by certificate owners, specified in rfc , and called certificate transparency (ct): google's certificate transparency project fixes several structural flaws in the ssl certificate system, which is the main cryptographic system that underlies all https connections. these flaws weaken the reliability and effectiveness of encrypted internet connections and can compromise critical tls/ssl mechanisms, including domain validation, end-to-end encryption, and the chains of trust set up by certificate authorities. if left unchecked, these flaws can facilitate a wide range of security attacks, such as website spoofing, server impersonation, and man-in-the-middle attacks. certificate transparency helps eliminate these flaws by providing an open framework for monitoring and auditing ssl certificates in nearly real time. specifically, certificate transparency makes it possible to detect ssl certificates that have been mistakenly issued by a certificate authority or maliciously acquired from an otherwise unimpeachable certificate authority. it also makes it possible to identify certificate authorities that have gone rogue and are maliciously issuing certificates. the basic idea is to accompany the certificate with a hash of the certificate signed by a trusted third party, attesting that the certificate holder told the third party that the certificate with that hash was current. thus in order to spoof a service, an attacker would have to both obtain a fraudulent certificate from a ca, and somehow persuade the third party to sign a statement that the service had told them the fraudulent certificate was current. clearly this is: more secure than the current situation, which requires only compromising a ca, and: more effective than client-only approaches, which can detect that a certificate has changed but not whether the change was authorized. ct also requires participation from browser manufacturers: in order to improve the security of extended validation (ev) certificates, google chrome requires certificate transparency (ct) compliance for all ev certificates issued after jan . clients now need two lists of trusted third parties, the cas and the sources of ct attestations. the need for these trusted third parties is where the blockchain enthusiasts would jump in and claim (falsely) that using a blockchain would eliminate the need for trust. but ct has a much more sophisticated approach, ronald reagan's "trust, but verify". in the real world it isn't feasible to solve the problem of untrustworthy cas by eliminating the need for trust. ct's approach instead is to provide a mechanism by which breaches of trust, both by the cas and by the attestors, can be rapidly and unambiguously detected. the ct team write: one of the problems is that there is currently no easy or effective way to audit or monitor ssl certificates in real time, so when these missteps happen (malicious or otherwise), the suspect certificates aren’t usually detected and revoked for weeks or even months. source i don't want to go into too much detail, for that you can read rfc , but here is an overview of how ct works to detect breaches of trust. the system has the following components: logs, to which cas report their current certificates, and from which they obtain attestations, called signed certificate timestamps (scts) that owners can attach to their certificates. clients can verify the signature on the sct, then verify that the hash it contains matches the certificate. if it does, the certificate was the one that the ca reported to the log, and the owner validated. it is envisaged that there will be tens but not thousands of logs; chrome currently trusts logs. each log maintains a merkle tree data structure of the certificates for which it has issued scts. monitors, which periodically download all newly added entries from the logs that they monitor, verify that they have in fact been added to the log, and perform a series of validity checks on them. they also thus act as backups for the logs they monitor. auditors, which use the merkle tree of the logs they audit to verify that certificates have been correctly appended to the log, and that no retroactive insertions, deletions or modifications of the certificates in the log have taken place. clients can use auditors to determine whether a certificate appears in a log. if it doesn't, they can use the sct to prove that the log misbehaved. in this way, auditors, monitors and clients cooperate to verify the correct operation of logs, which in turn provides clients with confidence in the [certificate,attestation] pairs they use to secure their communications. source figure of how certificate transparency works shows how the team believes it would normally be configured. as you can see, the monitor is actually part of the ca, checking at intervals that the log contains only the certificates the ca sent it. the auditor is part of the client, checking at intervals that the certificates in the scts it receives from web servers are correctly stored in the log. although ct works if certificate owners each obtain their scts from only one log, rfc recommends that: tls servers should send scts from multiple logs in case one or more logs are not acceptable to the client (for example, if a log has been struck off for misbehavior or has had a key compromise). in other words, each certificate should be submitted to multiple logs. this is only one of the features of ct that are important for resilience: each log operates independently. each log gets its content directly from the cas, not via replication from other logs. each log contains a subset of the total information content of the system. there is no consensus mechanism operating between the logs, so it cannot be abused by, for example, a % attack. monitoring and auditing is asynchronous to web content delivery, so denial of service against the monitors and auditors cannot prevent clients obtaining service. sustained over long periods it would gradually decrease clients' confidence in the web sites' certificates. looking at the list of logs chrome currently trusts, it is clear that almost all are operated by cas themselves. assuming that each monitor at each ca is monitoring some of the other logs as well as the one it operates, this does not represent a threat, because misbehavior by that ca would be detected by other cas. a ca's monitor that was tempted to cover up misbehavior by a different ca's log it was monitoring would risk being "named and shamed" by some other ca monitoring the same log, just as the misbehaving ca would be "named and shamed". it is important to observe that, despite the fact that cas operate the majority of the ct infrastructure, its effectiveness in disciplining cas is not impaired. arnbak et al write: information asymmetry prevents buyers from knowing what cas are really doing. buyers are paying for the perception of security, a liability shield, and trust signals to third parties. none of these correlates verifiably with actual security. given that ca security is largely unobservable, buyers’ demands for security do not necessarily translate into strong security incentives for cas. negative externalities of the weakest-link security of the system exacerbate these incentive problems. the failure of a single ca impacts the whole ecosystem, not just that ca’s customers. all other things being equal, these interdependencies undermine the incentives of cas to invest, as the security of their customers depends on the efforts of all other cas. and: the market for ssl certificates is highly concentrated, despite the large number of issuers. in fact, both data sets find that around percent of ssl certificates in use on the public web have been issued by just three companies: symantec, godaddy, and comodo. let's encrypt may have had some effect on these numbers, which are from . all three major cas have suffered reputational damage from recent security failures, although because they are "too big to fail" this hasn't impacted their business much. however, as whales in a large school of minnows it is in their interest to impose costs (for implementing ct) and penalties (for security lapses) on the minnows. note that google was sufficiently annoyed with symantec's persistent lack of security that it set up its own ca. the threat that their business could be taken away by the tech oligopoly is real, and cooperating with google may have been the least bad choice. -yr bitcoin "price" because these major corporations have an incentive to pay for the ct infrastructure, it is sustainable in a way that a market of separate businesses, or a permissionless blockchain supported by speculation in a cryptocurrency would not be. fundamentally, if applications such as ct attempt to provide absolute security they are doomed to fail, and their failures will be abrupt and complete. it is more important to provide the highest level of security compatible with resilience, so that the inevitable failures are contained and manageable. this is one of the reasons why permissionless blockchains, subject to % attacks, and permissioned blockchains, with a single, central point of failure, are not suitable. lockss archives care about the authenticity of the content they collect, preserve and disseminate. from the start of the lockss program in we realized that: the lockss system preserves e-journals that have intrinsic value and contain information that powerful interests might want changed or suppressed. to a much greater extent now than when we started, nodes in a lockss network (lockss boxes) collect and disseminate content via https, so ct is an important advance in archival authenticity. five years ago i compared the problem of verifying certificates to the problem of verifying content in a digital preservation system such as lockss: a bad guy subverts an ssl connection from a browser to a website by intercepting the connection and supplying a valid certificate that is different from the one that actually certifies the website. for example, it might be signed by the dept. of homeland security instead of by thawte. the browser trusts both. thus the question is: "how to detect that the certificate is not the one it should be?" this is a similar question to the digital preservation question: "how to detect that the content is not what it should be?" on a second or subsequent visit the browser can compare the certificate with the one it saw earlier, but this doesn't help the first time and it isn't conclusive. there are valid reasons why a certificate can change, for example to replace one that has expired. from the start in the lockss system used similar resilience techniques to ct, for the same reason. each journal article was stored in some, but probably not all of the boxes, and: each box operates independently. each box gets its content directly from the publisher, not via replication from other boxes. each box contains a subset of the total information content of the system. unlike ct, there is a consensus mechanism operating between the boxes. but, unlike blockchains, it is not designed to enforce lockstep agreement among the boxes. it allows a box to discover whether its content matches the consensus of a sample of the other boxes. this is similar to the way that ct's monitors and auditors examine subsets of the logs to detect misbehavior. monitoring and auditing is performed by the boxes themselves, but is asynchronous to web content delivery, so denial of service against the boxes cannot prevent clients obtaining service. sustained over long periods it would gradually decrease clients' confidence in the preserved web content. a decade later, wendlandt et al's perspectives: improving ssh-style host authentication with multi-path probing applied similar "voting among a sample" techniques to the certificate problem: our system, perspectives, thwarts many of these attacks by using a collection of “notary” hosts that observes a server’s public key via multiple network vantage points (detecting localized attacks) and keeps a record of the server’s key over time (recognizing short-lived attacks). clients can download these records on-demand and compare them against an unauthenticated key, detecting many common attacks. it is important to protect against compromised or malign notaries, and this is where lockss-style "voting among a sample" comes in: the final aspect of the notary design is data redundancy, a cross-validation mechanism that limits the power of a compromised or otherwise malicious notary server. to implement data redundancy each notary acts as a shadow server for several other notaries. as described below, a shadow server stores an immutable record of each observation made by another notary. whenever a client receives a query reply from a notary, the client also checks with one or more of that notary’s shadow servers to make sure that the notary reply is consistent with the history stored by the shadow server. just as in the lockss system, if there are enough notaries and enough shadow servers the bad guy can't know which notaries the client will ask, and which shadow servers the client will ask to vote on the replies. posted by david. at : am labels: bitcoin, digital preservation, security, web archiving comments: david. said... tim anderson's google to bury indicator for extended validation certs in chrome because users barely took notice reports that, as troy hunt tweeted: "and that’s that - for all intents and purposes, ev is now dead: “the chrome security ux team has determined that the ev ui does not protect users as intended” https://t.co/w kckccjr " there are two reasons for its demise. the one anderson points to is: "the team have concluded that positive security indicators are largely ineffective. the direction for chrome will be to highlight negative indicators like unencrypted (http) connections, which are marked as "not secure", rather than emphasise when a connection is secure." the other one is that, because certificate authorities have a long history of abusing their trust, users are in fact right to place little reliance on extended validation. as anderson writes: "google's announcement will make it harder for certificate providers to market ev certificates. this is also another reason why you might just as well use free let’s encrypt certificates – no ev from let's encrypt, but it no longer matters." the cas bought this on themselves, and it couldn't happen to a better bunch of crooks. and there's more! shaun nichols reports that web body mulls halving https cert lifetimes. that screaming in the distance is https cert sellers fearing orgs will bail for let's encrypt: "ca/browser forum – an industry body of web browser makers, software developers, and security certificate issuers – is considering slashing the lifetime of https certs from months to months. ... slashing the lifetime may drive organizations into using let's encrypt for free, rather than encourage them to cough up payment more regularly to outfits like digicert. digicert and its ilk charge, typically, hundreds of dollars for their certs: forcing customers to fork out more often may be more of a turn off than a money spinner." august , at : am david. said... cory doctorow writes in creating a "coercion resistant" communications system: "eleanor saitta's ... essay "coercion-resistant design" (which is new to me) is an excellent introduction to the technical countermeasures that systems designers can employ to defeat non-technical, legal attacks: for example, the threat of prison if you don't back-door your product. saitta's paper advises systems designers to contemplate ways to arbitrage both the rule of law and technical pre-commitments to make it harder for governments to force you to weaken the security of your product or compromise your users. a good example of this is certificate transparency, a distributed system designed to catch certificate authorities that cheat and issue certificates to allow criminals or governments to impersonate popular websites like google." september , at : pm post a comment newer post older post home subscribe to: post comments (atom) blog rules posts and comments are copyright of their respective authors who, by posting or commenting, license their work under a creative commons attribution-share alike . united states license. off-topic or unsuitable comments will be deleted. dshr dshr in anwr recent comments full comments blog archive ► ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ▼ ( ) ► december ( ) ▼ november ( ) certificate transparency cryptocurrency collapse john wharton rip cryptocurrencies' seven deadly paradoxes kids today have no idea what's happening to storage? making pies is hard ithaka's perspective on digital preservation ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► july ( ) ► june ( ) ► february ( ) ► ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) lockss system has permission to collect, preserve, and serve this archival unit. simple theme. powered by blogger. none none none none library hat library hat http://www.bohyunkim.net/blog/ skip to content ↓ bohyunkim.net about publications presentations cv / resume blockchain: merits, issues, and suggestions for compelling use cases jul th, by bohyun (library hat). comments are off for this post * this post was also published in acrl techconnect.*** blockchain holds a great potential for both innovation and disruption. the adoption of blockchain also poses certain risks, and those risks will need to be addressed and mitigated before blockchain becomes mainstream. a lot of people have heard of blockchain at this point. but many are unfamiliar with how this new technology exactly works and unsure about under which circumstances or on what conditions it may be useful to libraries. in this post, i will provide a brief overview of the merits and the issues of blockchain. i will also make some suggestions for compelling use cases of blockchain at the end of this post. what blockchain accomplishes blockchain is the technology that underpins a well-known decentralized cryptocurrency, bitcoin. to simply put, blockchain is a kind of distributed digital ledger on a peer-to-peer (p p) network, in which records are confirmed and encrypted. blockchain records and keeps data in the original state in a secure and tamper-proof manner[ ] by its technical implementation alone, thereby obviating the need for a third-party authority to guarantee the authenticity of the data. records in blockchain are stored in multiple ledgers in a distributed network instead of one central location. this prevents a single point of failure and secures records by protecting them from potential damage or loss. blocks in each blockchain ledger are chained to one another by the mechanism called ‘proof of work.’ (for those familiar with a version control system such as git, a blockchain ledger can be thought of as something similar to a p p hosted git repository that allows sequential commits only.[ ]) this makes records in a block immutable and irreversible, that is, tamper-proof. in areas where the authenticity and security of records is of paramount importance, such as electronic health records, digital identity authentication/authorization, digital rights management, historic materials that may be contested or challenged due to the vested interests of certain groups, and digital provenance to name a few, blockchain can lead to efficiency, convenience, and cost savings. for example, with blockchain implemented in banking, one will be able to transfer funds across different countries without going through banks.[ ] this can drastically lower the fees involved, and the transaction will take effect much more quickly, if not immediately. similarly, adopted in real estate transactions, blockchain can make the process of buying and selling a property more straightforward and efficient, saving time and money.[ ] disruptive potential of blockchain the disruptive potential of blockchain lies in its aforementioned ability to render the role of a third-party authority obsolete, which records and validates transactions and guarantees their authenticity, should a dispute arise. in this respect, blockchain can serve as an alternative trust protocol that decentralizes traditional authorities. since blockchain achieves this by public key cryptography, however, if one loses one’s own personal key to the blockchain ledger holding one’s financial or real estate asset, for example, then that will result in the permanent loss of such asset. with the third-party authority gone, there will be no institution to step in and remedy the situation. issues this is only some of the issues with blockchain. other issues include (a) interoperability between different blockchain systems, (b) scalability of blockchain at a global scale with large amount of data, (c) potential security issues such as the % attack [ ], and (d) huge energy consumption [ ] that a blockchain requires to add a block to a ledger. note that the last issue of energy consumption has both environmental and economic ramifications because it can cancel out the cost savings gained from eliminating a third-party authority and related processes and fees. challenges for wider adoption there are growing interests in blockchain among information professionals, but there are also some obstacles to those interests gaining momentum and moving further towards wider trial and adoption. one obstacle is the lack of general understanding about blockchain in a larger audience of information professionals. due to its original association with bitcoin, many mistake blockchain for cryptocurrency. another obstacle is technical. the use of blockchain requires setting up and running a node in a blockchain network, such as ethereum[ ], which may be daunting to those who are not tech-savvy. this makes a barrier to entry high to those who are not familiar with command line scripting and yet still want to try out and test how a blockchain functions. the last and most important obstacle is the lack of compelling use cases for libraries, archives, and museums. to many, blockchain is an interesting new technology. but even many blockchain enthusiasts are skeptical of its practical benefits at this point when all associated costs are considered. of course, this is not an insurmountable obstacle. the more people get familiar with blockchain, the more ways people will discover to use blockchain in the information profession that are uniquely beneficial for specific purposes. suggestions for compelling use cases of blockchain in order to determine what may make a compelling use case of blockchain, the information profession would benefit from considering the following. what kind of data/records (or the series thereof) must be stored and preserved exactly the way they were created. what kind of information is at great risk to be altered and compromised by changing circumstances. what type of interactions may need to take place between such data/records and their users.[ ] how much would be a reasonable cost for implementation. these will help connecting the potential benefits of blockchain with real-world use cases and take the information profession one step closer to its wider testing and adoption. to those further interested in blockchain and libraries, i recommend the recordings from the library . online mini-conference, “blockchain applied: impact on the information profession,” held back in june. the blockchain national forum, which is funded by imls and is to take place in san jose, ca on august th, will also be livestreamed. notes [ ] for an excellent introduction to blockchain, see “the great chain of being sure about things,” the economist, october , , https://www.economist.com/news/briefing/ -technology-behind-bitcoin-lets-people-who-do-not-know-or-trust-each-other-build-dependable. [ ] justin ramos, “blockchain: under the hood,” thoughtworks (blog), august , , https://www.thoughtworks.com/insights/blog/blockchain-under-hood. [ ] the world food programme, the food-assistance branch of the united nations, is using blockchain to increase their humanitarian aid to refugees. blockchain may possibly be used for not only financial transactions but also the identity verification for refugees. russ juskalian, “inside the jordan refugee camp that runs on blockchain,” mit technology review, april , , https://www.technologyreview.com/s/ /inside-the-jordan-refugee-camp-that-runs-on-blockchain/. [ ] joanne cleaver, “could blockchain technology transform homebuying in cook county — and beyond?,” chicago tribune, july , , http://www.chicagotribune.com/classified/realestate/ct-re- -blockchain-homebuying- -story.html. [ ] “ % attack,” investopedia, september , , https://www.investopedia.com/terms/ / -attack.asp. [ ] sherman lee, “bitcoin’s energy consumption can power an entire country — but eos is trying to fix that,” forbes, april , , https://www.forbes.com/sites/shermanlee/ / / /bitcoins-energy-consumption-can-power-an-entire-country-but-eos-is-trying-to-fix-that/# ff aa bc . [ ] osita chibuike, “how to setup an ethereum node,” the practical dev, may , , https://dev.to/legobox/how-to-setup-an-ethereum-node- a . [ ] the interaction can also be a self-executing program when certain conditions are met in a blockchain ledger. this is called a “smart contract.” see mike orcutt, “states that are passing laws to govern ‘smart contracts’ have no idea what they’re doing,” mit technology review, march , , https://www.technologyreview.com/s/ /states-that-are-passing-laws-to-govern-smart-contracts-have-no-idea-what-theyre-doing/. posted in: coding, library, technology. tagged: bitcoin · blockchain · distributed ledger technology · dlt taking diversity to the next level dec th, by bohyun (library hat). comments are off for this post ** this post was also published in acrl techconnect on dec. , .*** “building bridges in a divisive climate: diversity in libraries, archives, and museums,” panel discussion program held at the university of rhode island libraries on thursday november , . getting minorities on board i recently moderated a panel discussion program titled “building bridges in a divisive climate: diversity in libraries, archives, and museums.” participating in organizing this program was interesting experience. during the whole time, i experienced my perspective constantly shifting back and forth as (i) someone who is a woman of color in the us who experiences and deals with small and large daily acts of discrimination, (ii) an organizer/moderator trying to get as many people as possible to attend and participate, and (iii) a mid-career librarian who is trying to contribute to the group efforts to find a way to move the diversity agenda forward in a positive and inclusive way in my own institution. in the past, i have participated in multiple diversity-themed programs either as a member of the organizing committee or as an attendee and have been excited to see colleagues organize and run such programs. but when asked to write or speak about diversity myself, i always hesitated and declined. this puzzled me for a long time because i couldn’t quite pinpoint where my own resistance was coming from. i am writing about this now because i think it may shed some light on why it is often difficult to get minorities on board with diversity-related efforts. a common issue that many organizers experience is that often these diversity programs draw many allies who are already interested in working on the issue of diversity, equity, and inclusion but not necessarily a lot of those who the organizers consider to be the target audience, namely, minorities. what may be the reason? perhaps i can find a clue for the answer to this question from my own resistance regarding speaking or writing about diversity, preferring rather to be in the audience with a certain distance or as an organizer helping with logistics behind the scene. to be honest, i always harbored a level of suspicion about how much of the sudden interests in diversity is real and how much of it is simply about being on the next hot trend. trends come and go, but issues lived through many lives of those who belong to various systematically disadvantaged and marginalized groups are not trends. although i have been always enthusiastic about participating in diversity-focused programs as attendees and was happy to see diversity, equity, and inclusion discussed in articles and talks, i wasn’t ready to sell out my lived experience as part of a hot trend, a potential fad. to be clear, i am not saying that any of the diversity-related programs or events were asking speakers or authors to be a sell-out. i am only describing how things felt to me and where my own resistance was originating. i have been and am happy to see diversity discussed even as a one-time fad. better a fad than no discussion at all. one may argue that that diversity has been actively discussed for quite some time now. a few years, maybe several, or even more. some of the prominent efforts to increase diversity in librarianship i know, for example, go as far back as when oregon state university libraries sponsored two scholarships to the code lib conference, one for women and the other for minorities, which have continued from then on as the code lib diversity scholarship. but if one has lived the entire life as a member of a systematically disadvantaged group either as a woman, a person of color, a person of certain sexual orientation, a person of a certain faith, a person with a certain disability, etc., one knows better than expecting some sudden interests in diversity to change the world we live in and most of the people overnight. i admit i have been watching the diversity discussion gaining more and more traction in librarianship with growing excitement and concern at the same time. for i felt that all of what is being achieved through so many people’s efforts may get wiped out at any moment. the more momentum it accrues, i worried, the more serious backlash it may come to face. for example, it was openly stated that seeking racial/ethnic diversity is superficial and for appearance’s sake and that those who appear to belong to “team diversity” do not work as hard as those in “team mainstream.” people make this type of statements in order to create and strengthen a negative association between multiple dimensions of diversity that are all non-normative (such as race/ethnicity, religion, sexual orientation, immigration status, disability) and unfavorable value judgements (such as inferior intellectual capacity or poor work ethic). according to this kind of flawed reasoning, a tech company whose entire staff consists of twenty-something white male programmers with a college degree, may well have achieved a high level of diversity because the staff might have potentially (no matter how unlikely) substantial intellectual and personal differences in their thinking, background, and experience, and therefore their clear homogeneity is no real problem. that’s just a matter of trivial “appearance.” the motivation behind this kind of intentional misdirection is to derail current efforts towards expanding diversity, equity, and inclusion by taking people’s attention away from the real issue of systematic marginalization in our society. of course, the ultimate goal of all diversity efforts should be not the mere inclusion of minorities but enabling them to have agency as equal as the agency those privileged already possess. but note that objections are being raised against mere inclusion. anti-diversity sentiment is real, and people will try to rationalize it in any way they can. then of course, the other source of my inner resistance to speaking or writing about diversity has been the simple fact that thinking about diversity, equity, and inclusion does not take me to a happy place. it reminds me of many bad experiences accumulated over time that i would rather not revisit. this is why i admire those who have spoken and written about their lived experience as a member of a systematically discriminated and marginalized group. their contribution is a remarkably selfless one. i don’t have a clear answer to how this reflection on my own resistance against actively speaking or writing about diversity will help future organizers. but clearly, being asked to join many times had an effect since i finally did accept the invitation to moderate a panel and wrote this article. so, if you are serious about getting more minorities – whether in different religions, genders, disabilities, races, etc. – to speak or write on the issue, then invite them and be ready to do it over and over again even if they decline. don’t expect that they will trust you at the first invitation. understand that by accepting such an invitation, minorities do risk far more than non-minorities will ever do. the survey i ran for the registrants of the “building bridges in a divisive climate: diversity in libraries, archives, and museums” panel discussion program showed several respondents expressing their concern about the backlash at their workplaces that did or may result from participating in diversity efforts as a serious deterrent. if we would like to see more minorities participate in diversity efforts, we must create a safe space for everyone and take steps to deal with potential backlash that may ensue afterwards. a gentle intro or a deep dive? another issue that many organizers of diversity-focused events, programs, and initiatives struggle with is two conflicting expectations from their audience. on one hand, there are those who are familiar with diversity, equity, and inclusion issues and want to see how institutions and individuals are going to take their initial efforts to the next level. these people often come from organizations that already implemented certain pro-diversity measures such as search advocates for the hiring process. and educational programs that familiarize the staff with the topic of diversity, equity, and inclusion. on the other hand, there are still many who are not quite sure what diversity, equity, and inclusion exactly mean in a workplace or in their lives. those people would continue to benefit from a gentle introduction to things such as privilege, microaggression, and unconscious biases. the feedback surveys collected after the “building bridges in a divisive climate: diversity in libraries, archives, and museums” panel discussion program showed these two different expectations. some people responded that they deeply appreciated the personal stories shared by the panelists, noting that they did not realize how often minorities are marginalized even in one day’s time. others, however, said they would be like to hear more about actionable items and strategies that can be implemented to further advance the values of diversity, equity, and inclusion that go beyond personal stories. balancing these two different demands is a hard act for organizers. however, this is a testament to our collective achievement that more and more people are aware of the importance of continuing efforts to improve diversity, equity, and inclusion in libraries, archives, and museums. i do think that we need to continue to provide a general introduction to diversity-related issues, exposing people to everyday experience of marginalized groups such as micro-invalidation, impostor syndrome, and basic concepts like white privilege, systematic oppression, colonialism, and intersectionality. one of the comments we received via the feedback survey after our diversity panel discussion program was that the program was most relevant in that it made “having colleagues attend with me to hear what i myself have never told them” possible. general programs and events can be an excellent gateway to more open and less guarded discussion. at the same time, it seems to be high time for us in libraries, museums, and archives to take a deep dive into different realms of diversity, equity, and inclusion as well. diversity comes in many dimensions such as age, disability, religion, sexual orientation, race/ethnicity, and socioeconomic status. many of us feel more strongly about one issue than others. we should create opportunities for ourselves to advocate for specific diversity issues that we care most. the only thing i would emphasize is that one specific dimension of diversity should not be used as an excuse to neglect others. exploring socioeconomic inequality issues without addressing how they work combined with the systematic oppression of marginalized groups such as native americans, women, or immigrants at the same time can be an example of such a case. all dimensions of diversity are closely knitted with one another, and they do not exist independently. for this reason, a deep dive into different realms of diversity, equity, and inclusion must be accompanied by the strong awareness of their intersectionality. recommendations and resources for future organizers organizing a diversity-focused program takes a lot of effort. while planning the “building bridges in a divisive climate: diversity in libraries, archives, and museums” panel discussion program at the university of rhode island libraries, i worked closely with my library dean, karim boughida, who originally came up with the idea of having a panel discussion program at the university of rhode island libraries, and renee neely in the libraries’ diversity initiatives for approximately two months. for panelists, we decided to recruit as many minorities from diverse institutions and backgrounds. we were fortunate to find panelists from a museum, an archive, both a public and an academic library with varying degrees of experience in the field from only a few years to over twenty-five years, ranging from a relatively new archivist to an experienced museum and a library director. our panel consisted of one-hundred percent people of color. the thoughts and perspectives that those panelists shared were, as a result, remarkably diverse and insightful. for this reason, i recommend spending some time to get the right speakers for your program if your program will have speakers. discussion at the “building bridges in a divisive climate: diversity in libraries, archives, and museums,” at the university of rhode island libraries another thing i would like to share is the questions that i created for the panel discussion. even though we had a whole hour, i was able to cover only several of them. but since i discussed all these questions in advance with the panelists and they helped me put a final touch on some of those, i think these questions can be useful to future organizers who may want to run a similar program. they can be utilized for a panel discussion, an unconference, or other types of programs. i hope this is helpful and save time for other organizers. sample questions for the diversity panel discussion why should libraries, archives, museums pay attention to the issues related to diversity, equity, and inclusion? in what ways do you think the lack of diversity in our profession affects the perception of libraries, museums, and archives in the communities we serve? do you have any personal or work-related stories that you would like to share that relate to diversity, equity, and inclusion issues? how did you get interested in diversity, equity, and inclusion issues? suppose you discovered that your library’s, archive’s or museum’s collection includes prejudiced information, controversial objects/ documents, or hate-inducing material. what would you do? suppose a group of your library / archive / museum patrons want to use your space to hold a local gathering that involves hate speech. what would you do? what would you be mostly concerned about, and what would the things that you would consider to make a decision on how you will respond? do you think libraries, archives, and museums are a neutral place? what do you think neutrality means to a library, an archive, a museum in practice in a divisive climate such as now? what are some of the areas in libraries, museums, and archives where you see privileges and marginalization function as a barrier to achieving our professional values – equal access and critical thinking? what can we do to remove those barriers? could you tell us how colonialist thinking and practice are affecting libraries, museums, and archives either consciously or unconsciously? since not everyone is familiar with what colonialism is, please begin with first your brief interpretation of what colonialist thinking or practice look like in libraries, museums, and archives first? what do you think libraries, archives, and museums can do more to improve critical thinking in the community that we serve? although libraries, archives, museums have been making efforts to recruit, hire, and retain diverse personnel in recent years, the success rate has been relatively low. for example, in librarianship, it has been reported that often those hired through these efforts experienced backlash at their own institutions, were subject to unrealistic expectations, and met with unsupportive environment, which led to burnout and a low retention rate of talented people. from your perspective – either as a manager hiring people or a relatively new librarian who looked for jobs – what do you think can be done to improve this type of unfortunate situation? many in our profession express their hesitation to actively participate in diversity, equity, and inclusion-related discussion and initiatives at their institutions because of the backlash from their own coworkers. what do you think we can do to minimize such backlash? some people in our profession express strong negative feelings regarding diversity, equity, and inclusion-related initiatives. how much of this type of anti-diversity sentiment do you think exist in your field? some worry that this is even growing faster in the current divisive and intolerant climate. what do you think we can do to counter such anti-diversity sentiment? there are many who are resistant to the values of diversity, equity, and inclusion. have you taken any action to promote and advance these values facing such resistance? if so, what was your experience like, and what would be some of the strategies you may recommend to others working with those people? many people in our profession want to take our diversity, equity, and inclusion initiatives to the next level, beyond offering mere lip service or simply playing a numbers game for statistics purpose. what do you think that next level may be? lastly, i felt strongly about ensuring that the terms and concepts often thrown out in diversity/equity/inclusion-related programs and events – such as intersectionality, white privilege, microaggression, patriarchy, colonialism, and so on – are not used to unintentionally alienate those who are unfamiliar with them. these concepts are useful and convenient shortcuts that allow us to communicate a large set of ideas previously discussed and digested, so that we can move our discussion forward more efficiently. they should not make people feel uncomfortable nor generate any hint of superiority or inferiority. to this end, i create a pre-program survey which all program registrants were encouraged to take. my survey simply asked people how familiar and how comfortable they are with a variety of terms. at the panel discussion program, we also distributed the glossary of these terms, so that people can all become familiar with them. also, videos can quickly bring all attendees up-to-speed with some basic concepts and phenomena in diversity discussion. for example, in the beginning of our panel discussion program, i played two short videos, “life of privilege explained in a $ race” and “what if we treated white coworkers the way we treat minority coworkers?”, which were well received by the attendees. i am sharing the survey questions, the video links, and the glossary in the hope that they may be helpful as a useful tool for future organizers. for example, one may decide to provide a glossary like this before the program or run an unconference that aims at unpacking the meanings of these terms and discussing how they relate to people’s daily lives. in closing: diversity, libraries, technology, and our own biases disagreements on social issues are natural. but the divisiveness that we are currently experiencing seems to be particularly intense. this deeply concerns us, educators and professionals working in libraries, archives, and museums. libraries, archives, and museums are public institutions dedicated to promoting and advancing civic values. diversity, equity, and inclusion are part of those core civic values that move our society forward. this task, however, has become increasingly challenging as our society moves in a more and more divisive direction. to make matters even more complicated, libraries, archives, museums in general lack diversity in their staff composition. this homogeneity can impede achieving our own mission. according to the recent report from ithaka s+r released this august, we do not appear to have gotten very far. their report “inclusion, diversity, and equity: members of the association of research (arl) libraries – employee demographics and director perspectives,” shows that libraries and library leadership/administration are both markedly white-dominant ( % and % white non-hispanic respectively). also, while librarianship in general are female dominant ( %), the technology field in libraries is starkly male ( %) along with makerspace ( %), facilities ( %), and security ( %) positions. the survey results in the report show that while the majority of library directors say there are barriers to achieving more diversity in their library, they attribute those barriers to external rather than internal factors such as the library’s geographic location and the insufficiently diverse application pool resulting from the library’s location. what is fascinating, however, is that this directly conflicts with the fact that libraries do show little variation in the ratio of white staff based on degree of urbanization. equally interesting is that the staff in more homogeneous and less diverse (over % white non-hispanic) libraries think that their libraries are much more equitable than the library community ( % vs %) and that library directors (and staff) consider their own library to be more equitable, diverse, and inclusive than the library community with respect to almost every category such as race/ethnicity, gender, lgbtq, disabilities, veterans, and religion. while these findings in the ithaka s+r report are based upon the survey results from arl libraries, similar staff composition and attitudes can be assumed to apply to libraries in general. there is a great need for both the library administration and the staff to understand their own unconscious and implicit biases, workplace norms, and organizational culture that may well be thwarting their own diversity efforts. diversity, equity, and inclusion have certainly been a topic of active discussion in the recent years. many libraries have established a committee or a task force dedicated to improving diversity. but how are those efforts paying out? are they going beyond simply paying a lip service? is it making a real difference to everyday experience of minority library workers? can we improve, and if so where and how? where do we go from here? those would be the questions that we will need to examine in order to take our diversity efforts in libraries, archives, and museums to the next level. notes the program description is available at https://web.uri.edu/library/ / / /building-bridges-in-a-divisive-climate-diversity-in-libraries-archives-and-museums/ ↩ carol bean, ranti junus, and deborah mouw, “conference report: code libcon ,” the code lib journal, no. (march , ), http://journal.code lib.org/articles/ . ↩ note that this kind of biased assertions often masquerades itself as an objective intellectual pursuit in academia when in reality, it is a direct manifestation of an existing prejudice reflecting the limited and shallow experience of the person posting the question him/herself. a good example of this is found in the remark in made by larry summers, the former harvard president. he suggested that one reason for relatively few women in top positions in science may be “issues of intrinsic aptitude” rather than widespread indisputable everyday discrimination against women. he resigned after the harvard faculty of arts and sciences cast a vote of no confidence. see scott jaschik, “what larry summers said,” inside higher ed, february , , https://www.insidehighered.com/news/ / / /summers _ . ↩ our pre-program survey questions can be viewed at https://docs.google.com/forms/d/e/ faipqlscp-nqnkhaqli_ pvdidw-dqzraflycdikutu dzjqm f ra/viewform. ↩ for this purpose, asking all participants to respect one another’s privacy in advance can be a good policy. in addition to this, we specifically decided not to stream or record our panel discussion program, so that both panelists and attendees can freely share their experience and thoughts. ↩ a good example is the search advocate program from oregon state university. see http://searchadvocate.oregonstate.edu/. ↩ for an example, see the workshops offered by the office of community, equity, and inclusion of the university of rhode island at https://web.uri.edu/diversity/ced-inclusion-courses-overview/. ↩ for the limitations of the mainstream diversity discussion in lis (library and information science) with the focus on inclusion and cultural competency, see david james hudson, “on ‘diversity’ as anti-racism in library and information studies: a critique,” journal of critical library and information studies , no. (january , ), https://doi.org/https://doi.org/ . /jclis.v i . . ↩ you can see our glossary at https://drive.google.com/file/d/ uci huuytrelgny-dbnsoxf_ilpm n/view?usp=sharing; this glossary was put together by renee neely. ↩ for the nitty-gritty logistical details for organizing a large event with a group of local and remote volunteers, check the organizer’s toolkit created by the #critlib unconference organizers at https://critlib .wordpress.com/organizers-toolkit/. ↩ roger schonfeld and liam sweeney, “inclusion, diversity, and equity: members of the association of research libraries,” ithaka s+r, august , , http://www.sr.ithaka.org/publications/inclusion-diversity-and-equity-arl/. ↩ for the early discussion of diversity-focused recruitment in library technology, see jim hahn, “diversity recruitment in library information technology,” acrl techconnect blog, august , , https://acrl.ala.org/techconnect/post/diversity-recruitment-in-library-information-technology. ↩ see april hathcock, “white librarianship in blackface: diversity initiatives in lis,” in the library with the lead pipe, october , , http://www.inthelibrarywiththeleadpipe.org/ /lis-diversity/ and angela galvan, “soliciting performance, hiding bias: whiteness and librarianship,” in the library with the lead pipe (blog), june , , http://www.inthelibrarywiththeleadpipe.org/ /soliciting-performance-hiding-bias-whiteness-and-librarianship. ↩ posted in: diversity. tagged: equity · inclusion · resources from need to want: how to maximize social impact for libraries, archives, and museums oct th, by bohyun (library hat). comments are off for this post at the ndp at three event organized by imls yesterday, sayeed choudhury on the “open scholarly communications” panel suggested that libraries think about return on impact in addition to return on investment (roi). he further elaborated on this point by proposing a possible description of such impact. his description was that when an object or resource created through scholarly communication efforts is being used by someone we don’t know and is interpreted correctly without contacting us (=libraries, archives, museums etc.), that is an impact; to push that further, if someone uses the object or the resource in a way we didn’t anticipate, that’s an impact; if it is integrated into someone’s workflow, that’s also an impact. this emphasis on impact as a goal for libraries, archives, and museums (or non-profit organizations in general to apply broadly) resonated with me particularly because i gave a talk just a few days ago to a group of librarians at the iolug conference about how libraries can and should maximize their social impact in the context of innovation in the way many social entrepreneurs have been already doing for quite some time. in this post, i would like to revisit one point that i made in that talk. it is a specific interpretation of the idea of maximizing social impact as a conscious goal for libraries, archives, and museums (lam). hopefully, this will provide a useful heuristic for lam institutions in mapping out the future efforts. considering that roi is a measure of cost-effectiveness, i believe impact is a much better goal than roi for lam institutions. we often think that to collect, organize, provide equitable access to, and preserve information, knowledge, and cultural heritage is the goal of a library, an archive, and a museum. but doing that well doesn’t mean simply doing it cost-effectively. our efforts no doubt aim at achieving better-collected, better-organized, better-accessed, and better-preserved information, knowledge, and cultural heritage. however, our ultimate end-goal is attained only when such information, knowledge, and cultural heritage is better used by our users. not simply better accessed, but better used in the sense that the person gets to leverage such information, knowledge, and cultural heritage to succeed in whatever endeavor that s/he was making, whether it be career success, advanced education, personal fulfillment, or private business growth. in my opinion, that’s the true impact that lam institutions should aim at. if that kind of impact were a destination, cost-effectiveness is simply one mode of transportation, preferred one maybe but not quite comparable to the destination in terms of importance. but what does “better used” exactly mean? “integrated into people’s workflow” is a hint; “unanticipated use” is another clue. if you are like me and need to create and design that kind of integrated or unanticipated use at your library, archive, or museum, how will you go about that? this is the same question we ask over and over again. how do you plan and implement innovation? yes, we will go talk to our users, ask what they would like to see, meet with our stakeholders and find out their interests and concerns are, discuss ourselves what we can do to deliver things that our users want, and go from there to another wonderful project we work hard for. then after all that, we reach a stage where we stop and wonder where that “greater social impact” went in almost all our projects. and we frantically look for numbers. how many people accessed what we created? how many downloads? what does the satisfaction survey say? in those moments, how does the “impact” verbiage help us? how does that help us in charting our actual path to creating and maximizing our social impact more than the old-fashioned “roi” verbiage? at least roi is quantifiable and measurable. this, i believe, is why we need a more concrete heuristic to translate the lofty “impact” to everyday “actions” we can take. maybe not quite as specific as to dictate what exactly those actions are at each project level but a bit more specific to enable us to frame the value we are attempting to create and deliver at our lam institutions beyond cost-effectiveness. i think the heuristic we need is the conversion of need to demand. what is an untapped need that people are not even aware of in the realm of information, knowledge, and cultural heritage? when we can identify any such need in a specific form and successfully convert that need to a demand, we make an impact. by “demand,” i mean the kind of user experience that people will desire and subsequently fulfill by using that object, resource, tool, service, etc., we create at our library, archive, and museum. (one good example of such desirable ux that comes to my mind is nypl photo booth: https://www.nypl.org/blog/ / / /snapshots-nypl.) when we create a demand out of such an untapped need, when the fulfillment of that kind of demand effectively creates, strengthens, and enriches our society in the direction of information, knowledge, evidence-based decisions, and truth being more valued, promoted, and equitably shared, i think we get to maximize our social impact. in the last “going forward” panel where the information discovery was discussed, loretta parham pointed out that in the corporate sector, information finds consumers, not the other way. by contrast, we (by which i mean all of us working at lam institutions) still frame our value in terms of helping and supporting users access and use our material, resources, and physical and digital objects and tools. this is a mistake in my opinion, because it is a self-limiting value proposition for libraries, archives, and museums. what is the point of us lam institutions, working so hard to get the public to use their resources and services? the end goal is so that we can maximize our social impact through such use. the rhetoric of “helping and supporting people to access and use our resources” does not adequately convey that. businesses want their clients to use their goods and services, of course. but their real target is the making of profit out of those uses, aka purchases. similarly, but far more importantly, the real goal of libraries, archives and museums is to move the society forward, closer in the direction of knowledge, evidence-based decisions, and truth being more valued, promoted, and equitably shared. one person at a time, yes, but the ultimate goal reaching far beyond individuals. the end goal is maximizing our impact on this side of the public good. posted in: librarianship, library, management, usability, user experience. tagged: archives · change · d d · design thinking · digital collection · goal · impact · innovation · libraries · museums · ndpthree · social entrepreneurship · ux how to price d printing service fees may nd, by bohyun (library hat). comments are off for this post ** this post was originally published in acrl techconnect on may. , .*** many libraries today provide d printing service. but not all of them can afford to do so for free. while free d printing may be ideal, it can jeopardize the sustainability of the service over time. nevertheless, many libraries tend to worry about charging service fees. in this post, i will outline how i determined the pricing schema for our library’s new d printing service in the hope that more libraries will consider offering d printing service if having to charge the fee is a factor stopping them. but let me begin with libraries’ general aversion to fees. a d printer in action at the health sciences and human services library (hs/hsl), univ. of maryland, baltimore service fees are not your enemy charging fees for the library’s service is not something librarians should regard as a taboo. we live in the times in which a library is being asked to create and provide more and more new and innovative services to help users successfully navigate the fast-changing information landscape. a makerspace and d printing are certainly one of those new and innovative services. but at many libraries, the operating budget is shrinking rather than increasing. so, the most obvious choice in this situation is to aim for cost-recovery. it is to be remembered that even when a library aims for cost-recovery, it will be only partial cost-recovery because there is a lot of staff time and expertise that is spent on planning and operating such new services. libraries should not be afraid to introduce new services requiring service fees because users will still benefit from those services often much more greatly than a commercial equivalent (if any). think of service fees as your friend. without them, you won’t be able to introduce and continue to provide a service that your users need. it is a business cost to be expected, and libraries will not make profit out of it (even if they try). still bothered? almost every library charges for regular (paper) printing. should a library rather not provide printing service because it cannot be offered for free? library users certainly wouldn’t want that. determining your service fees what do you need in order to create a pricing scheme for your library’s d printing service? (a) first, you need to list all cost-incurring factors. those include (i) the equipment cost and wear and tear, (ii) electricity, (iii) staff time & expertise for support and maintenance, and (iv) any consumables such as d print filament, painter’s tape. remember that your new d printer will not last forever and will need to be replaced by a new one in - years. also, some of these cost-incurring factors such as staff time and expertise for support is fixed per d print job. on the other hand, another cost-incurring factor, d print filament, for example, is a cost factor that increases in proportion to the size/density of a d model that is printed. that is, the larger and denser a d print model is, the more filament will be used incurring more cost. (b) second, make sure that your pricing scheme is readily understood by users. does it quickly give users a rough idea of the cost before their d print job begins? an obscure pricing scheme can confuse users and may deter them from trying out a new service. that would be bad user experience. also in d printing, consider if you will also charge for a failed print. perhaps you do. perhaps you don’t. maybe you want to charge a fee that is lower than a successful print. whichever one you decide on, have that covered since failed prints will certainly happen. (c) lastly, the pricing scheme should be easily handled by the library staff. the more library staff will be involved in the entire process of a library patron using the d printing service from the beginning to the end, the more important this becomes. if the pricing scheme is difficult for the staff to work with when they need charge for and process each d print job, the new d printing service will increase their workload significantly. which staff will be responsible for which step of the new service? what would be the exact tasks that the staff will need to do? for example, it may be that several staff at the circulation desk need to learn and handle new tasks involving the d printing service, such as labeling and putting away completed d models, processing the payment transaction, delivering the model, and marking the job status for the paid d print job as ‘completed’ in the d printing staff admin portal if there is such a system in place. below is the screenshot of the hs/hsl d printing staff admin portal developed in-house by the library it team. the hs/hsl d printing staff admin portal, university of maryland, baltimore examples – d printing service fees it’s always helpful to see how other libraries are doing when you need to determine your own pricing scheme. here are some examples that shows ten libraries’ d printing pricing scheme changed over the recent three years. unr delamare library https://guides.library.unr.edu/ dprinting – $ . per cubic inch of modeling material (raised to $ . starting july, ). – uprint – model material: $ . per cubic inch (= . gm= . lb) – uprint – support materials: $ . per cubic inch ncsu hunt library https://www.lib.ncsu.edu/do/ d-printing - uprint d printer: $ per cubic inch of material (abs), with a $ minimum – makerbot d printer: $ . per gram of material (pla), with a $ minimum – uprint – $ per cubic inch of material, $ minimum – f – $ . per gram of material, $ minimum southern illinois university library http://libguides.siue.edu/ d/request – originally $ per hour of printing time; reduced to $ as the demand grew. – lulzbot taz , luzbot mini – $ . per hour of printing time. byu library http://guides.lib.byu.edu/c.php?g= &p= – – makerbot replicator / ultimaker extended $ . per gram for standard ( . mm) resolution; $ . per gram for high ( . mm) resolution. university of michigan library the cube d printer checkout is no longer offered. – cost for professional d printing service; open access d printing is free. gvsu library https://www.gvsu.edu/techshowcase/makerspace- .htm – $ . per gram with a $ . minimum – free (ultimaker +, makerbot replicator , , x) university of tennessee, chattanooga library http://www.utc.edu/library/services/studio/ d-printing/index.php – – makerbot th, th – $ . per gram port washington public library http://www.pwpl.org/ d-printing/ d-printing-guidelines/ – makerbot – $ per hour of printing time miami university – $ . per gram of the finished print; – ? ucla library, dalhousie university library ( ) free types of d printing service fees from the examples above, you will notice that many d printing service fee schemes are based upon the weight of a d-print model. this is because these libraries are trying recover the cost of the d filament, and the amount of filament used is most accurately reflected in the weight of the resulting d-printed model. however, there are a few problems with the weight-based d printing pricing scheme. first, it is not readily calculable by a user before the print job, because to do so, the user will have to weigh a model that s/he won’t have until it is d-printed. also, once d-printed, the staff will have to weigh each model and calculate the cost. this is time-consuming and not very efficient. for this reason, my library considered an alternative pricing scheme based on the size of a d model. the idea was that we will have roughly three different sizes of an empty box – small, medium, and large – with three different prices assigned. whichever box into which a user’s d printed object fits will determine how much the user will pay for her/his d-printed model. this seemed like a great idea because it is easy to determine how much a model will cost to d-print to both users and the library staff in comparison to the weight-based pricing scheme. unfortunately, this size-based pricing scheme has a few significant flaws. a smaller model may use more filament than a larger model if it is denser (meaning the higher infill ratio). second, depending on the shape of a model, a model that fits in a large box may use much less filament than the one that fits in a small box. think about a large tree model with think branches. then compare that with a % filled compact baseball model that fits into a smaller box than the tree model does. thirdly, the resolution that determines a layer height may change the amount of filament used even if what is d-printed is a same model. different infill ratios – image from https://www.packtpub.com/sites/default/files/article-images/ os_ _ .png charging based upon the d printing time so we couldn’t go with the size-based pricing scheme. but we did not like the problems of the weight-based pricing scheme, either. as an alternative, we decided to go with the time-based pricing scheme because printing time is proportionate to how much filament is used, but it does not require that the staff weigh the model each time. a d-printing software gives an estimate of the printing time, and most d printers also display actual printing time for each model printed. first, we wanted to confirm the hypothesis that d printing time and the weight of the resulting model are proportionate to each other. i tested this by translating the weight-based cost to the time-based cost based upon the estimated printing time and the estimated weight of several cube models. here is the result i got using the makerbot replicator x. . gm/ min= . gm per min. . gm/ min= . gm per min. . gm/ min= . gm per min. . gm/ min= . gm per min. . gm/ min= . gm per min. . gm/ min= . gm per min. there is some variance, but the hypothesis holds up. based upon this, now let’s calculate the d printing cost by time. d plastic filament is $ for abs/pla and $ for the dissolvable per . kg (= . lb) from makerbot. that means that filament cost is $ . per gram for abs/pla and $ . per gram for the dissolvable. so, d filament cost is cents per gram on average. finalizing the service fee for d printing for an hour of d printing time, the amount of filament used would be . gm (= . x min). this gives us the filament cost of cents per hour of d printing (= . gm x cents). so, for the cost-recovery of filament only, i get roughly $ per hour of d printing time. earlier, i mentioned that filament is only one of the cost-incurring factors for the d printing service. it’s time to bring in those other factors, such as hardware wear/tear, staff time, electricity, maintenance, etc., plus “no-charge-for-failed-print-policy,” which was adopted at our library. those other factors will add an additional amount per d print job. and at my library, this came out to be about $ . (i will not go into details about how these have been determined because those will differ at each library.) so, the final service fee for our new d printing service was set to be $ up to hour of d printing + $ per additional hour of d printing. the $ is broken down to $ per hour of d printing that accounts for the filament cost and $ fixed cost for every d print job. to help our users to quickly get an idea of how much their d print job will cost, we have added a feature to the hs/hsl d print job submission form online. this feature automatically calculates and displays the final cost based upon the printing time estimate that a user enters. the hs/hsl d print job submission form, university of maryland, baltimore don’t be afraid of service fees i would like to emphasize that libraries should not be afraid to set service fees for new services. as long as they are easy to understand and the staff can explain the reasons behind those service fees, they should not be a deterrent to a library trying to introduce and provide a new innovative service. there is a clear benefit in running through all cost-incurring factors and communicating how the final pricing scheme was determined (including the verification of the hypothesis that d printing time and the weight of the resulting model are proportionate to each other) to all library staff who will be involved in the new d printing service. if any library user inquire about or challenges the service fee, the staff will be able to provide a reasonable explanation on the spot. i implemented this pricing scheme at the same time as the launch of my library’s makerspace (the hs/hsl innovation space at the university of maryland, baltimore – http://www.hshsl.umaryland.edu/services/ispace/) back in april . we have been providing d printing service and charging for it for more than two years. i am happy to report that during that entire duration, we have not received any complaint about the service fee. no library user expected our new d printing service to be free, and all comments that we received regarding the service fee were positive. many expressed a surprise at how cheap our d printing service is and thanked us for it. to summarize, libraries should be willing to explore and offer new innovating services even when they require charging service fees. and if you do so, make sure that the resulting pricing scheme for the new service is (a) sustainable and accountable, (b) readily graspable by users, and (c) easily handled by the library staff who will handle the payment transaction. good luck and happy d printing at your library! an example model with the d printing cost and the filament info displayed at the hs/hsl, university of maryland, baltimore posted in: library, management, technology, user experience. tagged: d printer · d printing · budget · charge · cost · funding · makerspace · service fees · sustainability · user experience · ux post-election statements and messages that reaffirm diversity nov th, by bohyun (library hat). comments are off for this post these are statements and messages sent out publicly or internally to re-affirm diversity, equity, and inclusion by libraries or higher ed institutions. i have collected these – some myself and many others through my fellow librarians. some of them were listed on my blog post, “finding the right words in post-election libraries and higher ed.” so there are some duplicates. if you think that your organization is already so much pro-diversity that there is no need to confirm or re-affirm diversity, you can’t be farther from the everyday reality that minorities experience. sometimes, saying isn’t much. but right now, saying it out loud can mean everything. if you support those who belong to minority groups but don’t say it out loud, how would they know it? right now, nothing is obvious other than there is a lot of hate and violence towards minorities. feel free to use these as your resource to craft a similar message. feel free to add if you have similar messages you have received or created in the comments section. if you haven’t heard from the organization you belong to, please ask for a message reaffirming and committing to diversity, equity, and inclusion. [update / / : statements from ala and lita have been released. i have added them below.] i will continue to add additional statements as i find them. if you see anything missing, please add below in the comment or send it via twitter @bohyunkim. thanks! from librarians but i know that there will be libraries librarian zoe fisher to other librarians care for one another director chris bourg to the mit libraries staff finding the right words in post-election libraries and higher ed (my e-mail sent to the it team at university of maryland, baltimore health sciences and human services library) with a a pin and a prayer dean k. g. schneider to the sonoma state university library staff from library associations lita ala pla arl dlf code lib [draft in github] from libraries james madison university libraries northwestern university libraries university of oregon libraries from higher ed institutions clarke university cuny duke universitymit loyola university, maryland northwestern university penn state university the catholic university of america university of california university of michigan university of nebraska, lincoln university of nevada, reno university of oregon university of rochester and rochester institute of technology university of florida addressing racially charged flyers on the campus marshall university president jerome a. gilbert’s statement regarding post-election tweet drexel university moving on as a community after the election dear members of the drexel community, it is heartening to me to see the drexel community come together over the last day to digest the news of the presidential election — and to do so in the spirit of support and caring that is so much a part of this university. we gathered family-style, meeting in small, informal groups in several places across campus, including the student center for inclusion and culture, our residence halls, and as colleagues over a cup of coffee. many student leaders, particularly from our multicultural organizations, joined the conversation. this is not a process that can be completed in just one day, of course. so i hope these conversations will continue as long as students, faculty and professional staff feel they are needed, and i want to assure you that our professional staff in student life, human resources, faculty affairs, as well as our colleagues in the lindy center for civic engagement, will be there for your support. without question, many members of our community were deeply concerned by the inflammatory rhetoric and hostility on the campaign trail that too often typified this bitter election season. as i wrote over the summer, the best response to an uncertain and at times deeply troubling world is to remain true to our values as an academic community. in the context of a presidential election, it is vital that we understand and respect that members of our broadly diverse campus can hold similarly diverse political views. the expression of these views is a fundamental element of the free exchange of ideas and intellectual inquiry that makes drexel such a vibrant institution. at the same time, drexel remains committed to ensuring a welcoming, inclusive, and respectful environment. those tenets are more important than ever. while we continue to follow changes on the national scene, it is the responsibility of each of us at drexel to join together to move ahead, unified in our commitment to open dialogue, civic engagement and inclusion. i am grateful for all you do to support drexel as a community that welcomes and encourages all of its members. lane community college good morning, colleagues, i am in our nation’s capital today. i’d rather be at home! like me, i am guessing that many of you were glued to the media last night to find out the results of the election. though we know who our next president will be, this transition still presents a lot of uncertainty. it is not clear what our future president’s higher education policies will be but we will be working with our national associations to understand and influence where we can. during times like this there is an opening for us to decide how we want to be with each other. moods will range from joy to sadness and disbelief. it seems trite but we do need to work together, now more than ever. as educators we have a unique responsibility to create safe learning environments where every student can learn and become empowered workers and informed citizens. this imperative seems even more important today. our college values of equity and inclusion have not changed and will not change and it is up to each of us to assure that we live out our values in every classroom and in each interaction. preparing ourselves and our students for contentious discussions sparked by the election is work we must do. it is quite likely that some of our faculty, staff and students may be feeling particularly vulnerable right now. can we reach out to each other and let each other know that we all belong at lane? during my inservice remarks i said that “we must robustly reject the calculated narrative of cynicism, division and despair. instead of letting this leak into our narratives, together we can bet on hope not fear, respect not hate, unity not division.” at lane we have the intellect (and proud of it) and wherewithal to do this. i am attaching a favorite reading from meg wheatley which is resonating with me today and will end with gary snyder’s words from to the children …..stay together learn the flowers go light. maryland institute college of art post-election community forums and support dear campus community, no matter how each of us voted yesterday, most of us likely agree that the presidential campaign has been polarizing on multiple fronts. as a result, today is a difficult day for our nation and our campus community. in our nation, regardless of how one has aligned with a candidate, half of our country feels empowered and the other half sad and perhaps angry. because such dynamics and feelings need to be addressed and supported on campus, this memo outlines immediate resources for our community of students, faculty and staff, and describes opportunities for fashioning dialogues and creative actions going forward. before sharing the specifics, let me say unambiguously that mica will always stand firm in our commitment to diversity and inclusion. this morning’s presidential task force on diversity, inclusion, equity, and globalization meeting discussed measures to ensure that, as a creative community, we will continue to build a culture where everyone is honored and supported for success. the impact of exhibitions such as the current baltimore rising show remains as critical as ever, and mica fosters an educational environment that is welcoming of all. in the short term our focus is to support one another. whether you are happy or distressed with the results, there has been sufficient feedback to indicate that our campus community is struggling with how to make sense of such a divisive election process. you may find the following services helpful and are encouraged to take advantage of them: for students: student counseling maintains walk-in hours from : – : pm every day. students are welcome to stop by the student counseling center ( mt. royal avenue) during that time or call - - and enter x once the recording begins to schedule an appointment. for faculty and staff: the employee assistance program (eap) is available to provide free, confidential support hours a day. the eap can be reached by calling - - - or visiting healthadvocate.com/members and providing the username “maryland institute college of art”. for all mica community members: mica’s chaplain, the rev, maintains standing hours every monday and can be reached in the reflection room (meyerhoff house) or by calling the office of diversity and intercultural development at - - . there are three events this week that can provide a shared space for dialogue; all are welcome: the “after the baltimore uprising: still waiting for change” community forum attached to the baltimore rising exhibition takes place tonight from : pm to : pm in the lazarus center. an open space for all mica community members will be hosted by the black student union tonight at : pm in the meyerhoff house underground. in partnership with our student nami group, mica will host a “messages of hope” event for the entire mica community that will allow for shared space and reflection. this event will be on friday, november th, and will begin at : pm in cohen plaza. in various upcoming meetings we look forward to exploring with campus members other appropriate activities that can be created to facilitate expressions and dialogues. a separate communication is coming from provost david bogen to the faculty regarding classroom conversations with students regarding the election. northwestern university women’s center dear northwestern students, faculty, staff and community members: the women’s center is open today. our staff members are all here and available to talk, to provide resources and tools, or to help however you might need it. most importantly, the space itself is available for whatever you need, whether that is to gather as a group, to sit alone somewhere comfortable and quiet, or to talk to someone who will listen. we are still here, and we are here for all people as an intentionally intersectional space. you are welcome to drop by physically, make a call to our office, or send an email. know that this space is open and available to you. portland community college to the pcc staff as someone who spent the last several years in washington d.c. working to advance community colleges, i feel a special poignancy today hearing so many students, colleagues, and friends wonder and worry about the future—and about their futures. we must acknowledge that this political season has highlighted deep divisions in our society. today i spent time with cabinet speaking about how we can assert our shared values and take positive action as a pcc community to deepen our commitment to equity, inclusion and civic engagement. pcc will always welcome students and colleagues who bring a rich array of perspectives and experiences. that diversity is among our greatest strengths. today it is imperative that we stand by faculty, staff and students who may be experiencing fear or uncertainty—affirming with our words and deeds that pcc is about equitable student success and educational opportunity for all. never has this mission been more powerful or more essential. i have only been here a few months, but have already learned that pcc is a remarkable and caring community. much is happening right now in real time, and i appreciate the efforts of all. for my part, i promise to communicate often as we continue to plan for our shared future. p.s. today and in the days ahead, we will be holding space for people to be together in community. here are a few of the opportunities identified so far. portland community college to students dear students: as someone who spent the last several years working in washington d.c., i feel a special poignancy this week hearing many of you express worry and uncertainty about the future. there is little doubt that this political season has highlighted some deep divisions in our society. both political candidates have acknowledged as much. at the same time, people representing the full and diverse spectrum of our country come to our nation’s community colleges in hopes of a better life. pcc is such a place – where every year thousands of students find their path and pursue their dreams. all should find opportunity here, and all should feel safe and welcome. the rich diversity of pcc offers an amazing opportunity for dialogue across difference, and for developing skills that are the foundation of our democratic society. let this moment renew your passion for making a better life for yourself, your community and your country and for becoming the kind of leader you want to follow. rutgers university aaup-aft (american association of university professors – american federation of teachers) resisting donald trump we are shocked and horrified that donald trump, who ran on a racist, xenophobic, misogynist platform, is now the president of the us. in response to this new political landscape, the administrative heads of several universities have issued statements embracing their diverse student, faculty, and staff bodies and offering support and protection. (see statements from the university of california and the california state university). president barchi has yet to address the danger to the rutgers community and its core mission. this afternoon, our faculty union and the rutgers one coalition held an emergency meeting of students, faculty, and community activists in new brunswick. we discussed means of responding to the attacks that people may experience in the near future. most immediately, we approved the following statement by acclamation at the -strong meeting: “rutgers one, a coalition of faculty, staff, students and community members, calls upon the rutgers administration to join us in condemning all acts of bigotry on this campus and refuse to tolerate any attacks on immigrants, women, arabs, muslims, people of color, lgbtq people and all others in our diverse community. we demand that president barchi and his administration provide sanctuary, support, and protection to those who are already facing attacks on our campuses. we need concrete action that can ensure a safe environment for all. further, we commit ourselves to take action against all attempts by the trump administration to target any of our students, staff or faculty. we are united in resistance to bigotry of every kind and welcome all to join us in solidarity.” we also resolved to take the following steps: we will be holding weekly friday meetings at pm in our union office in new brunswick to bring together students, faculty and staff to organize against the trump agenda. we hope to expand these to camden and newark as well. (if you are willing to help organize this, please email back.) we will be creating a list serve to coordinate our work. if you want to join this list, please reply to this email. we are making posters and stickers which declare sanctuaries from racism, xenophobia, sexism, bigotry, religious intolerance, and attacks on unions. once these materials are ready we will write to you so that you may post them on windows, office doors, cars etc. in the meantime, we urge you to talk to your students and colleagues of color as well as women and offer them your support and solidarity. as you may recall, the executive committee issued a denunciation of donald trump on october , . now our slogan, one from the labor movement, is “don’t mourn. organize!” that is where we are now – all the more poignantly because of donald trump’s appeal to workers. let us organize, and let us also expand our calling of education. in your classrooms, your communities, and your families, find the words and sentiments that will redeem all of us from tuesday’s disgrace. university of chicago message from president and provost early in the fall quarter, we sent a message welcoming each of you to the new academic year and affirming our strong commitment to two foundational values of the university – fostering an environment of free expression and open discourse; and ensuring that diversity and inclusion are essential features of the fabric of our campus community and our interactions beyond campus. recent national events have generated waves of disturbing, exclusionary and sometimes threatening behavior around the country, particularly concerning gender and minority status. as a result, many individuals are asking whether the nation and its institutions are entering a period in which supporting the values of diversity and inclusion, as well as free expression and open discourse, will be increasingly challenging. as the president and provost of the university of chicago, we are writing to reaffirm in the strongest possible terms our unwavering commitment to these values, and to the importance of the university as a community acting on these values every day. fulfilling our highest aspirations with respect to these values and their mutual reinforcement will always demand ongoing attention and work on the part of all of us. the current national environment underscores the importance of this work. it means that we need to manifest these values more rather than less, demand more of ourselves as a community, and together be forthright and bold in demonstrating what our community aspires to be. we ask all of you for your help and commitment to the values of diversity and inclusion, free expression, and open discourse and what they mean for each of us working, learning, and living in this university community every day. university of illinois, chicago dear students, faculty, and staff, the events of the past week have come with mixed emotions for many of you. we want you to know that uic remains steadfast in its commitment to creating and sustaining a community that recognizes and values the inherent worth and dignity of every person, while fostering an environment of mutual respect among all members. today, we reaffirm the university’s commitment to access, equity, inclusion and nondiscrimination. critical to this commitment is the work of several offices on campus that provide resources to help you be safe and successful. if you have questions, need someone to talk to, or a place to express yourself, you should consider contacting these offices: office for access and equity (oae). oae is responsible for assuring campus compliance in matters of equal opportunity, affirmative action, and nondiscrimination in the academic and work environment. oae also offers dispute resolution services (drs) to assist with conflict in the workplace not involving unlawful discrimination matters. uic counseling center. the uic counseling center is a primary resource providing comprehensive mental health services that foster personal, interpersonal, academic, and professional thriving for uic students. student legal services. uic’s student legal services (sls) is a full-service law office dedicated to providing legal solutions for currently enrolled students. office of diversity. the office of diversity leads strategic efforts to advance access, equity, and inclusion as fundamental principles underpinning all aspects of university life. it initiates programs that promote an inclusive university climate, partner with campus units to formulate systems of accountability, and develop links with the local community and alumni groups. centers for cultural understanding and social change. the centers for cultural understanding and social change (ccusc) are a collaborative group of seven centers with distinct histories, missions, and locations that promote the well-being of and cultural awareness about underrepresented and underserved groups at uic. uic dialogue initiative. the uic dialogue initiative seeks to build an inclusive campus community where students, faculty, and staff feel welcomed in their identities, valued for their contributions, and feel their identities can be openly expressed. through whatever changes await us, as a learning community we have a special obligation to ensure that our conversations and dialogues over the next weeks and months respect our varied backgrounds and beliefs. university of maryland, baltimore to the umb community: last week, we elected a new president for our country. i think most will agree that the campaign season was long and divisive, and has left many feeling separated from their fellow citizens. in the days since the election, i’ve heard from the leaders of umb and of the university of maryland medical center and of the many programs we operate that serve our neighbors across the city and state. these leaders have relayed stories of students, faculty, staff, families, and children who feel anxious and unsettled, who feel threatened and fearful. it should be unnecessary to reaffirm umb’s commitment to diversity, inclusion, and respect — these values are irrevocable — but when i hear that members of our family are afraid, i must reiterate that the university will not tolerate incivility of any kind, and that the differences we celebrate as a diverse community include not just differences of race, religion, nationality, gender, and sexual identity, but also of experience, opinion, and political affiliation and ideology. if you suffer any harassment, please contact your supervisor or your student affairs dean. in the months ahead, we will come together as a university community to talk about how the incoming administration might influence the issues we care about most: health care access and delivery; education; innovation; social justice and fair treatment for all. we will talk about the opportunities that lay ahead to shape compassionate policy and to join a national dialogue on providing humane care and services that uplift everyone in america. for anyone who despairs, we will talk about building hope. should you want to share how you’re feeling post-election, counselors are available. please contact the student counseling center or the employee assistance program to schedule an appointment. i look forward to continuing this conversation about how we affirm our fundamental mission to improve the human condition and serve the public good. like the values we uphold, this mission endures — irrespective of the person or party in political power. it is our binding promise to the leaders of this state and, even more importantly, to the citizens we serve together. university of west georgia dear colleagues, as we head into the weekend concluding a week, really several weeks, of national and local events, i am reminded of the incredible opportunity of reflection and discourse we have as a nation and as an institution of higher learning. this morning, we held on campus a moving ceremony honoring our veterans–those who have served and who have given the ultimate sacrifice to uphold and protect our freedoms. it is those freedoms that provide the opportunity to elect a president and those freedoms that provide an environment of civil discourse and opinion. clearly, the discourse of this election cycle has tested the boundaries. this is an emotional time for many of our faculty, staff, and students. i ask that as a campus community we hold true to the intended values of our nation and those who sacrificed to protect those values and the core values of our institution–caring, collaboration, inclusiveness, and wisdom. we must acknowledge and allow the civil discourse and opinion of all within a safe environment. that is what should set us apart. it is part of our dna in higher education to respect and encourage variance and diversity of belief, thought, and culture. i call on your professionalism during these times and so appreciate your passion and care for each other and our students. virginia commonwealth university to staff election message dear vcu and vcu health communities, yesterday, we elected new leaders for our city, commonwealth and nation. i am grateful to those of you who made your voice heard during the electoral process, including many of our students who voted for the first time. whether or not your preferred candidate won, you were a part of history and a part of the process that moves our democracy forward. thank you. i hope you will always continue to make your voice heard, both as voters and as well-educated leaders in our society. as with any election, some members of our community are enthusiastic about the winners, others are not. for many, this election cycle was notably emotional and difficult. now is the time, then, to demonstrate the values that make virginia commonwealth university such a remarkable place. we reaffirm our commitment to working together across boundaries of discipline or scholarship, as members of one intellectual community, to achieve what’s difficult. we reaffirm our commitment to inclusion, to ensuring that every person who comes to vcu is respected and emboldened to succeed. we reaffirm that we will always be a place of the highest integrity, accountability, and we will offer an unyielding commitment to serving those who need us. history changes with every election. what does not change are the commitments we share as one community that is relentlessly focused on advancing the human experience for all people. you continue to inspire me. and i know you will continue to be a bright light for richmond, virginia, our nation and our world. virginia commonwealth university school of education to students election message dear students, on tuesday we elected new leaders for our city, our commonwealth and our nation. although leadership will be changing, i echo dr. rao’s message below in that our mission outlined by the quest for distinction to support student success, advance knowledge and strengthen our communities remains steadfast. at the vcu school of education, we work to create safe spaces where innovation, inclusion and collaboration can thrive. we actively work across boundaries and disciplines to address the complex challenges facing our communities, schools and families. the election of new leaders provides new opportunities for our students, faculty and staff to build bridges that help us reach our goal of making an impact in urban and high need environments. i encourage you to engage in positive dialogues with one another as the city, commonwealth and nation adjust to the change in leadership, vision and strategy. virginia commonwealth university division of student affairs dear students, we are writing to you, collectively, as leaders in the division of student affairs. we acknowledge that this election season was stressful for many individuals in our vcu community, culminating with the election of the next president. some members of our campus community have felt disrespected, attacked and further marginalized by political rhetoric during the political process. we want to affirm support of all of our students while also recognizing the unique experiences and concerns of individuals. we want all students to know that we are here to support you, encourage you and contribute to your success. we now live in a space of uncertainty as we transition leadership in our nation. often, with this uncertainty comes a host of thoughts and feelings. we hope that you will take advantage of some of the following services and programs we offer through our division to support your well-being, including: office of multicultural student affairs, self-care space, university counseling services , the wellness resource center, trans lives matter panel and survivor solidarity support, recreational sports, restorative yoga and mind & body classes. we encourage students to express their concerns and engage in conversations that further the core values articulated in quest, the vcu strategic plan. we continue to have an opportunity to make individual and collective choices about how we work to bridge differences in a manner that builds up our community. our staff will have a table each day next week on the vcu compass from noon to : p.m. to receive your concerns, suggestions and just listen. please stop by to meet us. we want you to know you have our full support. other organizations aclu joint statement from california legislative leaders on result of presidential election posted in: diversity, librarianship, library, management. tagged: college · communication · diversity · election · equity · higher ed · inclusion · library · university finding the right words in post-election libraries and higher ed nov th, by bohyun (library hat). comments are off for this post ** this post was originally published in acrl techconnect on nov. , .*** this year’s election result has presented a huge challenge to all of us who work in higher education and libraries. usually, libraries, universities, and colleges do not comment on presidential election result and we refrain from talking about politics at work. but these are not usual times that we are living in. a black female student was shoved off the sidewalk and called the ‘n’ word at baylor university. the ku klux klan is openly holding a rally. west virginia officials publicly made a racist comment about the first lady. steve bannon’s prospective appointment as the chief strategist and senior counsel to the new president is being praised by white nationalist leaders and fiercely opposed by civil rights groups at the same time. bannon is someone who calls for an ethno-state, openly calls martin luther king a fraud, and laments white dispossession and the deconstruction of occidental civilization. there are people drawing a swastika at a park. the ‘whites only’ and ‘colored’ signs were put up over water fountains in a florida school. a muslim student was threatened with a lighter. asian-american women are being assaulted. hostile acts targeting minority students are taking place on college campuses. libraries and educational institutions exist because we value knowledge and science. knowledge and science do not discriminate. they grow across all different races, ethnicities, religions, nationalities, sexual identities, and disabilities. libraries and educational institutions exist to enable and empower people to freely explore, investigate, and harness different ideas and thoughts. they support, serve, and belong to ‘all’ who seek knowledge. no matter how naive it may sound, they are essential to the betterment of human lives, and they do so by creating strength from all our differences, not likeness. this is why diversity, equity, inclusion are non-negotiable and irrevocable values in libraries and educational institutions. how do we reconcile these values with the president-elect who openly dismissed and expressed hostility towards them? his campaign made remarks and promises that can be interpreted as nothing but the most blatant expressions of racism, sexism, intolerance, bigotry, harassment, and violence. what will we do to address the concerns of our students, staff, and faculty about their physical safety on campus due to their differences in race, ethnicity, religion, nationality, gender, and sexual identity? how do we assure them that we will continue to uphold these values and support everyone regardless of what they look like, how they identify their gender, what their faiths are, what disabilities they may have, who they love, where they come from, what languages they speak, or where they live? how? we say it. explicitly. clearly. and repeatedly. if you think that your organization is already very much pro-diversity that there is no need to confirm or reaffirm diversity, you can’t be farther from the everyday life minorities experience. sometimes, saying isn’t much. but right now, saying it out loud can mean everything. if you support those who belong to minority groups but don’t say it out loud, how would they know it? right now, nothing is obvious other than there is a lot of hate and violence towards minorities. the entire week after the election, i agonized about what to say to my small team of it people whom i supervise at work. as a manager, i felt that it was my responsibility to address the anxiety and uncertainty that some of my staff – particularly those in minority groups – would be experiencing due to the election result. i also needed to ensure that whatever dialogue takes place regarding the differences of opinions between those who were pleased and those who were distressed with the election result, those dialogues remain civil and respectful. crafting an appropriate message was much more challenging than i anticipated. i felt very strongly about the need to re-affirm the unwavering support and commitment to diversity, equity, and inclusion particularly in relation to libraries and higher education, no matter how obvious it may seem. i also felt the need to establish (within the bounds of my limited authority) that we will continue to respect, value, and celebrate diversity in interacting with library users as well as other library and university staff members. employees are held to the standard expectations of their institutions, such as diversity, equity, inclusion, tolerance, civil dialogue, and no harassment or violence towards minorities, even if their private opinions conflict with them. at the same time, i wanted to strike a measured tone and neither scare nor upset anyone, whichever side they were on in the election. as a manager, i have to acknowledge that everyone is entitled to their private opinions as long as they do not harm others. i suspect that many of us – either a manager or not – want to say something similar about the election result. not so much about who was and should have been as about what we are going to do now in the face of these public incidences of anger, hatred, harassment, violence, and bigotry directed at minority groups, which are coming out at an alarming pace because it affects all of us, not just minorities. finding the right words, however, is difficult. you have to carefully consider your role, audience, and the message you want to convey. the official public statement from a university president is going to take a tone vastly different from an informal private message a supervisor sends out to a few members of his or her team. a library director’s message to library patrons assuring the continued service for all groups of users with no discrimination will likely to be quite different from the one she sends to her library staff to assuage their anxiety and fear. for such difficulty not to delay and stop us from what we have to and want to say to everyone we work with and care for, i am sharing the short message that i sent out to my team last friday, days after the election. (n.b. ‘cats’ stands for ‘computing and technology services’ and umb refers to ‘university of maryland, baltimore.’) this is a customized message to address my own team. i am sharing this as a potential template for you to craft your own message. i would like to see more messages that reaffirm diversity, equity, and inclusion as non-negotiable values, explicitly state that we will not step backwards, and make a commitment to continued unwavering support for them. dear cats, this year’s close and divisive election left a certain level of anxiety and uncertainty in many of us. i am sure that we will hear from president perman and the university leadership soon. in the meantime, i want to remind you of something i believe to be very important. we are all here – just as we have been all along – to provide the most excellent service to our users regardless of what they look like, what their faiths are, where they come from, what languages they speak, where they live, and who they love. a library is a powerful place where people transform themselves through learning, critical thinking, and reflection. a library’s doors have been kept open to anyone who wants to freely explore the world of ideas and pursue knowledge. libraries are here to empower people to create a better future. a library is a place for mutual education through respectful and open-minded dialogues. and, we, the library staff and faculty, make that happen. we get to make sure that people’s ethnicity, race, gender, disability, socio-economic backgrounds, political views, or religious beliefs do not become an obstacle to that pursuit. we have a truly awesome responsibility. and i don’t have to tell you how vital our role is as a cats member in our library’s fulfilling that responsibility. whichever side we stood on in this election, let’s not forget to treat each other with respect and dignity. let’s use this as an opportunity to renew our commitment to diversity, one of the umb’s core values. inclusive excellence is one of the themes of the umb - strategic plan. each and every one of us has a contribution to make because we are stronger for our differences. we have much work ahead of us! i am out today, but expect lots of donuts monday. have a great weekend, bohyun monday, i brought in donuts of many different kinds and told everyone they were ‘diversity donuts.’ try it. i believe it was successful in easing some stress and tension that was palpable in my team after the election. photo from flickr: https://www.flickr.com/photos/vnysia/ before crafting your own message, i recommend re-reading your institution’s core values, mission and vision statements, and the most recent strategic plan. most universities, colleges, and libraries include diversity, equity, inclusion, or something equivalent to these somewhere. also review all public statements or internal messages that came from your institution that reaffirms diversity, equity, and inclusion. you can easily incorporate those into your own message. make sure to clearly state your (and your institution’s) continued commitment to and unwavering support for diversity and inclusion and explicitly oppose bigotry, intolerance, harassment, and acts of violence. encourage civil discourse and mutual respect. it is very important to reaffirm the values of diversity, equity, and inclusion ‘before’ listing any resources and help that employees or students may seek in case of harassment or assault. without the assurance from the institution that it indeed upholds those values and will firmly stand by them, those resources and help mean little. below i have also listed messages, notes, and statements sent out by library directors, managers, librarians, and university presidents that reaffirm the full support for and commitment to diversity, equity, and inclusion. i hope to see more of these come out. if you have already received or sent out such a message, i invite you to share in the comments. if you have not, i suggest doing so as soon as possible. send out a message if you are in a position where doing so is appropriate. don’t forget to ask for a message addressing those values if you have not received any from your organization. director chris bourg to the mit libraries staff https://chrisbourg.wordpress.com/ / / /care-for-one-another/ dean k. g. schneider to the sonoma state university library staff http://freerangelibrarian.com/ / / /pin-and-a-prayer/ librarian zoe fisher to other librarians https://quickaskzoe.com/ / / /but-i-know-that-there-will-be-libraries/ university of california statement on presidential election results https://www.universityofcalifornia.edu/press-room/university-california-statement-election university of nevada, reno http://www.unr.edu/president/communications/ - - -election university of michigan http://president.umich.edu/news-communications/letters-to-the-community/ -election-message/ university of rochester and rochester institute of technology http://wxxinews.org/post/ur-presidents-post-election-letter-strikes-sour-note-some duke university https://today.duke.edu/ / /statement-president-brodhead-following- -election clarke university http://www.clarke.edu/page.aspx?id= mit https://news.mit.edu/ /letter-mit-community-new-administration-washington- northwestern university https://news.northwestern.edu/stories/ / /president-schapiro-on-the-election-and-the-university/ “post-election statements and messages that reaffirm diversity” (a list of more post-election statements and messages that reaffirm diversity) posted in: diversity, librarianship, library, management. tagged: diversity · election · equity · inclusion · message · post-election · statement · template · tolerance say it out loud – diversity, equity, and inclusion nov th, by bohyun (library hat). comments are off for this post i usually and mostly talk about technology. but technology is so far away from my thought right now. i don’t feel that i can afford to worry about internet surveillance or how to protect privacy at this moment. not that they are unimportant. such a worry is real and deserves our attention and investigation. but at a time like this when there are so many reports of public incidences of hatred, bigotry, harassment, and violence reported on university and college campuses, on streets, and in many neighborhoods coming in at an alarming pace, i don’t find myself reflecting on how we can use technology to deal with this problem. for the problem is so much bigger. there are people drawing a swastika at a park. the ‘whites only’ and ‘colored’ signs were put up over water fountains in a florida school. a muslim student was threatened with a lighter. asian-american women are being assaulted. hostile acts targeting minority students are taking place on college campuses. a black female student was shoved off the sidewalk and called the ‘n’ word at baylor university. newt gingrich called for a house committee for un-american activities. the ku klux klan is openly holding a rally. the list goes on and on. photo from http://www.wftv.com/news/local/investigation-underway-after- -racist-signs-posted-above-water-fountains-at-first-coast-high-school/ we are justified to be freaking out. i suspect this is a deal breaker to not just democrats, not just clinton supporters, but a whole lot more people. not everyone who voted for donald trump endorse the position that women, people of color, muslims, lgbt, and all other minority groups deserve and should be deprived of the basic human right to be not publicly threatened, harassed, and assaulted, i hope. i am sure that many who voted for donald trump do support diversity, equity, and inclusion as important and non-negotiable values. i believe that many who voted for donald trump do not want a society where some of their family, friends, colleagues, and neighbors have to live in constant fear for their physical safety at minimum. there are very many white people who absolutely condemn bigotry, threat, hatred, discrimination, harassment, and violence directed at minorities and give their unwavering support to diversity, equity, and inclusion. the problem is that i don’t hear it said loudly enough, clearly enough, publicly enough. i realized that we – myself included – do not say this enough. one of my fellow librarians, steve, wrote this on his facebook wall after the election. i am a year old white guy. … i go out into the world today and i’m trying to hold a look on my face that says i don’t hate you black people, hispanic people, gay people, muslim people. i mean you no harm. i don’t want to deport you or imprison you. you are my brothers and sisters. i want for you all of the benefits, the rights, the joys (such as they are) that are afforded to everybody else in our society. i don’t think this look on my face is effective. why should they trust me? you can never appear to be doing the right thing. it requires doing the right thing. of course, steve doesn’t want to harm me because i am not white, i know. i am % positive that he wouldn’t assault me because i am female. but by stating this publicly (i mean as far as his fb friends can see the post), he made a difference to me. steve is not republican. but i would feel so much better if people i know tell me the same thing whether they are democrat or republican. and i think it will make a huge difference to others when we all say this together. sometimes, saying isn’t much. but right now, saying it aloud can mean everything. if you support those who belong to minority groups but don’t say it out loud, how would they know it? because right now, nothing is obvious other than there is a lot of hate and violence towards minorities. at this point, which candidate you voted for doesn’t matter. what matters is whether you will condone open hatred and violence towards minorities and women, thereby making it acceptable in our society. there is a lot at stake here, and this goes way beyond party politics. publicly confirming our continued support for and unwavering commitment to diversity is a big deal. people who are being insulted, threatened, harassed, and assaulted need to hear it. and when we say this together loudly enough, clearly enough, explicitly enough, it will deafen the voice of hatred, bigotry, and intolerance and chase it away to the margins of our society again. so i think i am going to say this whenever i have a chance whether formally or informally whether it is in a written form or in a conversation. if you are a librarian, you should say this to your library users. if you are a teacher, you should say this to your students. if you run a business, you need to say this to your employees and customers. if you manage a team at work, tell your team. say this out loud to your coworkers, friends, family, neighbors, and everyone you interact with. “i support all minorities and stand for diversity, equity, and inclusion.” “i object to and will not condone the acts of harassment, violence, hatred, and threats directed at minorities.” “i will not discriminate anyone based upon their ethnicity, race, sexual orientation, disability, political views, socio-economic backgrounds, or religious beliefs.” we cannot allow diversity, equity, and inclusion to become minority opinions. and it is up to us to keep it mainstream and to make it prevail. say it aloud and act on it. in times like this, many of us look to institutions that we belong to, the organizations we work for, professionally participate in, or personally support. we expect them to reconfirm the very basic values of diversity, equity, and inclusion. since i work for a university, i have been looking up and reading statements from higher education institutions. so far, not a great number of universities have made public statements confirming their continued support for diversity. i am sure more are on the way. but i expected more of them would come out more promptly. this is unfortunate because many of them openly expressed their support for diversity and even include diversity in their values, mission, and goals. if your organization hasn’t already confirmed their support for these values and expressed their commitment to provide safety for all minorities, ask for it. you may even be in a position to actually craft and issue one. for those in need of right words to express your intention clearly, here are some good examples below. “the university of california is proud of being a diverse and welcoming place for students, faculty, and staff with a wide range of backgrounds, experiences and perspectives. diversity is central to our mission. we remain absolutely committed to supporting all members of our community and adhering to uc’s principles against intolerance. as the principles make clear, the university ‘strives to foster an environment in which all are included’ and ‘all are given an equal opportunity to learn and explore.’ the university of california will continue to pursue and protect these principles now and in the future, and urges our students, faculty, staff, and all others associated with the university to do so as well.” – university of california “our responsibility is to remain committed to education, discovery and intellectual honesty – and to diversity, equity and inclusion. we are at our best when we come together to engage respectfully across our ideological differences; to support all who feel marginalized, threatened or unwelcome; and to pursue knowledge and understanding, as we always have, as the students, faculty and staff of the university of michigan.” – university of michigan “northwestern is committed to being a welcoming and inclusive community for all, regardless of their beliefs, and i assure you that will not change.” – northwestern university “as a catholic university, clarke will not step away from its many efforts to heighten our awareness of the individuals and groups who are exclude and marginalized in so many ways and to take action for their protection and inclusion. today, i call on us as a community to step up our efforts to promote understanding and inclusion and to reach out to those among us who are feeling further disenfranchised, fearful and confused as a result of the election.” – clarke university “as president, i need to represent all of rit, and i therefore do not express preferences for political candidates. i do feel it important, however, to represent and reinforce rit’s shared commitment to the value of inclusive diversity. i have heard from many in our community that the result of the recent election has raised concerns from those in our minority populations, those who come from immigrant families, those from countries outside of the u.s., those in our lgbtqia+ community, those who practice islam, and even those in our female population about whether they should be concerned for their safety and well-being as a result of the horrific discourse that accompanied the presidential election process and some of the specific views and proposals presented. at rit, we have treasured the diverse contributions of members of these groups to our campus community, and i want to reassure all that one of rit’s highest priorities is to demonstrate the extraordinary value of inclusive diversity and that we will continue to respect, appreciate, and benefit from the contributions of all. anyone who feels unsafe here should make their feelings known to me and to others in a position to address their concerns. concerned members of our community can also take advantage of opportunities to engage in open discourse about the election in the mosaic center and at tomorrow’s grey matter discussion.” – rochester institute of technology please go ahead and say these out loud to people around you if you mean them. no matter how obvious and cheesy they sound, i assure you, they are not obvious and cheesy to those who are facing open threats, harassment, and violence. let’s boost the signal; let’s make it loud; let’s make it overwhelming. “i support all minorities and stand for diversity, equity, and inclusion.” “i object to and will not condone the acts of harassment, violence, hatred, and threats directed at minorities.” “i will not discriminate anyone based upon their ethnicity, race, sexual orientation, disability, political views, socio-economic backgrounds, or religious beliefs.” posted in: diversity. tagged: · election · hate crime · racism cybersecurity, usability, online privacy, and digital surveillance may th, by bohyun (library hat). comments are off for this post ** this post was originally published in acrl techconnect on may. , .*** cybersecurity is an interesting and important topic, one closely connected to those of online privacy and digital surveillance. many of us know that it is difficult to keep things private on the internet. the internet was invented to share things with others quickly, and it excels at that job. businesses that process transactions with customers and store the information online are responsible for keeping that information private. no one wants social security numbers, credit card information, medical history, or personal e-mails shared with the world. we expect and trust banks, online stores, and our doctor’s offices to keep our information safe and secure. however, keeping private information safe and secure is a challenging task. we have all heard of security breaches at j.p morgan, target, sony, anthem blue cross and blue shield, the office of personnel management of the u.s. federal government, university of maryland at college park, and indiana university. sometimes, a data breach takes place when an institution fails to patch a hole in its network systems. sometimes, people fall for a phishing scam, or a virus in a user’s computer infects the target system. other times, online companies compile customer data into personal profiles. the profiles are then sold to data brokers and on into the hands of malicious hackers and criminals. image from flickr – https://www.flickr.com/photos/topgold/ cybersecurity vs. usability to prevent such a data breach, institutional it staff are trained to protect their systems against vulnerabilities and intrusion attempts. employees and end users are educated to be careful about dealing with institutional or customers’ data. there are systematic measures that organizations can implement such as two-factor authentication, stringent password requirements, and locking accounts after a certain number of failed login attempts. while these measures strengthen an institution’s defense against cyberattacks, they may negatively affect the usability of the system, lowering users’ productivity. as a simple example, security measures like a captcha can cause an accessibility issue for people with disabilities. or imagine that a university it office concerned about the data security of cloud services starts requiring all faculty, students, and staff to only use cloud services that are soc type ii certified as an another example. soc stands for “service organization controls.” it consists of a series of standards that measure how well a given service organization keeps its information secure. for a business to be soc certified, it must demonstrate that it has sufficient policies and strategies that will satisfactorily protect its clients’ data in five areas known as “trust services principles.” those include the security of the service provider’s system, the processing integrity of this system, the availability of the system, the privacy of personal information that the service provider collects, retains, uses, discloses, and disposes of for its clients, and the confidentiality of the information that the service provider’s system processes or maintains for the clients. the soc type ii certification means that the business had maintained relevant security policies and procedures over a period of at least six months, and therefore it is a good indicator that the business will keep the clients’ sensitive data secure. the dropbox for business is soc certified, but it costs money. the free version is not as secure, but many faculty, students, and staff in academia use it frequently for collaboration. if a university it office simply bans people from using the free version of dropbox without offering an alternative that is as easy to use as dropbox, people will undoubtedly suffer. some of you may know that the usps website does not provide a way to reset the password for users who forgot their usernames. they are instead asked to create a new account. if they remember the account username but enter the wrong answers to the two security questions more than twice, the system also automatically locks their accounts for a certain period of time. again, users have to create a new account. clearly, the system that does not allow the password reset for those forgetful users is more secure than the one that does. however, in reality, this security measure creates a huge usability issue because average users do forget their passwords and the answers to the security questions that they set up themselves. it’s not hard to guess how frustrated people will be when they realize that they entered a wrong mailing address for mail forwarding and are now unable to get back into the system to correct because they cannot remember their passwords nor the answers to their security questions. to give an example related to libraries, a library may decide to block all international traffic to their licensed e-resources to prevent foreign hackers who have gotten hold of the username and password of a legitimate user from accessing those e-resources. this would certainly help libraries to avoid a potential breach of licensing terms in advance and spare them from having to shut down compromised user accounts one by one whenever those are found. however, this would make it impossible for legitimate users traveling outside of the country to access those e-resources as well, which many users would find it unacceptable. furthermore, malicious hackers would probably just use a proxy to make their ip address appear to be located in the u.s. anyway. what would users do if their organization requires them to reset passwords on a weekly basis for their work computers and several or more systems that they also use constantly for work? while this may strengthen the security of those systems, it’s easy to see that it will be a nightmare having to reset all those passwords every week and keeping track of them not to forget or mix them up. most likely, they will start using less complicated passwords or even begin to adopt just one password for all different services. some may even stick to the same password every time the system requires them to reset it unless the system automatically detects the previous password and prevents the users from continuing to use the same one. ill-thought-out cybersecurity measures can easily backfire. security is important, but users also want to be able to do their job without being bogged down by unwieldy cybersecurity measures. the more user-friendly and the simpler the cybersecurity guidelines are to follow, the more users will observe them, thereby making a network more secure. users who face cumbersome and complicated security measures may ignore or try to bypass them, increasing security risks. image from flickr – https://www.flickr.com/photos/topgold/ cybersecurity vs. privacy usability and productivity may be a small issue, however, compared to the risk of mass surveillance resulting from aggressive security measures. in , the guardian reported that the communication records of millions of people were being collected by the national security agency (nsa) in bulk, regardless of suspicion of wrongdoing. a secret court order prohibited verizon from disclosing the nsa’s information request. after a cyberattack against the university of california at los angeles, the university of california system installed a device that is capable of capturing, analyzing, and storing all network traffic to and from the campus for over days. this security monitoring was implemented secretly without consulting or notifying the faculty and those who would be subject to the monitoring. the san francisco chronicle reported the it staff who installed the system were given strict instructions not to reveal it was taking place. selected committee members on the campus were told to keep this information to themselves. the invasion of privacy and the lack of transparency in these network monitoring programs has caused great controversy. such wide and indiscriminate monitoring programs must have a very good justification and offer clear answers to vital questions such as what exactly will be collected, who will have access to the collected information, when and how the information will be used, what controls will be put in place to prevent the information from being used for unrelated purposes, and how the information will be disposed of. we have recently seen another case in which security concerns conflicted with people’s right to privacy. in february , the fbi requested apple to create a backdoor application that will bypass the current security measure in place in its ios. this was because the fbi wanted to unlock an iphone c recovered from one of the shooters in san bernadino shooting incident. apple ios secures users’ devices by permanently erasing all data when a wrong password is entered more than ten times if people choose to activate this option in the ios setting. the fbi’s request was met with strong opposition from apple and others. such a backdoor application can easily be exploited for illegal purposes by black hat hackers, for unjustified privacy infringement by other capable parties, and even for dictatorship by governments. apple refused to comply with the request, and the court hearing was to take place in march . the fbi, however, withdrew the request saying that it found a way to hack into the phone in question without apple’s help. now, apple has to figure out what the vulnerability in their ios if it wants its encryption mechanism to be foolproof. in the meanwhile, ios users know that their data is no longer as secure as they once thought. around the same time, the senate’s draft bill titled as “compliance with court orders act of ,” proposed that people should be required to comply with any authorized court order for data and that if that data is “unintelligible” – meaning encrypted – then it must be decrypted for the court. this bill is problematic because it practically nullifies the efficacy of any end-to-end encryption, which we use everyday from our iphones to messaging services like whatsapp and signal. because security is essential to privacy, it is ironic that certain cybersecurity measures are used to greatly invade privacy rather than protect it. because we do not always fully understand how the technology actually works or how it can be exploited for both good and bad purposes, we need to be careful about giving blank permission to any party to access, collect, and use our private data without clear understanding, oversight, and consent. as we share more and more information online, cyberattacks will only increase, and organizations and the government will struggle even more to balance privacy concerns with security issues. why libraries should advocate for online privacy? the fact that people may no longer have privacy on the web should concern libraries. historically, libraries have been strong advocates of intellectual freedom striving to keep patron’s data safe and protected from the unwanted eyes of the authorities. as librarians, we believe in people’s right to read, think, and speak freely and privately as long as such an act itself does not pose harm to others. the library freedom project is an example that reflects this belief held strongly within the library community. it educates librarians and their local communities about surveillance threats, privacy rights and law, and privacy-protecting technology tools to help safeguard digital freedom, and helped the kilton public library in lebanon, new hampshire, to become the first library to operate a tor exit relay, to provide anonymity for patrons while they browse the internet at the library. new technologies brought us the unprecedented convenience of collecting, storing, and sharing massive amount of sensitive data online. but the fact that such sensitive data can be easily exploited by falling into the wrong hands created also the unparalleled level of potential invasion of privacy. while the majority of librarians take a very strong stance in favor of intellectual freedom and against censorship, it is often hard to discern a correct stance on online privacy particularly when it is pitted against cybersecurity. some even argue that those who have nothing to hide do not need their privacy at all. however, privacy is not equivalent to hiding a wrongdoing. nor do people keep certain things secrets because those things are necessarily illegal or unethical. being watched / will drive any person crazy whether s/he is guilty of any wrongdoing or not. privacy allows us safe space to form our thoughts and consider our actions on our own without being subject to others’ eyes and judgments. even in the absence of actual massive surveillance, just the belief that one can be placed under surveillance at any moment is sufficient to trigger self-censorship and negatively affects one’s thoughts, ideas, creativity, imagination, choices, and actions, making people more conformist and compliant. this is further corroborated by the recent study from oxford university, which provides empirical evidence that the mere existence of a surveillance state breeds fear and conformity and stifles free expression. privacy is an essential part of being human, not some trivial condition that we can do without in the face of a greater concern. that’s why many people under political dictatorship continue to choose death over life under mass surveillance and censorship in their fight for freedom and privacy. the electronic frontier foundation states that privacy means respect for individuals’ autonomy, anonymous speech, and the right to free association. we want to live as autonomous human beings free to speak our minds and think on our own. if part of a library’s mission is to contribute to helping people to become such autonomous human beings through learning and sharing knowledge with one another without having to worry about being observed and/or censored, libraries should advocate for people’s privacy both online and offline as well as in all forms of communication technologies and devices. posted in: library, technology, usability, user experience, web. tagged: data security · digital freedom · encryption · internet · password · soc · tor three recent talks of mine on ux, data visualization, and it management apr th, by bohyun (library hat). comments are off for this post i have been swamped at work and pretty quiet here in my blog. but i gave a few talks recently. so i wanted to share those at least. i presented about how to turn the traditional library it department and its operation that is usually behind the scene into a more patron-facing unit at the recent american library association midwinter meeting back in january. this program was organized by the lita heads of it interest group. in march, i gave a short lightning talk at the code lib conference about the data visualization project of library data at my library. i was also invited to speak at the usmai (university system of maryland and affiliated institutions) ux unconference and gave a talk about user experience, personas, and the idea of applying library personas to library strategic planning. here are those three presentation slides for those interested! strategically ux oriented with personas from bohyun kim visualizing library data from bohyun kim turning the it dept. outward from bohyun kim posted in: ala, library, presentation, technology, usability, user experience. tagged: code lib · data visualization · it · management · ux near us and libraries, robots have arrived oct th, by bohyun (library hat). comments are off for this post ** this post was originally published in acrl techconnect on oct. , .*** the movie, robot and frank, describes the future in which the elderly have a robot as their companion and also as a helper. the robot monitors various activities that relate to both mental and physical health and helps frank with various house chores. but frank also enjoys the robot’s company and goes on to enlist the robot into his adventure of breaking into a local library to steal a book and a greater heist later on. people’s lives in the movie are not particularly futuristic other than a robot in them. and even a robot may not be so futuristic to us much longer either. as a matter of fact, as of june , there is now a commercially available humanoid robot that is close to performing some of the functions that the robot in the movie ‘frank and robot’ does. pepper robot, image from aldebaran, https://www.aldebaran.com/en/a-robots/who-is-pepper a japanese company, softbank robotics corp. released a humanoid robot named ‘pepper’ to the market back in june. the pepper robot is feet tall, pounds, speaks languages and is equipped with an array of cameras, touch sensors, accelerometer, and other sensors in his “endocrine-type multi-layer neural network,” according to the cnn report. the pepper robot was priced at ¥ , ($ , ). the pepper owners are also responsible for an additional ¥ , ($ ) monthly data and insurance fee. while the pepper robot is not exactly cheap, it is surprisingly affordable for a robot. this means that the robot industry has now matured to the point where it can introduce a robot that the mass can afford. robots come in varying capabilities and forms. some robots are as simple as a programmable cube block that can be combined with one another to be built into a working unit. for example, cubelets from modular robotics are modular robots that are used for educational purposes. each cube performs one specific function, such as flash, battery, temperature, brightness, rotation, etc. and one can combine these blocks together to build a robot that performs a certain function. for example, you can build a lighthouse robot by combining a battery block, a light-sensor block, a rotator block, and a flash block. a variety of cubelets available from the modular robotics website. by contrast, there are advanced robots such as those in the form of an animal developed by a robotics company, boston dynamics. some robots look like a human although much smaller than the pepper robot. nao is a -cm tall humanoid robot that moves, recognizes, hears and talks to people that was launched in . nao robots are an interactive educational toy that helps students to learn programming in a fun and practical way. noticing their relevance to stem education, some libraries are making robots available to library patrons. westport public library provides robot training classes for its two nao robots. chicago public library lends a number of finch robots that patrons can program to see how they work. in celebration of the national robotics week back in april, san diego public library hosted their first robot day educating the public about how robots have impacted the society. san diego public library also started a weekly robotics club inviting anyone to join in to help build or learn how to build a robot for the library. haslet public library offers the robotics camp program for th to th graders who want to learn how to build with lego mindstorms ev kits. school librarians are also starting robotics clubs. the robotics club at new rochelle high school in new york is run by the school’s librarian, ryan paulsen. paulsen’s robotics club started with faculty, parent, and other schools’ help along with a grant from nasa and participated in a first robotics competition. organizations such as the robotics academy at carnegie mellon university provides educational outreach and resources. image from aldebaran website at https://www.aldebaran.com/en/humanoid-robot/nao-robot there are also libraries that offer coding workshops often with arduino or raspberry pi, which are inexpensive computer hardware. ames free library offers raspberry pi workshops. san diego public library runs a monthly arduino enthusiast meetup. arduinos and raspberry pis can be used to build digital devices and objects that can sense and interact the physical world, which are close to a simple robot. we may see more robotics programs at those libraries in the near future. robots can fulfill many other functions than being educational interactive toys, however. for example, robots can be very useful in healthcare. a robot can be a patient’s emotional companion just like the pepper. or it can provide an easy way to communicate for a patient and her/his caregiver with physicians and others. a robot can be used at a hospital to move and deliver medication and other items and function as a telemedicine assistant. it can also provide physical assistance for a patient or a nurse and even be use for children’s therapy. humanoid robots like pepper may also serve at a reception desk at companies. and it is not difficult to imagine them as sales clerks at stores. robots can be useful at schools and other educational settings as well. at a workplace, teleworkers can use robots to achieve more active presence. for example, universities and colleges can offer a similar telepresence robot to online students who want to virtually experience and utilize the campus facilities or to faculty who wish to offer the office hours or collaborate with colleagues while they are away from the office. as a matter of fact, the university of texas, arlington, libraries recently acquired several telepresence robots to lend to their faculty and students. not all robots do or will have the humanoid form as the pepper robot does. but as robots become more and more capable, we will surely get to see more robots in our daily lives. references alpeyev, pavel, and takashi amano. “robots at work: softbank aims to bring pepper to stores.” bloomberg business, june , . http://www.bloomberg.com/news/articles/ - - /robots-at-work-softbank-aims-to-bring-pepper-to-stores. “boston dynamics.” accessed september , . http://www.bostondynamics.com/. boyer, katie. “robotics clubs at the library.” public libraries online, june , . http://publiclibrariesonline.org/ / /robotics-clubs-at-the-library/. “finch robots land at cpl altgeld.” chicago public library, may , . https://www.chipublib.org/news/finch-robots-land-at-cpl/. mcnickle, michelle. “ medical robots that could change healthcare – informationweek.” informationweek, december , . http://www.informationweek.com/mobile/ -medical-robots-that-could-change-healthcare/d/d-id/ . singh, angad. “‘pepper’ the emotional robot, sells out within a minute.” cnn.com, june , . http://www.cnn.com/ / / /tech/pepper-robot-sold-out/. tran, uyen. “sdpl labs: arduino aplenty.” the library incubator project, april , . http://www.libraryasincubatorproject.org/?p= . “ut arlington library to begin offering programming robots for checkout.” university of texas arlington, march , . https://www.uta.edu/news/releases/ / /library-robots- .php. waldman, loretta. “coming soon to the library: humanoid robots.” wall street journal, september , , sec. new york. http://www.wsj.com/articles/coming-soon-to-the-library-humanoid-robots- . posted in: library, technology. tagged: education · libraries · robotics · robots · stem ← earlier posts subscribe to our feed via rss search about libraryhat is a blog written by bohyun kim, cto & associate professor at the university of rhode island libraries (bohyun.kim.ois [at] gmail [dot] com; @bohyunkim). most popular - libraries meet the second machine age - future? libraries? what now? – after the ala summit on the future of libraries - query a google spreadsheet like a database with google visualization api query language - enabling the research ‘flow’ and serendipity in today’s digital library environment - research librarianship in crisis: mediate when, where, and how? - why not grow coders from the inside of libraries? - do you feel inadequate? for hard-working overachievers - redesigning the item record summary view in a library catalog and a discovery interface - fear no longer regular expressions - using git with bitbucket: basic commands – pull, add, commit, push - aaron swartz and too-comfortable research libraries - common misconceptions about library job search: what i have learned from the other side of the table - applying game dynamics to library services - how to make your writing less terrible - netflix and libraries: you are what “your users” think you are, not what you think you are archives july ( ) december ( ) october ( ) may ( ) november ( ) may ( ) april ( ) october ( ) september ( ) july ( ) march ( ) february ( ) september ( ) june ( ) may ( ) march ( ) december ( ) november ( ) october ( ) september ( ) july ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) march ( ) february ( ) january ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) april ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) tags acrl ala api change codeyear coding communication conference continuing education design election emerging technologies equity inclusion interview with brand-new librarians it javascript job job search jquery kindle libcodeyear librarian libraries library library day in the life lis lita makerspace management mls mobile new librarians post-mls presentation programming publication technology tips tweet-up twitter usability ux web © library hat | powered by wordpress a wordpress theme by ravi varma none none internet alchemy internet alchemy serverless: why microfunctions > microservices gorecipes: fin another blog refresh why service architectures should focus on workflows help me crowdfund my game amberfell andromeda yelton andromeda yelton archival face recognition for fun and nonprofit in , dominique luster gave a super good code lib talk about applying ai to metadata for the charles &# ;teenie&# ; harris collection at the carnegie museum of art &# ; more than , photographs of black life in pittsburgh. they experimented with solutions to various metadata problems, but the one that&# ;s stuck in my head since &# ; continue reading archival face recognition for fun and&# ;nonprofit &# ; sequence models of language: slightly irksome not much ai blogging this week because i have been buried in adulting all week, which hasn&# ;t left much time for machine learning. sadface. however, i&# ;m in the last week of the last deeplearning.ai course! (well. of the deeplearning.ai sequence that existed when i started, anyway. they&# ;ve since added an nlp course and a gans &# ; continue reading sequence models of language: slightly&# ;irksome &# ; adapting coursera’s neural style transfer code to localhost last time, when making cats from the void, i promised that i&# ;d discuss how i adapted the neural style transfer code from coursera&# ;s convolutional neural networks course to run on localhost. here you go! step : first, of course, download (as python) the script. you&# ;ll also need the nst_utils.py file, which you can access via &# ; continue reading adapting coursera&# ;s neural style transfer code to&# ;localhost &# ; dear internet, merry christmas; my robot made you cats from the void recently i learned how neural style transfer works. i wanted to be able to play with it more and gain some insights, so i adapted the coursera notebook code to something that works on localhost (more on that in a later post), found myself a nice historical cat image via dpla, and started mashing it &# ; continue reading dear internet, merry christmas; my robot made you cats from the&# ;void &# ; this week in my ai after visualizing a whole bunch of theses and learning about neural style transfer and flinging myself at t-sne i feel like i should have something meaty this week but they can&# ;t all be those weeks, i guess. still, i&# ;m trying to hold myself to friday ai blogging, so here are some work notes: finished course &# ; continue reading this week in my&# ;ai &# ; though these be matrices, yet there is method in them. when i first trained a neural net on , theses to make hamlet, one of the things i most wanted to do is be able to visualize them. if word vec places documents &# ;near&# ; each other in some kind of inferred conceptual space, we should be able to see some kind of map of them, yes? &# ; continue reading though these be matrices, yet there is method in&# ;them. &# ; of such stuff are (deep)dreams made: convolutional networks and neural style transfer skipped fridai blogging last week because of thanksgiving, but let&# ;s get back on it! top-of-mind today are the firing of ai queen timnit gebru (letter of support here) and a couple of grant applications that i&# ;m actually eligible for (this is rare for me! i typically need things for which i can apply in my &# ; continue reading of such stuff are (deep)dreams made: convolutional networks and neural style&# ;transfer &# ; let’s visualize some hamlet data! or, d and t-sne for the lols. in , i trained a neural net on ~ k graduate theses using the doc vec algorithm, in hopes that doing so would provide a backend that could support novel and delightful discovery mechanisms for unique library content. the result, hamlet, worked better than i hoped; it not only pulls together related works from different departments (thus &# ; continue reading let&# ;s visualize some hamlet data! or, d and t-sne for the&# ;lols. &# ; ai in the library, round one the san josé state university school of information wanted to have a half-course on artificial intelligence in their portfolio, and asked me to develop and teach it. (thanks!) so i got a blank canvas on which to paint eight weeks of&# ;whatever you might want graduate students in library &# ; information science students to know about &# ; continue reading ai in the library, round&# ;one &# ; adventures with parsing django uploaded csv files in python let&# ;s say you&# ;re having problems parsing a csv file, represented as an inmemoryuploadedfile, that you&# ;ve just uploaded through a django form. there are a bunch of answers on stackoverflow! they all totally work with python ! &# ;and lead to hours of frustration if, say, hypothetically, like me, you&# ;re using python . if you are getting &# ; continue reading adventures with parsing django uploaded csv files in&# ;python &# ; dshr's blog: stablecoins dshr's blog i'm david rosenthal, and this is a place to discuss the work i'm doing in digital preservation. tuesday, december , stablecoins i have long been skeptical of bitcoin's "price" and, despite its recent massive surge, i'm still skeptical. but it turns out i was wrong two years ago when i wrote in blockchain: what's not to like?: permissionless blockchains require an inflow of speculative funds at an average rate greater than the current rate of mining rewards if the "price" is not to collapse. to maintain bitcoin's price at $ k requires an inflow of $ k/hour. i found it hard to believe that this much actual money would flow in, but since then bitcoin's "price" hasn't dropped below $ k, so i was wrong. caution — i am only an amateur economist, and what follows below the fold is my attempt to make sense of what is going on. first, why did i write that? the economic argument is that, because there is a low barrier to entry for new competitors, margins for cryptocurrency miners are low. so the bulk of their income in terms of mining rewards has to flow out of the system in "fiat" currency to pay for their expenses such as power and hardware. these cannot be paid in cryptocurrencies. at the time, the bitcoin block reward was . btc/block, or btc/hour. at $ k/btc this was $ k/hour, so on average k usd/hour had to flow in from speculators if the system was not to run out of usd. source what has happened since then? miners' income comes in two parts, transaction fees (currently averaging around btc/day) and mining rewards ( btc/day) for a total around k btc/day. at $ k/btc, that is $ k/hour. the combination of halving of the block reward, increasing transaction fees, and quintupling the "price" has roughly tripled the required inflow. second, lets set the context for what has happened in cryptocurrencies in the last year. source in the last year bitcoin's "market cap" went from around $ b to around $ b ( . x) and its "price" went from about $ k to about $ k. source in the last year ethereum's "market cap" went from around $ b to around $ b ( . x) and its "price went from around $ to around $ . source the key observation that explains why i write "price" in quotes is shown in this graph. very little of the trading in btc is in terms of usd, most of it is in terms of tether (usdt). the "price" is set by how many usdt people are prepared to pay for btc, not by how many usd. the usd "price" follows because people believe that usdt ≅ usd. source in the past year, tether's "market cap" has gone from about b usdt to about b usdt ( x). tether (usdt) is a "stablecoin", intended to maintain a stable price of usd = usdt. initially, tether claimed that it maintained a stable "price" because every usdt was backed by an actual usd in a bank account. does that mean that investors transferred around sixteen billion us dollars into tether's bank account in the past year? no-one believes that. there has never been an audit of tether to confirm what is backing usdt. tether themselves admitted to the new york attorney general in october that: the $ . billion worth of tethers are only % backed: tether has cash and cash equivalents (short term securities) on hand totaling approximately $ . billion, representing approximately percent of the current outstanding tethers. if usdt isn't backed by usd, what is backing it, and is usdt really worth usd? source just in october, tether minted around b usdt. the graph tracks the "price" of bitcoin against the "market cap" of usdt. does it look like they're correlated? amy castor thinks so. tether transfers newly created usdt to an exchange, where one of two things can happen to it: it can be used to buy usd or an equivalent "fiat" currency. but only a few exchanges allow this. for example, coinbase, the leading regulated exchange, will not provide this "fiat off-ramp": please note that coinbase does not support usdt — do not send it to your bitcoin account on coinbase. because of usdt's history and reputation, exchanges that do offer a "fiat off-ramp" are taking a significant risk, so they will impose a spread; the holder will get less than $ . why would you send $ to tether to get less than $ back? it can be used to buy another cryptocurrency, such as bitcoin (btc) or ethereum (eth), increasing demand for that cryptocurrency and thus increasing its price. since newly created usdt won't be immediately sold for "fiat", they will pump the "price" of cryptocurrencies. for simplicity of explanation, lets imagine a world in which there are only usd, usdt and btc. in this world some proportion of the backing for usdt is usd and some is btc. someone sends usd to tether. why would they do that? they don't want usdt as a store of value, because they already have usd, which is obviously a better store of value than usdt. they want usdt in order to buy btc. tether adds the usd to the backing for usdt, and issues the corresponding number of usdt, which are used to buy btc. this pushes the "price" of btc up, which increases the "value" of the part of the backing for usdt that is btc. so tether issues the corresponding amount of usdt, which is used to buy btc. this pushes the "price" of btc up, which increases the "value" of the part of the backing for usdt that is btc. ... tether has a magic "money" pump, creating usdt out of thin air. but there is a risk. suppose for some reason the "price" of btc goes down, which reduces the "value" of the backing for usdt. now there are more usdt in circulation than are backed. so tether must buy some usdt back. they don't want to spend usd for this, because they know that usd are a better store of value than usdt created out of thin air. so they need to sell btc to get usdt. this pushes the "price" of btc down, which reduces the "value" of the part of the backing for usdt that is btc. so tether needs to buy more usdt for btc, which pushes the "price" of btc down. ... the magic "money" pump has gone into reverse, destroying the usdt that were created out of thin air. tether obviously wants to prevent this happening, so in our imaginary world what we would expect to see is that whenever the "price" of btc goes down, tether supplies the market with usdt, which are used to buy btc, pushing the price back up. over time, the btc "price" would generally go up, keeping everybody happy. but there is a second-order effect. over time, the proportion of the backing for usdt that is btc would go up too, because each usd that enters the backing creates r> usd worth of "value" of the btc part of the backing. and, over time, this effect grows because the greater the proportion of btc in the backing, the greater r becomes. source in our imaginary world we would expect to see: the "price" of btc correlated with the number of usdt in circulation. the graph shows this in the real world. both the "price" of btc and the number of usdt in circulation growing exponentially. the graph shows this in the real world. spikes in the number of usdt in circulation following falls in the "price" of btc. is bitcoin really untethered? by john griffin and amit shams shows that: rather than demand from cash investors, these patterns are most consistent with the supply‐based hypothesis of unbacked digital money inflating cryptocurrency prices. their paper was originally published in and updated in and . tether being extremely reluctant to be audited because that would reveal how little money and how much "money" was supporting the btc "price". our imaginary world replicates key features of the real world. of course, since tether has never been audited, we don't know the size or composition of usdt's backing. so we don't know whether tether has implemented a magic "money" pump. but the temptation to get rich quick by doing so clearly exists, and tether's history isn't reassuring about their willingness to skirt the law. because of the feedback loops i described, if they ever dipped a toe in the flow from a magic "money" pump, they would have to keep doubling down. apart from the work of griffin and shams, there is a whole literature pointing out the implausibility of tether's story. here are a few highlights: jp konig's things about tether stablecoins social capital's series explaining tether and the "stablecoin" scam: pumps, spoofs and boiler rooms tether, part one: the stablecoin dream tether, part two: pokedex tether, part three: crypto island price manipulation in the bitcoin ecosystem by neil gandal et al cryptocurrency pump-and-dump schemes by tao li et al patrick mckenzie's tether: the story so far: a friend of mine, who works in finance, asked me to explain what tether was. short version: tether is the internal accounting system for the largest fraud since madoff. bernie madoff's $ . b ponzi scheme was terminated in but credible suspicions had been raised nine years earlier, not least by the indefatigable harry markopolos. credible suspicions were raised against wirecard shortly after it was incorporated in , but even after the financial times published a richly documented series based on whistleblower accounts it took almost a year before wirecard declared bankruptcy owing € . b. massive frauds suffer from a "wile e. coyote" effect. because they are "too big to fail" there is a long time between the revelation that they are frauds, and the final collapse. it is hard for people to believe that, despite numbers in the billions, there is no there there. both investors and regulators get caught up in the excitement and become invested in keeping the bubble inflated by either attacking or ignoring negative news. for example, we saw this in the wirecard scandal: bafin conducted multiple investigations against journalists and short sellers because of alleged market manipulation, in response to negative media reporting of wirecard. ... critics cite the german regulator, press and investor community's tendency to rally around wirecard against what they perceive as unfair attack. ... after initially defending bafin's actions, its president felix hufeld later admitted the wirecard scandal is a "complete disaster". similarly, the cryptocurrency world has a long history of both attacking and ignoring realistic critiques. an example of ignoring is the dao: the decentralized autonomous organization (the dao) was released on th april , but on th may dino mark, vlad zamfir, and emin gün sirer posted a call for a temporary moratorium on the dao, pointing out some of its vulnerabilities; it was ignored. three weeks later, when the dao contained about % of all the ether in circulation, a combination of these vulnerabilities was used to steal its contents. source the graph shows how little of the trading in btc is in terms of actual money, usd. on coinmarketcap.com as i write, usdt has a "market cap" of nearly $ b and the next largest "stablecoin" is usdc, at just over $ . b. usdc is audited and[ ] complies with banking regulations, which explains why it is used so much less. the supply of usdc can't expand enough to meet demand. the total "market cap" of all the cryptocurrencies the site tracks is $ b, an increase of more than % in the last day! so just one day is around the same as bernie madoff's ponzi scheme. the top cryptocurrencies (btc, eth, xrp, usdt) account for $ b ( %) of the total "market cap"; the others are pretty insignificant. david gerard points out the obvious in tether is “too big to fail” — the entire crypto industry utterly depends on it: the purpose of the crypto industry, and all its little service sub-industries, is to generate a narrative that will maintain and enhance the flow of actual dollars from suckers, and keep the party going. increasing quantities of tethers are required to make this happen. we just topped twenty billion alleged dollars’ worth of tethers, sixteen billion of those just since march . if you think this is sustainable, you’re a fool. gerard links to bryce weiner's hopes, expectations, black holes, and revelations — or how i learned to stop worrying and love tether which starts from the incident in april of when bitfinex, the cryptocurrency exchange behind tether, encountered a serious problem: the wildcat bank backing tether was raided by interpol for laundering of criminally obtained assets to the tune of about $ , , . the percentage of that sum which was actually bitfinex is a matter of some debate but there’s no sufficient reason not to think it was all theirs. ... the nature of the problem also presented a solution: instead of backing tether in actual dollars, stuff a bunch of cryptocurrency in a basket to the valuation of the cash that got seized and viola! a black hole is successfully filled with a black hole, creating a stable asset. at the time, usdt's "market cap" was around $ . b, so assuming tether was actually backed by usd at that point, it lost % of its backing. this was a significant problem, more than enough to motivate shenanigans. weiner goes on to provide a detailed explanation, and argue that tether is impossible to shut down. he may be right, but it may be possible to effectively eliminate the "fiat off-ramp", thus completely detaching usdt and usd. this would make it clear that "prices" expressed in usdt are imaginary, not the same as prices expressed in usd. source postscript: david gerard recounts the pump that pushed btc over $ k: we saw about million tethers being lined up on binance and huobi in the week previously. these were then deployed en masse. you can see the pump starting at : utc on december. btc was $ , . on coinbase at : utc. notice the very long candles, as bots set to sell at $ , sell directly into the pump. see cryptocurrency pump-and-dump schemes by tao li, donghwa shin and baolian wang. source ki joung yu watched the pump in real time: lots of people deposited stablecoins to exchanges mins before breaking $ k. price is all about consensus. i guess the sentiment turned around to buy $btc at that time. ... eth block interval is - seconds. this chart means exchange users worldwide were trying to deposit #stablecoins in a single block — seconds. note that " mins" is about one bitcoin block time, and by "exchange users" he means "addresses — it could have been a pre-programmed "smart contract". [ ] david gerard points out that: usdc loudly touts claims that it’s well-regulated, and implies that it’s audited. but usdc is not audited — accountants grant thornton sign a monthly attestation that centre have told them particular things, and that the paperwork shows the right numbers. posted by david. at : am labels: bitcoin comments: david. said... xrp, the third-largest unstablecoin by "market cap", has lost almost % of its value over the last days. this might have something to do with the sec suing ripple labs, who control the centralized cryptocurrency, claiming that xrp is an unregistered security. the sec's argument, bolstered with copious statements by the founders, is at heart that the founders pre-mined and kept vast amounts of xrp, which they then pump and sell: "defendants continue to hold substantial amounts of xrp and — with no registration statement in effect — can continue to monetize their xrp while using the information asymmetry they created in the market for their own gain, creating substantial risk to investors." david gerard points out that ripple labs knew they should have registered: "ripple received legal advice in february and october that xrp could constitute an “investment contract,” thus making it a security under the howey test — particularly if ripple promoted xrp as an investment. the lawyers advised ripple to contact the sec for clarification before proceeding. ripple went ahead anyway, without sec advice — and raised “at least $ . billion” selling xrp from to the present day, promoting it as an investment all the while" izabella kaminska notes that other cryptocurrencies may have similar legal issues: "this may concern other cryptocurrencies such as ethereum and eos, which unlike bitcoin were pre-sold to the public in a similar fashion" december , at : am david. said... izabella kaminska has the highlights of the sec filing against ripple labs. they are really damning. december , at : am david. said... david gerard points to this transaction and asks: "don’t you hate it when you send $ . in btc with a fee of $ , ? i guess they can call bitcoin customer service and get it sorted out! it’s not clear if this transaction ever showed up in the mempool — or if it was the miner putting it directly into the block, and doing some bitcoin-laundering." december , at : pm david. said... the magic "money" pump is working overtime to make santa gifts for the children: "tether has issued million tethers in just the past few days, million of those just today. the market pumpers seem to have been blindsided by the sec suit against ripple, and are trying frantically to avert a christmas crash. i’m sure there’s a ton of institutional investors going all-out on christmas eve." december , at : pm david. said... amy castor collected predictions for from cryptocurrency skeptics in nocoiner predictions: will be a year of comedy gold. they're worth reading. for example: "since , the new york attorney general has been investigating tether and its sister company, crypto exchange bitfinex, for fraud. over the summer, the supreme court ruled that the companies need to hand over their financial records to show once and for all just how much money really is underlying the tethers they keep printing. the nyag said bitfinex/tether have agreed to do so by jan. ." david gerard expanded on his predictions in in crypto and blockchain: your % reliable guide to the future, including: "we’re currently in the throes of a completely fake bitcoin bubble. this is fueled by billions of tethers, backed by loans, or maybe bitcoins, or maybe hot air. large holders are spending corporate money on bitcoins, fundamentally to promote the value of their own holdings. retail hasn’t shown up — there’s a lack of actual dollars in the exchange system. one btc sale last night ( january) dropped the price $ , . if btc crashes the price, then almost nobody will be able to get out without massive losses. the dollars don’t appear to exist when tested." january , at : pm david. said... in parasitic stablecoins time swanson focuses in exhaustive detail on the dependence of stablecoins on the banking system: "this post will go through some of the background for what commercial bank-backed stablecoins are, the loopholes that the issuers try to reside in, how reliant the greater cryptocurrency world is dependent on u.s. and e.u. commercial banks, and how the principles for financial market structures, otherwise known as pfmis, are being ignored" january , at : pm david. said... cas piancy's brief history of tether entitled a tl; dr for tether and imf researcher john kiff's kiffmeister's #fintech daily digest ( / / ) are both worth reading for views on tether. january , at : pm david. said... further regulation of cryptocurrency on- and off-ramps is announced by fincen in the financial crimes enforcement network proposes rule aimed at closing anti-money laundering regulatory gaps for certain convertible virtual currency and digital asset transactions: "the proposed rule complements existing bsa requirements applicable to banks and msbs by proposing to add reporting requirements for cvc and ltda transactions exceeding $ , in value. pursuant to the proposed rule, banks and msbs will have days from the date on which a reportable transaction occurs to file a report with fincen. further, this proposed rule would require banks and msbs to keep records of a customer’s cvc or ltda transactions and counterparties, including verifying the identity of their customers, if a counterparty uses an unhosted or otherwise covered wallet and the transaction is greater than $ , ." january , at : pm david. said... amy castor has transcribed an interview with paolo ardino and stuart hoegner of tether. hoegner is their general counsel, and he said: "we were very clear last summer in court that part of it is in bitcoin. and if nothing else, there are transaction fees that need to be paid on the omni layer. so bitcoin was and is needed to pay for those transactions, so that shouldn’t come as a surprise to anyone. and we don’t presently comment on our asset makeup overall as a general manner, but we are contemplating starting a process of providing updates on that on the website in this year, in ." so my speculation in this post is confirmed. they do have a magic "money" machine. january , at : pm david. said... there's nothing new under the sun. david gerard's stablecoins through history — michigan bank commissioners report, starts: "a “stablecoin” is a token that a company issues, claiming that the token is backed by currency or assets held in a reserve. the token is usually redeemable in theory — and sometimes in practice. stablecoins are a venerable and well-respected part of the history of us banking! previously, the issuers were called “wildcat banks,” and the tokens were pieces of paper. the wildcat banking era, more politely called the “free banking era,” ran from to . banks at this time were free of federal regulation — they could launch just under state regulation. under the gold standard in operation at the time, these state banks could issue notes, backed by specie — gold or silver — held in reserve. the quality of these reserves could be a matter of some dispute. the wildcat banks didn’t work out so well. the national bank act was passed in , establishing the united states national banking system and the office of the comptroller of the currency — and taking away the power of state banks to issue paper notes." go read the whole post - the parallels with cryptocurrencies are striking. january , at : pm post a comment newer post older post home subscribe to: post comments (atom) blog rules posts and comments are copyright of their respective authors who, by posting or commenting, license their work under a creative commons attribution-share alike . united states license. off-topic or unsuitable comments will be deleted. dshr dshr in anwr recent comments full comments blog archive ► ( ) ► february ( ) ► january ( ) ▼ ( ) ▼ december ( ) michael nelson's group on archiving twitter stablecoins risc vs. cisc max ungrounding ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► july ( ) ► june ( ) ► february ( ) ► ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) lockss system has permission to collect, preserve, and serve this archival unit. simple theme. powered by blogger. none zbw labs zbw labs building the swib participants map here we describe the process of building the interactive swib participants map, created by a query to wikidata. the map was intended to support participants of swib to make contacts in the virtual conference space. however, in compliance with gdpr we want to avoid publishing personal details. so we choose to publish a map of institutions, to which the participants are affiliated. (obvious downside: the un-affiliated participants could not be represented on the map). we suppose that the method can be applied to other conferences and other use cases - e.g., the downloaders of scientific software or the institutions subscribed to an academic journal. therefore, we describe the process in some detail. we started with a list of institution names (with country code and city, but without person ids), extracted and transformed from our conftool registration system, saved it in csv format. country names were normalized, cities were not (and only used for context information). we created an openrefine project, and reconciled the institution name column with wikidata items of type q (organization, and all its subtypes). we included the country column (-> p , country) as relevant other detail, and let openrefine “auto-match candidates with high confidence”. of our original set of country/institution entries, were automaticaly matched via the wikidata reconciliation service. at the end of the conference, institutions were identified and put on the map (data set). we went through all un-matched entries and either a) selected one of the suggested items, or b) looked up and tweaked the name string in wikidata, or in google, until we found an according wikipedia page, openend the linked wikidata object from there, and inserted the qid in openrefine, or c) created a new wikidata item (if the institution seemed notable), or d) attached “not yet determined” (q ) where no wikidata item (yet) exists, or e) attached “undefined value” (q ) where no institution had been given the results were exported from openrefine into a .tsv file (settings) again via a script, we loaded conftool participants data, built a lookup table from all available openrefine results (country/name string -> wd item qid), aggregated participant counts per qid, and loaded that data into a custom sparql endpoint, which is accessible from the wikidata query service. as in step , for all (new) institution name strings, which were not yet mapped to wikidata, a .csv file was produced. (an additional remark: if no approved custom sparql endpoint is available, it is feasible to generate a static query with all data in it’s “values” clause.) during the preparation of the conference, more and more participants registered, which required multiple loops: use the csv file of step and re-iterate, starting at step . (since i found no straightforward way to update an existing openrefine project with extended data, i created a new project with new input and output files for every iteration.) finally, to display the map we could run a federated query on wdqs. it fetches the institution items from the custom endpoint and enriches them from wikidata with name, logo and image of the institution (if present), as well as with geographic coordinates, obtained directly or indirectly as follows: a) item has “coodinate location” (p ) itself, or b) item has “headquarters location” item with coordinates (p /p ), or c) item has “located in administrative entity” item with coordinates (p /p ), or c) item has “country” item (p /p ) applying this method, only one institution item could not be located on the map. data improvements the way to improve the map was to improve the data about the items in wikidata - which also helps all future wikidata users. new items for a few institutions, new items were created: burundi association of librarians, archivists and documentalists fao representation in kenya aurora information technology istituto di informatica giuridica e sistemi giudiziari for another institutions, mostly private companies, no items were created due to notability concerns. everything else already had an item in wikidata! improvement of existing items in order to improve the display on the map, we enhanced selected items in wikidata in various ways: add english label add type (instance of) add headquarter location add image and/or logo and we hope, that participants of the conference also took the opportunity to make their institution “look better”, by adding for example an image of it to the wikidata knowledge base. putting wikidata into use for a completely custom purpose thus created incentives for improving “the sum of all human knowledge” step by tiny step. wikidata for authorities linked data &# ; deutsch journal map: developing an open environment for accessing and analyzing performance indicators from journals in economics by franz osorio, timo borst introduction bibliometrics, scientometrics, informetrics and webometrics have been both research topics and practical guidelines for publishing, reading, citing, measuring and acquiring published research for a while (hood ). citation databases and measures had been introduced in the s, becoming benchmarks both for the publishing industry and academic libraries managing their holdings and journal acquisitions that tend to be more selective with a growing number of journals on the one side, budget cuts on the other. due to the open access movement triggering a transformation of traditional publishing models (schimmer ), and in the light of both global and distributed information infrastructures for publishing and communicating on the web that have yielded more diverse practices and communities, this situation has dramatically changed: while bibliometrics of research output in its core understanding still is highly relevant to stakeholders and the scientific community, visibility, influence and impact of scientific results has shifted to locations in the world wide web that are commonly shared and quickly accessible not only by peers, but by the general public (thelwall ). this has several implications for different stakeholders who are referring to metrics in dealing with scientific results: with the rise of social networks, platforms and their use also by academics and research communities, the term 'metrics' itself has gained a broader meaning: while traditional citation indexes only track citations of literature published in (other) journals, 'mentions', 'reads' and 'tweets', albeit less formal, have become indicators and measures for (scientific) impact. altmetrics has influenced research performance, evaluation and measurement, which formerly had been exclusively associated with traditional bibliometrics. scientists are becoming aware of alternative publishing channels and both the option and need of 'self-advertising' their output. in particular academic libraries are forced to manage their journal subscriptions and holdings in the light of increasing scientific output on the one hand, and stagnating budgets on the other. while editorial products from the publishing industry are exposed to a global competing market requiring a 'brand' strategy, altmetrics may serve as additional scattered indicators for scientific awareness and value. against this background, we took the opportunity to collect, process and display some impact or signal data with respect to literature in economics from different sources, such as 'traditional' citation databases, journal rankings and community platforms resp. altmetrics indicators: citec. the long-standing citation service maintainted by the repec community provided a dump of both working papers (as part of series) and journal articles, the latter with significant information on classic impact factors such as impact factor ( and years) and h-index. rankings of journals in economics including scimago journal rank (sjr) and two german journal rankings, that are regularly released and updated (vhb jourqual, handelsblatt ranking). usage data from altmetric.com that we collected for those articles that could be identified via their digital object identifier. usage data from the scientific community platform and reference manager mendeley.com, in particular the number of saves or bookmarks on an individual paper. requirements a major consideration for this project was finding an open environment in which to implement it. finding an open platform to use served a few purposes. as a member of the "leibniz research association," zbw has a commitment to open science and in part that means making use of open technologies to as great extent as possible (the zbw - open scienc...). this open system should allow direct access to the underlying data so that users are able to use it for their own investigations and purposes. additionally, if possible the user should be able to manipulate the data within the system. the first instance of the project was created in tableau, which offers a variety of means to express data and create interfaces for the user to filter and manipulate data. it also can provide a way to work with the data and create visualizations without programming skills or knowledge. tableau is one of the most popular tools to create and deliver data visualization in particular within academic libraries (murphy ). however, the software is proprietary and has a monthly fee to use and maintain, as well as closing off the data and making only the final visualization available to users. it was able to provide a starting point for how we wanted to the data to appear to the user, but it is in no way open. challenges the first technical challenge was to consolidate the data from the different sources which had varying formats and organizations. broadly speaking, the bibliometric data (citec and journal rankings) existed as a spread sheet with multiple pages, while the altmetrics and mendeley data came from a database dumps with multiple tables that were presented as several csv files. in addition to these different formats, the data needed to be cleaned and gaps filled in. the sources also had very different scopes. the altmetrics and mendeley data covered only journals, the bibliometric data, on the other hand, had more than , journals. transitioning from tableau to an open platform was big challenge. while there are many ways to create data visualizations and present them to users, the decision was made to use r to work with the data and shiny to present it. r is used widely to work with data and to present it (kläre ). the language has lots of support for these kinds of task over many libraries. the primary libraries used were r plotly and r shiny. plotly is a popular library for creating interactive visualizations. without too much work plotly can provide features including information popups while hovering over a chart and on the fly filtering. shiny provides a framework to create a web application to present the data without requiring a lot of work to create html and css. the transition required time spent getting to know r and its libraries, to learn how to create the kinds of charts and filters that would be useful for users. while shiny alleviates the need to create html and css, it does have a specific set of requirements and structures in order to function. the final challenge was in making this project accessible to users such that they would be able to see what we had done, have access to the data, and have an environment in which they could explore the data without needing anything other than what we were providing. in order to achieve this we used binder as the platform. at it's most basic binder makes it possible to share a jupyter notebook stored in a github repository with a url by running the jupyter notebook remotely and providing access through a browser with no requirements placed on the user. additionally, binder is able to run a web application using r and shiny. to move from a locally running instance of r shiny to one that can run in binder, instructions for the runtime environment need to be created and added to the repository. these include information on what version of the language to use, which packages and libraries to install for the language, and any additional requirements there might be to run everything. solutions given the disparate sources and formats for the data, there was work that needed to be done to prepare it for visualization. the largest dataset, the bibliographic data, had several identifiers for each journal but without journal names. having the journals names is important because in general the names are how users will know the journals. adding the names to the data would allow users to filter on specific journals or pull up two journals for a comparison. providing the names of the journals is also a benefit for anyone who may repurpose the data and saves them from having to look them up. in order to fill this gap, we used metadata available through research papers in economics (repec). repec is an organization that seeks to "enhance the dissemination of research in economics and related sciences". it contains metadata for more than million papers available in different formats. the bibliographic data contained repec handles which we used to look up the journal information as xml and then parse the xml to find the title of the journal. after writing a small python script to go through the repec data and find the missing names there were only journals whose names were still missing. for the data that originated in an mysql database, the major work that needed to be done was to correct the formatting. the data was provided as csv files but it was not formatted such that it could be used right away. some of the fields had double quotation marks and when the csv file was created those quotes were put into other quotation marks resulting doubled quotation marks which made machine parsing difficult without intervention directly on the files. the work was to go through the files and quickly remove the doubled quotation marks. in addition to that, it was useful for some visualizations to provide a condensed version of the data. the data from the database was at the article level which is useful for some things, but could be time consuming for other actions. for example, the altmetrics data covered only journals but had almost , rows. we could use the python library pandas to go through the all those rows and condense the data down so that there are only rows with the data for each column being the sum of all rows. in this way, there is a dataset that can be used to easily and quickly generate summaries on the journal level. shiny applications require a specific structure and files in order to do the work of creating html without needing to write the full html and css. at it's most basic there are two main parts to the shiny application. the first defines the user interface (ui) of the page. it says what goes where, what kind of elements to include, and how things are labeled. this section defines what the user interacts with by creating inputs and also defining the layout of the output. the second part acts as a server that handles the computations and processing of the data that will be passed on to the ui for display. the two pieces work in tandem, passing information back and forth to create a visualization based on user input. using shiny allowed almost all of the time spent on creating the project to be concentrated on processing the data and creating the visualizations. the only difficulty in creating the frontend was making sure all the pieces of the ui and server were connected correctly. binder provided a solution for hosting the application, making the data available to users, and making it shareable all in an open environment. notebooks and applications hosted with binder are shareable in part because the source is often a repository like github. by passing a github repository to binder, say one that has a jupyter notebook in it, binder will build a docker image to run the notebook and then serve the result to the user without them needing to do anything. out of the box the docker image will contain only the most basic functions. the result is that if a notebook requires a library that isn't standard, it won't be possible to run all of the code in the notebook. in order to address this, binder allows for the inclusion in a repository of certain files that can define what extra elements should be included when building the docker image. this can be very specific such as what version of the language to use and listing various libraries that should be included to ensure that the notebook can be run smoothly. binder also has support for more advanced functionality in the docker images such as creating a postgres database and loading it with data. these kinds of activities require using different hooks that binder looks for during the creation of the docker image to run scripts. results and evaluation the final product has three main sections that divide the data categorically into altmetrics, bibliometrics, and data from mendeley. there are additionally some sections that exist as areas where something new could be tried out and refined without potentially causing issues with the three previously mentioned areas. each section has visualizations that are based on the data available. considering the requirements for the project, the result goes a long way to meeting the requirements. the most apparent area that the journal map succeeds in is its goals is of presenting data that we have collected. the application serves as a dashboard for the data that can be explored by changing filters and journal selections. by presenting the data as a dashboard, the barrier to entry for users to explore the data is low. however, there exists a way to access the data directly and perform new calculations, or create new visualizations. this can be done through the application's access to an r-studio environment. access to r-studio provides two major features. first, it gives direct access to the all the underlying code that creates the dashboard and the data used by it. second, it provides an r terminal so that users can work with the data directly. in r-studio, the user can also modify the existing files and then run them from r-studio to see the results. using binder and r as the backend of the applications allows us to provide users with different ways to access and work with data without any extra requirements on the part of the user. however, anything changed in r-studio won't affect the dashboard view and won't persist between sessions. changes exist only in the current session. all the major pieces of this project were able to be done using open technologies: binder to serve the application, r to write the code, and github to host all the code. using these technologies and leveraging their capabilities allows the project to support the open science paradigm that was part of the impetus for the project. the biggest drawback to the current implementation is that binder is a third party host and so there are certain things that are out of our control. for example, binder can be slow to load. it takes on average + minutes for the docker image to load. there's not much, if anything, we can do to speed that up. the other issue is that if there is an update to the binder source code that breaks something, then the application will be inaccessible until the issue is resolved. outlook and future work the application, in its current state, has parts that are not finalized. as we receive feedback, we will make changes to the application to add or change visualizations. as mentioned previously, there a few sections that were created to test different visualizations independently of the more complete sections, those can be finalized. in the future it may be possible to move from binderhub to a locally created and administered version of binder. there is support and documentation for creating local, self hosted instances of binder. going that direction would give more control, and may make it possible to get the docker image to load more quickly. while the application runs stand-alone, the data that is visualized may also be integrated in other contexts. one option we are already prototyping is integrating the data into our subject portal econbiz, so users would be able to judge the scientific impact of an article in terms of both bibliometric and altmetric indicators. references william w. hood, concepcion s. wilson. the literature of bibliometrics, scientometrics, and informetrics. scientometrics , – springer science and business media llc, . link r. schimmer. disrupting the subscription journals’ business model for the necessary large-scale transformation to open access. ( ). link mike thelwall, stefanie haustein, vincent larivière, cassidy r. sugimoto. do altmetrics work? twitter and ten other social web services. plos one , e public library of science (plos), . link the zbw - open science future. link sarah anne murphy. data visualization and rapid analytics: applying tableau desktop to support library decision-making. journal of web librarianship , – informa uk limited, . link christina kläre, timo borst. statistic packages and their use in research in economics | edawax - blog of the project ’european data watch extended’. edawax - european data watch extended ( ). link journal map - binder application for displaying and analyzing metrics data about scientific journals integrating altmetrics into a subject repository - econstor as a use case back in the zbw leibniz information center for economics (zbw) teamed up with the göttingen state and university library (sub), the service center of götting library federation (vzg) and gesis leibniz institute for the social sciences in the *metrics project funded by the german research foundation (dfg). the aim of the project was: “… to develop a deeper understanding of *metrics, especially in terms of their general significance and their perception amongst stakeholders.” (*metrics project about). in the practical part of the project the following dspace based repositories of the project partners participated as data sources for online publications and – in the case of econstor – also as implementer for the presentation of the social media signals: econstor - a subject repository for economics and business studies run by the zbw, currently (aug. ) containing round about , downloadable files, goescholar - the publication server of the georg-august-universität göttingen run by the sub göttingen, offering approximately , publicly browsable items so far, ssoar - the “social science open access repository” maintained by gesis, currently containing about , publicly available items. in the work package “technology analysis for the collection and provision of *metrics” of the project an analysis of currently available *metrics technologies and services had been performed. as stated by [wilsdon ], currently suppliers of altmetrics “remain too narrow (mainly considering research products with dois)”, which leads to problems to acquire *metrics data for repositories like econstor with working papers as the main content. as up to now it is unusual – at least in the social sciences and economics – to create dois for this kind of documents. only the resulting final article published in a journal will receive a doi. based on the findings in this work package, a test implementation of the *metrics crawler had been built. the crawler had been actively deployed from early to spring at the vzg. for the aggregation of the *metrics data the crawler had been fed with persistent identifiers and metadata from the aforementioned repositories. at this stage of the project, the project partners still had the expectation, that the persistent identifiers (e.g. handle, urns, …), or their local url counterparts, as used by the repositories could be harnessed to easily identify social media mentions of their documents, e.g. for econstor: handle: “hdl: /…” handle.net resolver url: “http(s)://hdl.handle.net/ /…” econstor landing page url with handle: “http(s)://www.econstor.eu/handle/ /…” econstor bitstream (pdf) url with handle: “http(s)://www.econstor.eu/bitstream/ /…” this resulted in two datasets: one for publications identified by dois (doi: .xxxx/yyyyy) or the respective metadata from crossref and one for documents identified by the repository urls (https://www.econstor.eu/handle/ /xxxx) or the items metadata stored in the repository. during the first part of the project several social media platforms had been identified as possible data sources for the implementation phase. this had been done by interviews and online surveys. for the resulting ranking see the social media registry. additional research examined which social media platforms are relevant to researchers at different stages of their career and if and how they use them (see: [lemke ], [lemke ] and [mehrazar ]). this list of possible sources for social media citations or mentions had then been further reduced to the following six social media platforms which are offering free and open available online apis: facebook mendeley reddit twitter wikipedia youtube of particular interest to the econstor team were the social media services mendeley and twitter, as those had been found being among the “top most used altmetric sources …” for economic and business studies (ebs) journals “… - with mendeley being the most complete platform for ebs journals” [nuredini ]. *metrics integration in econstor in early the econstor team finally received a mysql data dump of the compiled data which had been collected by the *metrics crawler. in consultations between the project partners and based on the aforementioned research, it became clear, that only the collected data from mendeley, twitter and wikipedia were suitable to be embedded into econstor. it was also made clear, by the vzg, that it had been nearly impossible to use handle or respective local urls to extract social media mentions from the free of charge provided apis of the different social media services. instead, in case of wikipedia isbns had been used and for mendeley the title and author(s) as provided in the repository’s metadata. only for the search via the twitter api the handle urls had been used. the datasets used by the *metrics crawler to identify works from econstor included a dataset of , dois (~ % of the econstor content back then), sometimes representing other manifestations of the documents stored in econstor (e.g. pre- or postprint versions of an article), their respective metadata from the crossref doi registry and also a dataset of , econstor documents identified by the handle/url and metadata stored in the repository itself. this second dataset also included the documents related to the publications identified by the doi set. the following table (table ) shows the results of the *metrics crawler for items in econstor. it displays one row for each service and the used identifier set. each row also shows the time period during which the crawler harvested the service and how many unique items per identifier set were found during that period. social media service (set) harvested from harvested until unique econstor items mentioned mendeley (doi) - - - - , mendeley (url) - - - - , twitter (doi) - - (date of first captured tweet - - ) - - (date of last captured tweet - - ) twitter (url) - - (date of first captured tweet - - ) - - (date of last captured tweet - - ) wikipedia (doi) - - - - wikipedia (url) - - - - table : unique econstor items found per identifier set and social media service the following table (table ) shows how many of the econstor items were found with identifiers from both sets. as you can see, only for the service mendeley the sets have a significant overlap. which shows, that it is desirable for a service such as econstor, to expand the captured coverage of its items in social media by the use of other identifies than just dois. social media site unique items identified by both doi and url mendeley , twitter wikipedia table : overlap in found identifiers as a result of the project, the landing pages of econstor items, which have been mentioned on mendeley, twitter or wikipedia during the time of data gathering, have now, for the time being, a listing of “social media mentions”. this is in addition to the already existing cites and citations, based on the repec - citec service and the download statistics, which is displayed on separate pages. image : “econstor item landing page” the back end on the econstor server is realized as a small restful web service programmed in java that returns json formatted data (see figure ). given a list of identifiers (dois/handle) it returns the sum of mentions for mendeley, twitter and wikipedia in the database, per specified econstor item, as well as the links to the counted tweets and wikipedia articles. in case of wikipedia this is also grouped by the language of the wikipedia the mention was found in. { "_metrics": { "sum_mendeley": , "sum_twitter": , "sum_wikipedia": }, "identifier": " / ", "identifiertype": "handle", "repository": "econstor", "tweetdata": { " ": { "created_at": "wed dec : : + ", "description": "economist wettbewerb regulierung monopole economics @dicehhu @hhu_de vwl antitrust düsseldorf quakenbrück berlin fc st. pauli", "id_str": " ", "name": "justus haucap", "screen_name": "haucap" }, " ": { "created_at": "wed dec : : + ", "description": "twitterkanal des wirtschaftsdienst - zeitschrift für wirtschaftspolitik, hrsg. von @zbw_news; rt ≠ zustimmung; impressum: https://t.co/x gevzb lr", "id_str": " ", "name": "wirtschaftsdienst", "screen_name": "zeitschrift_wd" }, " ": { "created_at": "wed dec : : + ", "description": "professor for international economics at htw berlin - university of applied sciences; senior policy fellow at the european council on foreign relations", "id_str": " ", "name": "sebastian dullien", "screen_name": "sdullien" } }, "twitterids": [ " ", " ", " " ], "wikipediaquerys": {} } figure : “example json returned by webservice - twitter mentions” image : “mendeley and twitter mentions” during the creation of the landing page of an econstor item (see image ), a java servlet queries the web service and, if some social media mentions is detected, renders the result into the web page. for each of the three social media platforms the sum of the mentions is displayed and for twitter and wikipedia even backlinks to the mentioning tweets/articles are provided as a drop-down list, below the number of mentions (see image ). in case of wikipedia this is also grouped by the languages of the articles in wikipedia in which the isbn of the corresponding work has been found. conclusion while being an interesting addition to the existing download statistics and citations by repec/citec, that are already integrated into econstor, currently the gathered “social media mentions” offer only a limited additional value to the econstor landing pages. one reason might be, that only a fraction of all the documents of econstor are covered. another reason might be according to [lemke ], that there is currently a great reluctance to use social media services among economists and social scientists, as it is perceived as: “unsuitable for academic discourse; … to cost much time; … separating personal from professional matters is bothersome; … increases the efforts necessary to handle information overload.” theoretically, the prospect of a tool for the measurement of the scientific uptake, with a quicker response time than classical bibliometrics, could be very rewarding, especially for a repository like econstor with its many preprints (e.g. working papers) provided in open access. as [thelwall ] has stated: “in response, some publishers have turned to altmetrics, which are counts of citations or mentions in specific social web services because they can appear more rapidly than citations. for example, it would be reasonable to expect a typical article to be most tweeted on its publication day and most blogged within a month of publication.” and “social media mentions, being available immediately after publication—and even before publication in the case of preprints…”. but especially these preprints, that come without a doi, are still a challenge to be correctly identified, and therefore to be counted as social media mentions. this is something the *metrics crawler has not changed, since it is using title and author metadata to search in mendeley, which does not give a % sure identification and isbns to search in wikipedia. even though a quick check revealed that at the time of writing this article (aug. ) at least wikipedia offers a handle search. a quick search for econstor handles in the english wikipedia returns now a list of pages with mentions of “hdl: /”, the german wikipedia - but these are still very small numbers (aug. nd, : currently , full texts are available in econstor). https://en.wikipedia.org/w/api.php?action=query&list=search&srlimit= &srsearch=% hdl: % f% &srwhat=text&srprop&srinfo=totalhits&srenablerewrites= &format=jsonsearch via api in english wikipedia another problem is, that at the time of this writing, the *metrics crawler is not continuously operated, therefore our analysis is based on a data dump of social media mentions from spring to early . since it is one of the major benefits of altmetrics that it can be obtained much faster and is more recent then classical citation-based metrics, it reduces the value of the continued integration of this static and continuously getting older dataset being integrated into econstor landing pages. hence, we are looking for more recent and regular updates of social media data that could serve as a ‘real-time’ basis for monitoring social media usage in economics. as a consequence, we are currently looking for: a) an institution to commit itself to run the *metrics crawler and b) a more active social media usage in the sciences of economics and business studies. references [lemke ] lemke, steffen; mehrazar, maryam; mazarakis, athanasios; peters, isabella ( ): are there different types of online research impact?, in: building & sustaining an ethical future with emerging technology. proceedings of the st annual meeting, vancouver, canada, – november , isbn - - - - , association for information science and technology (asis&t), silver spring, pp. - http://hdl.handle.net/ / [lemke ] lemke, steffen; mehrazar, maryam; mazarakis, athanasios; peters, isabella ( ): “when you use social media you are not working”: barriers for the use of metrics in social sciences, frontiers in research metrics and analytics, issn - , vol. , iss. [article] , pp. - , http://dx.doi.org/ . /frma. . [mehrazar ] maryam mehrazar, christoph carl kling, steffen lemke, athanasios mazarakis, and isabella peters ( ): can we count on social media metrics? first insights into the active scholarly use of social media, websci ’ : th acm conference on web science, may – , , amsterdam, netherlands. acm, new york, ny, usa, article , pages, https://doi.org/ . / . [metrics ] einbindung von *metrics in econstor, https://metrics-project.net/downloads/ - - -econstor-metrics-abschluss-ws-sub-g%c %b .pptx [nuredini ] nuredini, kaltrina; peters, isabella ( ): enriching the knowledge of altmetrics studies by exploring social media metrics for economic and business studies journals, proceedings of the st international conference on science and technology indicators (sti conference ), valència (spain), september - , , http://hdl.handle.net/ / [or ] relevance and challenges of altmetrics for repositories - answers from the *metrics project. https://www.conftool.net/or /index.php/paper-p a- orth% cweiland_b.pdf?page=downloadpaper&filename=paper-p a- orth% cweiland_b.pdf&form_id= &form_index= &form_version=final [social media registry] social media registry - current status of social media plattforms and *metrics, https://docs.google.com/spreadsheets/d/ oals kxtmml naf shxh ctmone q efhtzmgpinv /edit?usp=sharing [thelwall ] thelwall m, haustein s, larivie`re v, sugimoto cr ( ): do altmetrics work? twitter and ten other social web services. plos one ( ): e . http://dx.doi.org/ . /journal.pone. [wilsdon ] wilsdon, james et al. ( ): next-generation metrics: responsible metrics and evaluation for open science. report of the european commission expert group on altmetrics, isbn - - - - , http://dx.doi.org/ . / integrating altmetrics data into econstor th century press archives: data donation to wikidata zbw is donating a large open dataset from the th century press archives to wikidata, in order to make it better accessible to various scientific disciplines such as contemporary, economic and business history, media and information science, to journalists, teachers, students, and the general public. the th century press archives (pm ) is a large public newspaper clippings archive, extracted from more than different sources published in germany and all over the world, covering roughly a full century ( - ). the clippings are organized in thematic folders about persons, companies and institutions, general subjects, and wares. during a project originally funded by the german research foundation (dfg), the material up to has been digitized. , folders with more than two million pages up to are freely accessible online. the fine-grained thematic access and the public nature of the archives makes it to our best knowledge unique across the world (more information on wikipedia) and an essential research data fund for some of the disciplines mentioned above. the data donation does not only mean that zbw has assigned a cc license to all pm metadata, which makes it compatible with wikidata. (due to intellectual property rights, only the metadata can be licensed by zbw - all legal rights on the press articles themselves remain with their original creators.) the donation also includes investing a substantial amount of working time (during, as planned, two years) devoted to the integration of this data into wikidata. here we want to share our experiences regarding the integration of the persons archive metadata. folders from the persons archive, in (credit: max-michael wannags) linking our folders to wikidatathe essential bit for linking the digitized folders was in place before the project even started: an external identifier property (pm folder id, p ), proposed by an administrator of the german wikipedia in order to link to pm person and company folders. we participated in the property proposal discussion and made sure that the links did not have to reference our legacy coldfusion application. instead, we created a "partial redirect" on the purl.org service (maintained formerly by oclc, now by the internet archive) for persistent urls which may redirect to another application on another server in future. secondly, the identifier and url format was extended to include subject and ware folders, which are defined by a combination of two keys, one for the country and another for the topic. the format of the links in wikidata is controlled by a regular expression, which covers all four archives mentioned above. that works pretty well - very few format errors occurred so far -, and it relieved us from creating four different archive-specific properties.shortly after the property creation, magnus manske, the author of the original mediawiki software and lots of related tools, scraped our web site and created a mix-n-match catalog from it. during the following two years, more than wikidata users contributed to matching wikidata items for humans to pm folder ids. for a start, deriving links from gnd many of the pm person and company folders were already identified by an identifier from the german integrated authority file (gnd). so, our first step was creating pm links for all wikidata items which had matching gnd ids. for all these items and folders, disambiguation had already taken place, and we could safely add all these links automatically. infrastructure: pm endpoint, federated queries and quickstatements to make this work, we relied heavily on linked data technologies. a pm sparql endpoint had already been set up for our contribution to coding da vinci (a "kultur-hackathon" in germany). almost all automated changes to wikidata we made are based on federated queries on our own endpoint, reaching out to the wikidata endpoint, or vice versa, from wikidata to pm . in the latter case, the external endpoint has to be registered at wikidata. wikidata maintains a help page for this type of queries. for our purposes, federated queries allow extracting current data from both endpoints. in the case of the above-mentioned missing_pm _id_via_gnd.rq query, this way we can skip all items, where a link to pm already exists. within the query itself, we create a statement string which we can feed into the quickstatements tool. that includes, for every single statement, a reference to pm with link to the actual folder, so that the provenance of these statements is always clear and traceable. via script, a statement file is extracted and saved with a timestamp. data imports via quickstatements are executed in batch mode, and an activity log keeps track of all data imports and other activities related to pm . creating missing items after the matching of about % of the person folders which include free documents in mix-n-match, and some efforts to discover more pre-existing wikidata items, we decided to create the missing person items, again via quickstatements input. we used the description field in wikidata by importing the content of the free-text "occupation" field in pm for better disambiguation of the newly created items. (here a rather minimal example of such an item created from pm metadata.) thus, all pm person folders which have digitized content were linked to wikidata in june . supplementing wikidata with pm metadata a second part of the integration of pm metadata into wikidata was the import of missing property values to the according items. this comprised simple facts like "date of birth/death", occupations such as "economist", "business economist", "social scientist", "earth scientist", which we could derive from the "field of activity" in pm , up to relations between existing items, e.g. a family member to the according family, or a board member to the according company. a few other source properties have been postponed, because alternative solutions exist, and the best one may depend on the intended use in future applications. the steps of this enrichment process and links to the code used - including the automatic generation of references - are online, too. complex statement added to wikidata item for friedrich krupp ag again, we used federated queries. often the target of a wikidata property is an item in itself. sometimes, we could directly get this via the target item's pm folder id (families, companies); sometimes we had to create lookup tables. for the latter, we used "values" clauses in the query (in case of "occupation"), or (in case of "country of citizenship"), we have to match countries from our internal classification in advance - a process for which we use openrefine. other than pm folder ids, which we avoided adding when folders do not contain digitized content, we added the metadata to all items which were linked to pm , and intend to repeat this process periodically when more items (e.g., companies) are identified by pm folder ids. in some housekeeping activity, we also add periodically the numbers of documents (online and total) and the exact folder names as qualifiers to newly emerging pm links in items. results of the data donation so far with all persons folder with digitized documents linked to wikidata, the data donation of the person folders metadata is completed. besides the folder links, which have already heavily been used to create links in wikipedia articles, we have got - more than statements which are sourced in pm (from "date of birth" to the track gauge of a brazilian railway line) - more than items, for which pm id is the only external identifier the data donation will be presented on the wikidatacon in berlin ( .- . . ) as a "birthday present" on the occasion wikidata's seventh birthday. zbw will further keep the digital content available, amended with a static landing page for every folder, which also will serve as source link for the metadata we have integrated into wikidata. but in future, wikidata will be the primary access path to our data, providing further metadata in multiple languages and links to a plethora of other external sources. and the best is, different from our current application, everybody will be able to enhance this open data through the interactive tools and data interfaces provided by wikidata.participate in wikiproject th century press archives for the topics, wares and companies archives, there is still a long way to go. the best structure for representing these archives and their folders - often defined by the combination of a country within a geographical hierarchy with a subject heading in a deeply nested topic classification -, has to be figured out. existing items have to be matched, and lots of other work is to be done. therefore, we have created the wikiproject th century press archives in wikidata to keep track of discussions and decisions, and to create a focal point for participation. everybody on wikidata is invited to participate - or just kibitz. it could be challenging particularly for information scientists, and people interested in historic systems for the organization of knowledge about the whole world, to take part in the mapping of one of these systems to the emerging wikidata knowledge graph. linked data &# ; open data &# ; zbw's contribution to "coding da vinci": dossiers about persons and companies from th century press archives at th and th of october, the kick-off for the "kultur-hackathon" coding da vinci is held in mainz, germany, organized this time by glam institutions from the rhein-main area: "for five weeks, devoted fans of culture and hacking alike will prototype, code and design to make open cultural data come alive." new software applications are enabled by free and open data. for the first time, zbw is among the data providers. it contributes the person and company dossiers of the th century press archive. for about a hundred years, the predecessor organizations of zbw in kiel and hamburg had collected press clippings, business reports and other material about a wide range of political, economic and social topics, about persons, organizations, wares, events and general subjects. during a project funded by the german research organization (dfg), the documents published up to (about , million pages) had been digitized and are made publicly accessible with according metadata, until recently solely in the "pressemappe . jahrhundert" (pm ) web application. additionally, the dossiers - for example about mahatma gandhi or the hamburg-bremer afrika linie - can be loaded into a web viewer. as a first step to open up this unique source of data for various communities, zbw has decided to put the complete pm metadata* under a cc-zero license, which allows free reuse in all contexts. for our coding da vinci contribution, we have prepared all person and company dossiers which already contain documents. the dossiers are interlinked among each other. controlled vocabularies (for, e.g., "country", or "field of activity") provide multi-dimensional access to the data. most of the persons and a good share of organizations were linked to gnd identifiers. as a starter, we had mapped dossiers to wikidata according to existing gnd ids. that allows to run queries for pm dossiers completely on wikidata, making use of all the good stuff there. an example query shows the birth places of pm economists on a map, enriched with images from wikimedia commons. the initial mapping was much extended by fantastic semi-automatic and manual mapping efforts by the wikidata community. so currently more than % of the dossiers about - often rather prominent - pm persons are linked not only to wikidata, but also connected to wikipedia pages. that offers great opportunities for mash-ups to further data sources, and we are looking forward to what the "coding da vinci" crowd may make out of these opportunities. technically, the data has been converted from an internal intermediate format to still quite experimental rdf and loaded into a sparql endpoint. there it was enriched with data from wikidata and extracted with a construct query. we have decided to transform it to json-ld for publication (following practices recommended by our hbz colleagues). so developers can use the data as "plain old json", with the plethora of web tools available for this, while linked data enthusiasts can utilize sophisticated semantic web tools by applying the provided json-ld context. in order to make the dataset discoverable and reusable for future research, we published it persistently at zenodo.org. with it, we provide examples and data documentation. a github repository gives you additional code examples and a way to address issues and suggestions. * for the scanned documents, the legal regulations apply - zbw cannot assign licenses here. pressemappe . jahrhundert linked data &# ; wikidata as authority linking hub: connecting repec and gnd researcher identifiers in the econbiz portal for publications in economics, we have data from different sources. in some of these sources, most notably zbw's "econis" bibliographical database, authors are disambiguated by identifiers of the integrated authority file (gnd) - in total more than , . data stemming from "research papers in economics" (repec) contains another identifier: repec authors can register themselves in the repec author service (ras), and claim their papers. this data is used for various rankings of authors and, indirectly, of institutions in economics, which provides a big incentive for authors - about , have signed into ras - to keep both their article claims and personal data up-to-date. while gnd is well known and linked to many other authorities, ras had no links to any other researcher identifier system. thus, until recently, the author identifiers were disconnected, which precludes the possibility to display all publications of an author on a portal page. to overcome that limitation, colleagues at zbw have matched a good , authors with ras and gnd ids by their publications (see details here). making that pre-existing mapping maintainable and extensible however would have meant to set up some custom editing interface, would have required storage and operating resources and wouldn't easily have been made publicly accessible. in a previous article, we described the opportunities offered by wikidata. now we made use of it. v\:* {behavior:url(#default#vml);} o\:* {behavior:url(#default#vml);} w\:* {behavior:url(#default#vml);} .shape {behavior:url(#default#vml);} normal false false false false de x-none x-none defsemihidden="true" defqformat="false" defpriority=" " latentstylecount=" "> unhidewhenused="false" qformat="true" name="normal"> unhidewhenused="false" qformat="true" name="heading "> unhidewhenused="false" qformat="true" name="title"> unhidewhenused="false" qformat="true" name="subtitle"> unhidewhenused="false" qformat="true" name="strong"> unhidewhenused="false" qformat="true" name="emphasis"> unhidewhenused="false" name="table grid"> unhidewhenused="false" qformat="true" name="no spacing"> unhidewhenused="false" name="light shading"> unhidewhenused="false" name="light list"> unhidewhenused="false" name="light grid"> unhidewhenused="false" name="medium shading "> unhidewhenused="false" name="medium shading "> unhidewhenused="false" name="medium list "> unhidewhenused="false" name="medium list "> unhidewhenused="false" name="medium grid "> unhidewhenused="false" name="medium grid "> unhidewhenused="false" name="medium grid "> unhidewhenused="false" name="dark list"> unhidewhenused="false" name="colorful shading"> unhidewhenused="false" name="colorful list"> unhidewhenused="false" name="colorful grid"> unhidewhenused="false" name="light shading accent "> unhidewhenused="false" name="light list accent "> unhidewhenused="false" name="light grid accent "> unhidewhenused="false" name="medium shading accent "> unhidewhenused="false" name="medium shading accent "> unhidewhenused="false" name="medium list accent "> unhidewhenused="false" qformat="true" name="list paragraph"> unhidewhenused="false" qformat="true" name="quote"> unhidewhenused="false" qformat="true" name="intense quote"> unhidewhenused="false" name="medium list accent "> unhidewhenused="false" name="medium grid accent "> unhidewhenused="false" name="medium grid accent "> unhidewhenused="false" name="medium grid accent "> unhidewhenused="false" name="dark list accent "> unhidewhenused="false" name="colorful shading accent "> unhidewhenused="false" name="colorful list accent "> unhidewhenused="false" name="colorful grid accent "> unhidewhenused="false" name="light shading accent "> unhidewhenused="false" name="light list accent "> unhidewhenused="false" name="light grid accent "> unhidewhenused="false" name="medium shading accent "> unhidewhenused="false" name="medium shading accent "> unhidewhenused="false" name="medium list accent "> unhidewhenused="false" name="medium list accent "> unhidewhenused="false" name="medium grid accent "> unhidewhenused="false" name="medium grid accent "> unhidewhenused="false" name="medium grid accent "> unhidewhenused="false" name="dark list accent "> unhidewhenused="false" name="colorful shading accent "> unhidewhenused="false" name="colorful list accent "> unhidewhenused="false" name="colorful grid accent "> unhidewhenused="false" name="light shading accent "> unhidewhenused="false" name="light list accent "> unhidewhenused="false" name="light grid accent "> unhidewhenused="false" name="medium shading accent "> unhidewhenused="false" name="medium shading accent "> unhidewhenused="false" name="medium list accent "> unhidewhenused="false" name="medium list accent "> unhidewhenused="false" name="medium grid accent "> unhidewhenused="false" name="medium grid accent "> unhidewhenused="false" name="medium grid accent "> unhidewhenused="false" name="dark list accent "> unhidewhenused="false" name="colorful shading accent "> unhidewhenused="false" name="colorful list accent "> unhidewhenused="false" name="colorful grid accent "> unhidewhenused="false" name="light shading accent "> unhidewhenused="false" name="light list accent "> unhidewhenused="false" name="light grid accent "> unhidewhenused="false" name="medium shading accent "> unhidewhenused="false" name="medium shading accent "> unhidewhenused="false" name="medium list accent "> unhidewhenused="false" name="medium list accent "> unhidewhenused="false" name="medium grid accent "> unhidewhenused="false" name="medium grid accent "> unhidewhenused="false" name="medium grid accent "> unhidewhenused="false" name="dark list accent "> unhidewhenused="false" name="colorful shading accent "> unhidewhenused="false" name="colorful list accent "> unhidewhenused="false" name="colorful grid accent "> unhidewhenused="false" name="light shading accent "> unhidewhenused="false" name="light list accent "> unhidewhenused="false" name="light grid accent "> unhidewhenused="false" name="medium shading accent "> unhidewhenused="false" name="medium shading accent "> unhidewhenused="false" name="medium list accent "> unhidewhenused="false" name="medium list accent "> unhidewhenused="false" name="medium grid accent "> unhidewhenused="false" name="medium grid accent "> unhidewhenused="false" name="medium grid accent "> unhidewhenused="false" name="dark list accent "> unhidewhenused="false" name="colorful shading accent "> unhidewhenused="false" name="colorful list accent "> unhidewhenused="false" name="colorful grid accent "> unhidewhenused="false" name="light shading accent "> unhidewhenused="false" name="light list accent "> unhidewhenused="false" name="light grid accent "> unhidewhenused="false" name="medium shading accent "> unhidewhenused="false" name="medium shading accent "> unhidewhenused="false" name="medium list accent "> unhidewhenused="false" name="medium list accent "> unhidewhenused="false" name="medium grid accent "> unhidewhenused="false" name="medium grid accent "> unhidewhenused="false" name="medium grid accent "> unhidewhenused="false" name="dark list accent "> unhidewhenused="false" name="colorful shading accent "> unhidewhenused="false" name="colorful list accent "> unhidewhenused="false" name="colorful grid accent "> unhidewhenused="false" qformat="true" name="subtle emphasis"> unhidewhenused="false" qformat="true" name="intense emphasis"> unhidewhenused="false" qformat="true" name="subtle reference"> unhidewhenused="false" qformat="true" name="intense reference"> unhidewhenused="false" qformat="true" name="book title"> /* style definitions */ table.msonormaltable {mso-style-name:"normale tabelle"; mso-tstyle-rowband-size: ; mso-tstyle-colband-size: ; mso-style-noshow:yes; mso-style-priority: ; mso-style-parent:""; mso-padding-alt: cm . pt cm . pt; mso-para-margin: cm; mso-para-margin-bottom:. pt; line-height: . pt; mso-pagination:widow-orphan; font-size: . pt; font-family:"times new roman","serif";} initial situation in wikidata economists were, at the start of this small project in april , already well represented among the . million persons in wikidata - though the precise extent is difficult to estimate. furthermore, properties for linking gnd and repec author identifiers to wikidata items were already in place: p “gnd id”, in ~ , items p “repec short-id” (further-on: ras id), in ~ , items both properties in ~ items for both properties, “single value” and “distinct values” constraints are defined, so that (with rare exceptions) a : relation between the authority entry and the wikidata item should exist. that, in turn, means that a : relation between both authority entries can be assumed. the relative amounts of ids in econbiz and wikidata is illustrated by the following image. person identifiers in wikidata and econbiz, with unknown overlap at the beginning of the project (the number of . million persons in econbiz is a very rough estimate, because most names – outside gnd and ras – are not disambiguated) since many economists have wikipedia pages, from which wikidata items have been created routinely, the first task was finding these items and adding gnd and/or ras identifiers to them. the second task was adding items for persons which did not already exist in wikidata. adding mapping-derived identifiers to wikidata items for items already identified by either gnd or ras, the reciprocal identifiers where added automatically: a federated sparql query on the mapping and the public wikidata endpoint retrieved the items and the missing ids. a script transformed that into input for wikidata’s quickstatements tool, which allows adding statements (as well as new items) to wikidata. the tool takes csv-formatted input via a web form and applies it in batch to the live dataset. import statements for quickstatements . the first input line adds the ras id “pan ” to the item for the economist james andreoni. the rest of the input line creates a reference to zbws mapping for this statement and so allows tracking its provenance in wikidata. that step resulted in added gnd ids to items identified by ras id, and, in the reverse direction, added ras ids to items identified by gnd id. for the future, it is expected that tools like wdmapper will facilitate such operations. identifying more wikidata items obviously, the previous step left out the already existing economists in wikidata, which up to then had neither a gnd nor a ras id. therefore, these items had to be identified by adding one of the identifiers. a semi-automatic approach was applied to that end, starting with the “most important” persons from repec and econbiz datasets. that was extended in an automatic step, taking advantage of existing viaf identifiers (a step which could have been also the first one). for repec, the “top economists” ranking page (~ , authors) was scraped and cross-linked to a custom-created basic rdf dataset of the repec authors. the result was transformed to an input file for wikidata’s mix’n’match tool, which had been developed for the alignment of external catalogs with wikidata. the tool takes a simple csv file, consisting of a name, a description and an identifier, and tries to automatically match against wikidata labels. in a subsequent interactive step, it allows to confirm or remove every match. if confirmed, the identifier is automatically added as value to the according property of the matched wikidata item. for gnd, all authors with more than publications in econbiz where selected in a custom sparql endpoint. just as the “repec top” matchset, a “gnd economists (de)” matchset with ~ , gnd ids, names and descriptions was loaded into mix’n’match and aligned to wikidata. becoming more familiar with the wikidata-related tools, policies and procedures, existing viaf property values were exploited as another opportunity for seeding gnd ids in wikidata. in a federated sparql query on a custom viaf and the public wikidata endpoint, about , missing gnd ids were determined and added to wikidata items which had been identified by viaf id. after each of these steps, the first task – adding mapping-derived gnd or ras identifiers – was repeated. that resulted in wikidata items carrying both ids. since zbws author mapping based on at least matching publications, the alignment of high-frequency resp. highly-ranked gnd and repec authors made it highly probable that authors already present in wikidata were identified in the previous steps. that reduced the danger of creating duplicates in the following task. creating new wikidata items from the mapped authorities for the rest of the authors in the mapping, new wikidata items were created. this task was carried out again by the quickstatements tool, for which the input statements were created by a script, based on a sparql query on the afore-mentioned endpoints for repec authors and gnd entries. the input statements were derived from both authorities, in the following fashion: the label (name of the person) was taken from gnd the occupation “economist” was derived from repec (and in particular from the occurrence in its “top economists” list) gender and date of birth/death were taken from gnd (if available) the english description was a concatenated string “economist” plus the affiliations from repec the german description was a concatenated string “wirtschaftswissenschaftler/in” plus the affiliations from gnd the use of wikidata’s description field for affiliations was a makeshift: in the absence of an existing mapping of repec (and mostly also gnd) organizations to wikidata, it allows for better identification of the individual researchers. in a later step, when according organization/institute items exist in wikidata and mappings are in place, the items for authors can be supplemented step-by-step by formal “affiliation” (p ) statements. according to wikidata’s policy, an extensive reference to the source for each statement in the synthesized new wikidata item was added. the creation of items in an automated fashion involves the danger of duplicates. however, such duplicates turned up only in very few cases. they have been solved by merging items, which technically is very easy in wikidata. interestingly, a number of “fake duplicates” indeed revealed multifarious quality issues, in wikidata and in both of the authority files, which, too, have been subsequently resolved. ... and even more new items for economists ... the good experiences so far let us get bolder, and we considered creating wikidata items for the still missing "top economists" (according to repec). for item creation, one aspect we had to consider was the compliance with wikidata's notability policy. this policy is much more relaxed than the policies of the large wikipedias. it states as one criterion sufficient for item creation that the item "refers to an instance of a clearly identifiable conceptual or material entity. the entity must be notable, in the sense that it can be described using serious and publicly available references." there seems to be some consensus in the community that authority files such as gnd or repec authors count as "serious and publicly available references". this of course should hold even more for a bibliometric ranked subset of these external identifiers. we thus inserted another , wikidata items for the rest of the repec top % list. additionally - to mitigate the immanent gender bias such selections often bear - we imported all missing researchers from repec's "top % female economists" list. again, we added reference statements to repec which allow wikidata users to keep track of the source of the information. results the immediate result of the project was: all of the pairs of identifiers from the initial mapping by zbw is incorporated now in wikidata items wikidata items in addition to these also have both identifiers (created by individual wikidata editors, or the efforts described above) (all numbers in this section as of - - .) while that still is only a beginning, given the total amount of authors represented in econbiz, it is a significant share of the “most important” ones: top % ras and frequent gnd in econbiz (> publications). “wikidata economists” is a rough estimate of the amount of persons in the field of economics (twice the number of those with the explicit occupation “economist”) while the top repec economists are now completely covered by wikidata, for gnd the overlap has been improved significantly during the last year. this occured in parts as a side-effect of the efforts described above, in parts it is caused by the genuine growth of wikidata in regard to the number of items as well as the increasing density of external identifiers. here the current percentages, compared to those one year earlier, which were presented in our previous article: large improvements in the coverage of the most frequent authors by wikidata (query, result) while the improvements in absolute numbers are impressive, too - the number of gnd ids for all econbiz persons (with at least one publication) has increased from , to , - the image demonstrates that particularly the coverage for our most frequent authors has risen largely. the addition of all repec top economists has created further opportunities for matching these items from the afore-mentioned gnd mix-n-match set, which will again will add up to the mapping. all matching and duplicates checking done, we may re-consider the option of adding the remaining frequent gnd persons (> publications in econbiz) automatically to wikidata. the mapping data can be retrieved by everyone, via sparql queries, by specialized tools such as wdmapper, or as part of the wikidata dumps. what is more, it can be extended by everybody – either as a by-product of individual edits adding identifiers to persons in wikidata, or by a directed approach. for directed extensions, any subset can be used as a starting point: either a new version of the above mentioned ranking, or other rankings also published by repec, covering in particular female, or economists from e.g. latin america; or all identifiers from a particular institution, either derived from gnd or ras. the results of all such efforts are available at once and add up continuously. yet, the benefits of using wikidata cannot be reduced to the publication and maintenance of mapping itself. in many cases it offers much more than just a linking point for two identifiers: links to wikipedia pages about the authors, possibly in multiple languages rich data about the authors in defined formats, sometimes with explicit provenance information access to pictures etc. from wikimedia commons, or quotations from wikiquote links to multiple other authorities as an example for the latter, the in total ras identifiers in wikidata are already mapped to viaf and loc authority ids (while orcid with ids is still remarkably low). at the same time, these repec-connected items were linked to english, german and spanish wikipedia pages which provide rich human-readable information. in turn, when we take the gnd persons in econbiz as a starting point, roughly , are already represented in wikidata. besides large amounts of other identifiers, the according wikidata items offer more than , links to german and more than , links to english wikipedia pages (query). for zbw, “releasing” the dataset into wikidata as a trustworthy and sustainable public database not only saves the “technical” costs of data ownership (programming, storage, operating, for access and for maintenance). responsibility for - and fun from - extending, amending and keeping the dataset current can be shared with many other interested parties and individuals. wikidata for authorities authority control &# ; wikidata &# ; deutsch new version of multi-lingual jel classification published in lod the journal of economic literature classification scheme (jel) was created and is maintained by the american economic association. the aea provides this widely used resource freely for scholarly purposes. thanks to andré davids (ku leuven), who has translated the originally english-only labels of the classification to french, spanish and german, we provide a multi-lingual version of jel. it's lastest version (as of - ) is published in the formats rdfa and rdf download files. these formats and translations are provided "as is" and are not authorized by aea. in order to make changes in jel tracable more easily, we have created lists of inserted and removed jel classes in the context of the skos-history project. jel klassifikation für linked open dataskos-history linked data &# ; economists in wikidata: opportunities of authority linking wikidata is a large database, which connects all of the roughly wikipedia projects. besides interlinking all wikipedia pages in different languages about a specific item – e.g., a person -, it also connects to more than different sources of authority information. the linking is achieved by a „authority control“ class of wikidata properties. the values of these properties are identifiers, which unambiguously identify the wikidata item in external, web-accessible databases. the property definitions includes an uri pattern (called „formatter url“). when the identifier value is inserted into the uri pattern, the resulting uri can be used to look up the authoritiy entry. the resulting uri may point to a linked data resource - as it is the case with the gnd id property. this, on the one hand, provides a light-weight and robust mechanism to create links in the web of data. on the other hand, these links can be exploited by every application which is driven by one of the authorities to provide additional data: links to wikipedia pages in multiple languages, images, life data, nationality and affiliations of the according persons, and much more. wikidata item for the indian economist bina agarwal, visualized via the sqid browser in , a group of students under the guidance of jakob voß published a handbook on "normdaten in wikidata" (in german), describing the structures and the practical editing capabilities of the the standard wikidata user interface. the experiment described here focuses on persons from the subject domain of economics. it uses the authority identifiers of the about , economists referenced by their gnd id as creators, contributors or subjects of books, articles and working papers in zbw's economics search portal econbiz. these gnd ids were obtained from a prototype of the upcoming econbiz research dataset (ebds). to , of these persons, or . %, a person in wikidata is connected by gnd. if we consider the frequent (more than publications) and the very frequent (more than publications) authors in econbiz, the coverage increases significantly: economics-related persons in econbiz number of publications total in wikidata percentage datasets: ebds as of - - ; wikidata as of - - (query, result) > , , . % > , , . % > , . % these are numbers "out of the box" - ready-made opportunities to link out from existing metadata in econbiz and to enrich user interfaces with biographical data from wikidata/wikipedia, without any additional effort to improve the coverage on either the econbiz or the wikidata side. however: we can safely assume that many of the econbiz authors, particularly of the high-frequency authors, and even more of the persons who are subject of publications, are "notable" according the wikidata notablitiy guidelines. probably, their items exist and are just missing the according gnd property. to check this assumption, we take a closer look to the wikidata persons which have the occupation "economist" (most wikidata properties accept other wikidata items - instead of arbitrary strings - as values, which allows for exact queries and is indispensible in a multilingual environment). of these approximately , persons, less than % have a gnd id property! even if we restrict that to the , "internationally recognized economists" (which we define here as having wikipedia pages in three or more different languages), almost half of them lack a gnd id property. when we compare that with the coverage by viaf ids, more than % of all and % the internationally recognized wikidata economists are linked to viaf (sparql lab live query). therefore, for a whole lot of the persons we have looked at here, we can take it for granted the person exists in wikidata as well as in the gnd, and the only reason for the lack of a gnd id is that nobody has added it to wikidata yet. as an aside: the information about the occupation of persons is to be taken as a very rough approximation: some wikidata persons were economists by education or at some point of their career, but are famous now for other reasons (examples include vladimir putin or the president of liberia, ellen johnson sirleaf). on the other hand, econbiz authors known to wikidata are often qualified not as economist, but as university teacher, politican, historican or sociologist. nevertheless, their work was deemed relevant for the broad field of economics, and the conclusions drawn at the "economists" in wikidata and gnd will hold for them, too: there are lots of opportunities for linking already well defined items. what can we gain? the screenshot above demonstrates, that not only data about the person itself, her affiliations, awards received, and possibly many other details can be obtained. the "identifiers" box on the bottom right shows authoritiy entries. besides the gnd id, which served as an entry point for us, there are links to viaf and other national libraries' authorities, but also to non-library identifier systems like isni and orcid. in total, wikidata comprises more than million authority links, more than millions of these for persons. when we take a closer look at the , econbiz persons which we can look up by their gnd id in wikidata, an astonishing variety of authorities is addressed from there: different authorities are linked from the subset, ranging from "almost complete" (viaf, library of congress name authority file) to - in the given context- quite exotic authorities of, e.g., members of the belgian senate, chess players or swedish olympic committee athletes. some of these entries link to carefully crafted biographies, sometimes behind a paywall (notable names database, oxford dictionary of national biography, munzinger archiv, sächsische biographie, dizionario biografico degli italiani), or to free text resources (project gutenberg authors). links to the world of museums and archives are also provided, from the getty union list of artist names to specific links into the british museum or the musée d'orsay collections. a particular use can be made of properties which express the prominence of the according persons: nobel prize ids, for example, definitivly should be linked to according gnd ids (and indeed, they are). but also ted speakers or persons with an entry in the munzinger archive (a famous and long-established german biographical service) are assumed to have gnd ids. that opens a road to a very focused improvement of the data quality: a list of persons with that properties, restricted to the subject field (e.g., "occupation economist"), can be easily generated from wikidata's sparql query service. in wikidata, it is very easy to add the missing id entries discovered during such cross-checks interactively. and if it turns out that an "very important" person from the field is missing from the gnd at all, that is a all-the-more valuable opportunity to improve the data quality at the source. how can we start improving? as a prove of concept, and as a practical starting point, we have developed a micro-application for adding missing authority property values. it consists of two sparql lab scripts: missing_property creates a list of wikidata persons, which have a certain authority property (by default: ted speaker id) and lacks another one (by default: gnd id). for each entry in the list, a link to an application is created, which looks up the name in the according authority file (by default: search_person, for a broad yet ranked full-text search of person names in gnd). if we can identify the person in the gnd list, we can copy its gnd id, return to the first one, click on the link to the wikidata item of the person and add the property value manually through wikidata's standard edit interface. (wikidata is open and welcoming such contributions!) it takes effect within a few seconds - when we reload the missing_property list, the improved item should not show up any more. instead of identifying the most prominent economics-related persons in wikidata, the other way works too: while most of the gnd-identified persons are related to only one or twe works, as an according statistics show, few are related to a disproportionate amount of publications. of the , persons related to more than publications, less than are missing links to wikidata by their gnd id. by adding this property (for the vast majority of these persons, a wikidata item should already exist), we could enrich, at a rough estimate, more than , person links in econbiz publications. another micro-application demonstrates, how the work could be organized: the list of econbiz persons by descending publication count provides "search in wikidata" links (functional on a custom endpoint): each link triggers a query which looks up all name variants in gnd and executes a search for these names in a full-text indexed wikidata set, bringing up an according ranked list of suggestions (example with the gnd id of john h. dunning). again, the gnd id can be added - manually but straightforward - to an identified wikidata item. while we can not expect to reduce the quantitative gap between the , persons in econbiz and the , of them linked to wikidata significantly by such manual efforts, we surely can step-by-step improve for the most prominent persons. this empowers applications to show biographical background links to wikipedia where our users expect them most probably. other tools for creating authority links and more automated approaches will be covered in further blog posts. and the great thing about wikidata is: all efforts add up - while we are doing modest improvements in our field of interest, many others do the same, so wikidata already features an impressive overall amont of authority links. ps. all queries used in this analysis are published at github. the public wikidata endpoint cannot be used for research involving large datasets due to its limitations (in particular the second timeout, the preclusion of the "service" clause for federated queries, and the lack of full-text search). therefore, we’ve loaded the wikidata dataset (along with others) into custom apache fuseki endpoints on a performant machine. even there, a „power query“ like the one on the number of all authority links in wikidata takes about minutes. therefore, we publish the according result files in the github repository alongside with the queries. wikidata for authorities wikidata &# ; authority control &# ; linked data &# ; integrating a research data repository with established research practices authors: timo borst, konstantin ott in recent years, repositories for managing research data have emerged, which are supposed to help researchers to upload, describe, distribute and share their data. to promote and foster the distribution of research data in the light of paradigms like open science and open access, these repositories are normally implemented and hosted as stand-alone applications, meaning that they offer a web interface for manually uploading the data, and a presentation interface for browsing, searching and accessing the data. sometimes, the first component (interface for uploading the data) is substituted or complemented by a submission interface from another application. e.g., in dataverse or in ckan data is submitted from remote third-party applications by means of data deposit apis [ ]. however the upload of data is organized and eventually embedded into a publishing framework (data either as a supplement of a journal article, or as a stand-alone research output subject to review and release as part of a ‘data journal’), it definitely means that this data is supposed to be made publicly available, which is often reflected by policies and guidelines for data deposit. in clear contrast to this publishing model, the vast majority of current research data however is not supposed to be published, at least in terms of scientific publications. several studies and surveys on research data management indicate that at least in the social sciences there is a strong tendency and practice to process and share data amongst peers in a local and protected environment (often with several local copies on different personal devices), before eventually uploading and disseminating derivatives from this data to a publicly accessible repository. e.g., according to a survey among austrian researchers, the portion of researchers agreeing to share their data either on request or among colleagues is % resp. %, while the agreement to share on a disciplinary repository is only % [ ]. and in another survey among researchers from a local university and cooperation partner, almost % preferred an institutional local archive, while only % agreed on a national or international archive. even if there is data planned to be published via a publicly accessible repository, it will first be stored and processed in a protected environment, carefully shared with peers (project members, institutional colleagues, sponsors) and often subject to access restrictions – in other words, it is used before being published.with this situation in mind, we designed and developed a central research data repository as part of a funded project called ‘sowidatanet’ (sdn - network of data from social sciences and economics) [ ]. the overall goal of the project is to develop and establish a national web infrastructure for archiving and managing research data in the social sciences, particularly quantitative (statistical) data from surveys. it aims at smaller institutional research groups or teams, which often do lack an institutional support or infrastructure for managing their research data. as a front-end application, the repository based on dspace software provides a typical web interface for browsing, searching and accessing the content. as a back-end application, it provides typical forms for capturing metadata and bitstreams, with some enhancements regarding the integration of authority control by means of external webservices. from the point of view of the participating research institutions, a central requirement is the development of a local view (‘showcase’) on the repository’s data, so that this view can be smoothly integrated into the website of the institution. the web interface of the view is generated by means of the play framework in combination with the bootstrap framework for generating the layout, while all of the data is retrieved and requested from the dspace backend via its discover interface and rest-api. sdn architecturediagram: sowidatanet software componentsthe purpose of the showcase application is to provide an institutional subset and view of the central repository’s data, which can easily be integrated into any institutional website, either as an iframe to be embedded by the institution (which might be considered as an easy rather than a satisfactory technical solution), or as a stand-alone subpage being linked from the institution’s homepage, optionally using a proxy server for preserving the institutional domain namespace. while these solutions imply the standard way of hosting the showcase software, a third approach suggests the deployment of the showcase software on an institution’s server for customizing the application. in this case, every institution can modify the layout of their institutional view by customizing their institutional css file. because using bootstrap and less compiling the css file, a lightweight possibility might be to modify only some less variables compiling to an institutional css file.as a result from the requirement analysis conducted with the project partners (two research institutes from the social sciences), and in accordance with the survey results cited, there is a strong demand for managing not only data which is to be published in the central repository, but also data which is protected and circulating only among the members of the institution. moreover, this data is described by additional specific metadata containing internal hints on the availability restrictions and access conditions. hence, we had to distinguish between the following two basic use cases to be covered by the showcase: to provide a view on the public sdn data (‘data published’) to provide a view on the public sdn data plus the internal institutional data resp. their corresponding metadata records, the latter only visible and accessible for institutional members (‘data in use’) from the perspective of a research institution and data provider, the second use case turned out to be the primary one, since it covers more the institutional practices and workflows than the publishing model does. as a matter of fact, research data is primarily generated, processed and shared in a protected environment, before it may eventually be published and distributed to a wider, potentially abstract and unknown community – and this fact must be acknowledged and reflected by a central research data repository aiming at the contributions from researchers which are bound to an institution.if ‘data in use’ is to be integrated into the showcase as an internal view on protected data to be shared only within an institution, it means to restrict the access to this data on different levels. first, for every community (in the sense of an institution), we introduce a dspace collection for just those internal data, and protect it by assigning it to a dspace user role ‘internal[community_name]’. this role is associated with an ip range, so that only requests from that range will be assigned to the role ‘internal’ and granted access to the internal collection. in the context of our project, we enter only the ip of the showcase application, so that every user of this application will see the protected items. depending on the locality of the showcase application resp. server, we have to take further steps: if the application resp. server is located in the institution’s intranet, the protected items are only visible and accessible from the institution’s network. if the application is externally hosted and accessible via the world wide web – which is expected to be the default solution for most of the research institutes –, then the showcase application needs an authentication procedure, which is preferably realized by means of the central dspace sowidatanet repository, so that every user of the showcase application is granted access by becoming a dspace user.in the context of an r&d project where we are partnering with research institutes, it turned out that the management of research data is twofold: while repository providers are focused on the publishing and unrestricted access to research data, researchers are mainly interested in local archiving and sharing of their data. in order to manage this data, the researchers’ institutional practices need to be reflected and supported. for this purpose, we developed an additional viewing and access component. when it comes to their integration with existing institutional research practices and workflows, the implementation of research data repositories requires concepts and actions which go far beyond the original idea of a central publishing platform. further research and development is planned in order to understand and support better the sharing of data in both institutional and cross-institutional subgroups, so the integration with a public central repository will be fostered.link to prototype references[ ] dataverse deposit-api. retrieved may , from http://guides.dataverse.org/en/ . . /dataverse-api-main.html#data-deposit-api[ ] forschende und ihre daten. ergebnisse einer österreichweiten befragung – report . version . - zenodo. ( ). retrieved may , from https://zenodo.org/record/ #.vrhmkea pmm[ ] project homepage: https://sowidatanet.de/. retrieved may .[ ] research data management survey: report - nottingham eprints. ( ). retrieved may , from http://eprints.nottingham.ac.uk/ /[ ] university of oxford research data management survey  : the results | damaro. ( ). retrieved may , from https://blogs.it.ox.ac.uk/damaro/ / / /university-of-oxford-research-data-management-survey- -the-results/ institutional view on research data content recommendation by means of eexcess authors: timo borst, nils witt since their beginnings, libraries and related cultural institutions were confident in the fact that users had to visit them in order to search, find and access their content. with the emergence and massive use of the world wide web and associated tools and technologies, this situation has drastically changed: if those institutions still want their content to be found and used, they must adapt themselves to those environments in which users expect digital content to be available. against this background, the general approach of the eexcess project is to ‘inject’ digital content (both metadata and object files) into users' daily environments like browsers, authoring environments like content management systems or google docs, or e-learning environments. content is not just provided, but recommended by means of an organizational and technical framework of distributed partner recommenders and user profiles. once a content partner has connected to this framework by establishing an application program interface (api) for constantly responding to the eexcess queries, the results will be listed and merged with the results of the other partners. depending on the software component installed either on a user’s local machine or on an application server, the list of recommendations is displayed in different ways: from a classical, text-oriented list, to a visualization of metadata records. the recommender the eexcess architecture comprises three major components: a privacy-preserving proxy, multiple client-side tools for the chrome browser, wordpress, google docs and more, and the central server-side component, responsible for generating recommendations, called recommender. covering all of these components in detail is beyond the scope of this blog post. instead, we want to focus on one component: the federated recommender, as it is the heart of the eexcess infrastructure.the recommender’s task is to generate a list of objects like text documents, images and videos (hereafter called documents, for brevity’s sake) in response to a given query. the list is supposed to contain only documents relevant to the user. moreover, the list should be ordered (by descending relevance). to generate such a list, the recommender can pick documents from the content providers that participate in the eexcess infrastructure. technically speaking but somewhat oversimplified: the recommender receives a query and forwards it to all content provider systems (like econbiz, europeana, mendeley and others). after receiving results from each content provider, the recommender decides in which order documents will be recommended to the user and return it to the user who submitted the query.this raises some questions. how can we find relevant documents? the result lists from the content providers are already sorted by relevance; how can we merge them? can we deal with ambiguity and duplicates? can we respond within reasonable time? can we handle the technical disparities of the different content provider systems? how can we integrate the different document types? in the following, we will describe how we tackled some of these questions, by giving a more detailed explanation on how the recommender compiles the recommendation lists. recommendation process if the user wishes to obtain personalized recommendations, she can create a local profile (i.e. stored only on the user’s device). they can specify their education, age, field of interest and location. but to be clear here: this is optional. if the profile is used, the privacy proxy[ ] takes care of anonymizing the personal information. the overall process of creating personalized recommendations is depicted in figure and will be described in the following. after the user has sent a query as well as her user profile, a process called source selection is triggered. based on the user’s preferences, the source selection decides which partner systems will be queried. the reason for this is that most content providers cover only a specific discipline (see figure). for instance, queries from a user that is only interested in biology and chemistry will never receive econbiz recommendations, whereas a query from a user merely interested in politics and money will get econbiz recommendations (up to the present, this may change when other content provider participate). thereby, source selection lowers the network traffic and the latency of the overall process and increases the precision of the results at the expense of missing results and reduced diversity. optionally, the user can also select the sources manually. the subsequent query processing step alters the query: short queries are expanded using wikipedia knowledge long queries are split into smaller queries, which are then handled separately (see [ ] for more details). the queries from the query processing step are then used to query the content providers selected during the source selection step. with the results from the content providers, two post processing steps are carried out to generate the personalized recommendations: result processing: the purpose of the result processing is to detect duplicates. a technique called fuzzy hashing is used for this purpose. the words that make up a result list’s entry are sorted, counted and truncated by the md hash algorithm [ ], which allows convenient comparison. result ranking: after the duplicates have been removed, the results are re-ranked. to do so, a slightly modified version of the round robin method is used. where vanilla round robin would just concatenate slices of the result lists (i.e. first two documents from list a + first two document from list b + …), weighted round robinmodifies this behavior by taking the overlap of the query and the result’s meta-data into account. this is, before merging the lists, each individual list is modified. documents, whose meta data exhibit a high accordance to the query, are being promoted. partner wizard as the quality of the recommended documents increases with the number and diversity of the content providers that participate, a component called partner wizard was implemented. its goal is to simplify the integration of new content providers to a level that non-experts can manage this process without any support from the eexcess consortium. this is achieved by a semi-automatic process triggered from a web frontend that is provided by the eexcess consortium. given a search api, it is relatively easy to obtain search results, but the main point is to obtain results that are meaningful and relevant to the user. since every search service behaves differently, there is no point in treating all services equally. some sort of customization is needed. that’s where the partner wizard comes into play. it allows an employee from the new content provider to specify the search api. afterwards, the wizard submits pre-assembled pairs of search queries to the new service. each pair is similar but not identical, like for examp query : <term > or <term > query : <term > and <term >. the thereby generated result lists are presented to the user, which has to decide which list contains the more relevant results and suits the query better (see figure). finally, based on the previous steps, a configuration file is generated that configures the federated recommender. whereupon the recommender mimics the behavior, that was previously exhibited. the wizard can be completed within a few minutes and it only requires a publically available search api. the project started with five initial content providers. now, due to the contribution of the partner wizard, there are more than ten content providers and negotiations with further candidates are ongoing. since there are almost no technical issues anymore, legal issues dominate the consultations. as all programs developed within the eexcess project are published under open source conditions, the partner wizard can be found at [ ]. conclusions the eexcess project is about injecting distributed content from different cultural and scientific domains into everyday user environments, so this content becomes more visible and better accessible. to achieve this goal and to establish a network of distributed content providers, apart from the various organizational, conceptual and legal aspects some specification and engineering of software is to be done – not only one-time, but also with respect to maintaining the technical components. one of the main goals of the project is to establish a community of networked information systems, with a lightweight approach towards joining this network by easily setting up a local partner recommender. future work will focus on this growing network and the increasing requirements of integrating heterogeneous content via central processing of recommendations. eexcess recommender recommender system &# ; metadata &# ; economics &# ; search for full functionality of this site it is necessary to enable javascript. here are the instructions how to enable javascript in your web browser. perspectives: improving ssh-style host authentication with multi-path probing internet alchemy, the blog of ian davis internet alchemy est. · · · · · · · · · · · · · mon, oct , serverless: why microfunctions > microservices this post follows on from a post i wrote a couple of years back called why service architectures should focus on workflows. in that post i attempted to describe the fragility of microservice systems that were simply translating object-oriented patterns to the new paradigm. these systems were migrating domain models and their interactions from in-memory objects to separate networked processes. they were replacing in-process function calls with cross-network rpc calls, adding latency and infrastructure complexity. the goal was scalability and flexibility but, i argued, the entity modelling approach introduced new failure modes. i suggested a solution: instead of carving up the domain by entity, focus on the workflows. if i was writing that post today i would say “focus on the functions” because the future is serverless functions, not microservices. or, more brashly: microfunctions > microservices the industry has moved apace in the last years with a focus on solving the infrastructure challenges caused by running hundreds of intercommunicating microservices. containers have matured and become the de-facto standard for the unit of microservice deployment with management platforms such as kubernetes to orchestrate them and frameworks like grpc for robust interservice communication. the focus still tends to be on interacting entities though: when placing an order the “order service” talks to the “customer service” which reserves items by talking to the “stock service” and the “payment service” which talks to the “payment gateway” after first checking with the “fraud service”. when the order needs to be shipped the “shipping service” asks the “order service” for orders that need to be fulfilled and tells the “stock service” to remove the reservation, then to the “customer service” to locate the customer etc. all of these services are likely to be persisting state in various backend databases. microservices are organized as vertical slices through the domain: the same problems still exist: if the customer service is overwhelmed by the shipping service then the order service can’t take new orders. the container manager will, of course, scale up the number of customer service instances and register them with the appropriate load balancers, discovery servers, monitoring and logging. however, it cannot easily cope with a critical failure in this service, perhaps caused by a repeated bad request that panics the service and prevents multiple dependent services from operating properly. failures and slowdowns in response times are handled within client services through backoff strategies, circuit breakers and retries. the system as a whole increases in complexity but remains fragile. by contrast, in a serverless architecture, the emphasis is on the functions of the system. for this reason serverless is sometimes called faas – functions as a service. systems are decomposed into functions that encapsulate a single task in a single process. instead of each request involving the orchestration of multiple services the request uses an instance of the appropriate function. rather than the domain model being exploded into separate networked processes its entities are provided in code libraries compiled into the function at build time. calls to entity methods are in-process so don’t pay the network latency or reliability taxes. in this paradigm the “place order” function simply calls methods on customer, stock and payment objects, which may then interact with the various backend databases directly. instead of a dozen networked rpc calls, the function relies on - database calls. additionally, if a function is particularly hot it can be scaled directly without affecting the operation of other functions and, crucially, it can fail completely without taking down other functions. (modulo the reliability of databases which affect both styles of architecture identically.) microfunctions are horizontal slices through the domain: the advantages i wrote last time still hold up when translated to serverless terminology: deploying or retiring a function becomes as simple as switching it on or off which leads to greater freedom to experiment. scaling a function is limited to scaling a single type of process horizontally and the costs of doing this can be cleanly evaluated. the system as a whole becomes much more robust. when a function encounters problems it is limited to a single workflow such as issuing invoices. other functions can continue to operate independently. latency, bandwidth use and reliability are all improved because there are fewer network calls. the function still relies on the database and other support systems such as lock servers, but most of the data flow is controlled in-process. the unit of testing and deployment is a single function which reduces the complexity and cost of maintenance. one major advantage that i missed is the potential for extreme cost savings through scale, particularly the scale attainable by running on public shared infrastructure. since all the variability of microservice deployment configurations is abstracted away into a simple request/response interface the microfunctions can be run as isolated shared-nothing processes, billed only for the resources they use in their short lifetime. anyone who has costed for redundant microservices simply for basic resilience will appreciate the potential here. although there are number of cloud providers in this space (aws lambda, google cloud functions, azure functions) serverless is still an emerging paradigm with the problems that come with immaturity. adrian coyler recently summarized an excellent paper and presentation dealing with the challenges of building serverless systems which highlights many of these, including the lack of service level agreements and loose performance guarantees. it seems almost certain though that these will improve as the space matures and overtakes the microservice paradigm. other posts tagged as architecture, distributed-systems, technology, serverless, faas earlier posts gorecipes: fin wed, mar another blog refresh sun, feb why service architectures should focus on workflows mon, mar help me crowdfund my game amberfell mon, nov none none google takes symantec to the woodshed for mis-issuing , https certs [updated] | ars technica skip to main content biz & it tech science policy cars gaming & culture store forums subscribe close navigate store subscribe videos features reviews rss feeds mobile site about ars staff directory contact us advertise with ars reprints filter by topic biz & it tech science policy cars gaming & culture store forums settings front page layout grid list site theme black on white white on black sign in comment activity sign up or login to join the discussions! stay logged in | having trouble? sign up to comment and more sign up biz & it — google takes symantec to the woodshed for mis-issuing , https certs [updated] chrome to immediately stop recognizing ev status and gradually nullify all certs. dan goodin - mar , : pm utc enlarge nyttend reader comments with posters participating share this story share on facebook share on twitter share on reddit in a severe rebuke of one of the biggest suppliers of https credentials, google chrome developers announced plans to drastically restrict transport layer security certificates sold by symantec-owned issuers following the discovery they have allegedly mis-issued more than , certificates. effective immediately, chrome plans to stop recognizing the extended validation status of all certificates issued by symantec-owned certificate authorities, ryan sleevi, a software engineer on the google chrome team, said thursday in an online forum. extended validation certificates are supposed to provide enhanced assurances of a site's authenticity by showing the name of the validated domain name holder in the address bar. under the move announced by sleevi, chrome will immediately stop displaying that information for a period of at least a year. in effect, the certificates will be downgraded to less-secure domain-validated certificates. more gradually, google plans to update chrome to effectively nullify all currently valid certificates issued by symantec-owned cas. with symantec certificates representing more than percent of the internet's valid certificates by volume in , the move has the potential to prevent millions of chrome users from being able to access large numbers of sites. what's more, sleevi cited firefox data that showed symantec-issued certificates are responsible for percent of all certificate validations. to minimize the chances of disruption, chrome will stagger the mass nullification in a way that requires they be replaced over time. to do this, chrome will gradually decrease the "maximum age" of symantec-issued certificates over a series of releases. chrome will limit the expiration to no more than months after they were issued. by chrome , validity would be limited to nine months. thursday's announcement is only the latest development in google's -month critique of practices by symantec issuers. in october , symantec fired an undisclosed number of employees responsible for issuing test certificates for third-party domains without the permission of the domain holders. one of the extended-validation certificates covered google.com and www.google.com and would have given the person possessing it the ability to cryptographically impersonate those two addresses. a month later, google pressured symantec into performing a costly audit of its certificate issuance process after finding the mis-issuances went well beyond what symantec had first revealed. advertisement in january, an independent security researcher unearthed evidence that symantec improperly issued new certificates. thursday's announcement came after google's investigation revealed that over a span of years, symantec cas have improperly issued more than , certificates. such mis-issued certificates represent a potentially critical threat to virtually the entire internet population because they make it possible for the holders to cryptographically impersonate the affected sites and monitor communications sent to and from the legitimate servers. they are a major violation of the so-called baseline requirements that major browser makers impose of cas as a condition of being trusted by major browsers. in thursday's post, sleevi wrote: as captured in chrome's root certificate policy, root certificate authorities are expected to perform a number of critical functions commensurate with the trust granted to them. this includes properly ensuring that domain control validation is performed for server certificates, to audit logs frequently for evidence of unauthorized issuance, and to protect their infrastructure in order to minimize the ability for the issuance of fraudulent certs. on the basis of the details publicly provided by symantec, we do not believe that they have properly upheld these principles, and as such, have created significant risk for google chrome users. symantec allowed at least four parties access to their infrastructure in a way to cause certificate issuance, did not sufficiently oversee these capabilities as required and expected, and when presented with evidence of these organizations' failure to abide to the appropriate standard of care, failed to disclose such information in a timely manner or to identify the significance of the issues reported to them. these issues, and the corresponding failure of appropriate oversight, spanned a period of several years, and were trivially identifiable from the information publicly available or that symantec shared. the full disclosure of these issues has taken more than a month. symantec has failed to provide timely updates to the community regarding these issues. despite having knowledge of these issues, symantec has repeatedly failed to proactively disclose them. further, even after issues have become public, symantec failed to provide the information that the community required to assess the significance of these issues until they had been specifically questioned. the proposed remediation steps offered by symantec have involved relying on known-problematic information or using practices insufficient to provide the level of assurance required under the baseline requirements and expected by the chrome root ca policy. in an e-mailed statement, symantec officials wrote: advertisement as the world’s leading cyber security company and the market leading certificate authority, we understand the importance of the trust chain we provide for our customers and everyone who uses the internet. we learned of google’s proposal when they posted it on their blog today. their communication was unexpected and their proposed action is irresponsible. our ssl/tls certificate customers and partners need to know that this does not require any action at this time. symantec's repeated violations underscore one of the problems google and others have in enforcing terms of the baseline requirements. when violations are carried out by issuers with a big enough market share they're considered too big to fail. if google were to nullify all of the symantec-issued certificates overnight, it might cause widespread outages. the penalties outlined by sleevi seem to be aimed at minimizing such disruptions while still exacting a meaningful punishment. the penalties immediately revoke only the status of extended validation certificates issued by symantec, a move that is likely to be a major annoyance to many symantec customers and their website visitors, but not make sites unavailable. the untrusting of all symantec certificates, meanwhile, has a much higher potential of creating internet-wide problems. as sleevi explained it: "by phasing such changes in over a series of releases, we aim to minimize the impact any given release poses, while still continually making progress towards restoring the necessary level of security to ensure symantec-issued certificates are as trustworthy as certificates from other cas." update / / : pdt: in a blog post published friday morning, symantec officials once again criticized the google post. the officials also disputed the , certificate figure. "google's statements about our issuance practices and the scope of our past mis-issuances are exaggerated and misleading," they wrote. "for example, google’s claim that we have mis-issued , ssl/tls certificates is not true. in the event google is referring to, certificates—not , —were identified as mis-issued, and they resulted in no consumer harm. we have taken extensive remediation measures to correct this situation, immediately terminated the involved partner’s appointment as a registration authority (ra), and in a move to strengthen the trust of symantec-issued ssl/tls certificates, announced the discontinuation of our ra program." in an e-mail, google officials wrote: "we appreciate symantec's response. this remains an ongoing discussion, and we look forward to continuing our conversations with symantec about this issue. we want to enable an open and transparent assessment of the compatibility and interoperability risks, relative to potential security threats to our users." reader comments with posters participating share this story share on facebook share on twitter share on reddit dan goodin dan is the security editor at ars technica, which he joined in after working for the register, the associated press, bloomberg news, and other publications. email dan.goodin@arstechnica.com // twitter @dangoodin advertisement you must login or create an account to comment. channel ars technica ← previous story next story → related stories sponsored stories powered by today on ars store subscribe about us rss feeds view mobile site contact us staff advertise with us reprints newsletter signup join the ars orbital transmission mailing list to get weekly updates delivered to your inbox. sign me up → cnmn collection wired media group © condé nast. all rights reserved. use of and/or registration on any portion of this site constitutes acceptance of our user agreement (updated / / ) and privacy policy and cookie statement (updated / / ) and ars technica addendum (effective / / ). ars may earn compensation on sales from links on this site. read our affiliate link policy. your california privacy rights | do not sell my personal information the material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of condé nast. ad choices jodischneider.com/blog jodischneider.com/blog reading, technology, stray thoughts paid graduate hourly research position at uiuc for spring jodi schneider&# ;s information quality lab (http://infoqualitylab.org) seeks a graduate hourly student for a research project on bias in citation networks. biased citation benefits authors in the short-term by bolstering grants and papers, making them more easily accepted. however, it can have severe negative consequences for scientific inquiry. our goal is to find quantitative measures of [&# ;] avoiding long-haul air travel during the covid- pandemic i would not recommend long-haul air travel at this time. an epidemiological study of a . hour flight from the middle east to ireland concluded that groups ( people), traveling from continents in four groups, who used separate airport lounges, were likely infected in flight. the flight had % occupancy ( passengers/ seats; [&# ;] paid undergraduate research position at uiuc for fall & spring university of illinois undergraduates are encouraged to apply for a position in my lab. i particularly welcome applications from students in the new ischool bs/is degree or in the university-wide informatics minor. while i only have paid position open, i also supervise unpaid independent study projects. dr.&# ;jodi&# ;schneider and the information quality lab &# ;https://infoqualitylab.org&# ; seek [&# ;] #shutdownstem #strike blacklives #shutdownacademia i greatly appreciated receiving messages from senior people about their participation in the june th #shutdownstem #strike blacklives #shutdownacademia. in that spirit, i am sharing my email bounce message for tomorrow, and the message i sent to my research lab. email bounce: i am not available by email today:&# ;this june th is a day of action [&# ;] qotd: storytelling in protest and politics i recently read francesca polletta&# ;s book it was like a fever: storytelling in protest and politics ( , university of chicago press). i recommend it! it will appeal to researchers interested in topics such as&# ;narrative, strategic communication, (narrative) argumentation, or&# ;epistemology (here, of narrative). parts may also interest activists. the book&# ;s case studies are drawn from the [&# ;] knowledge graphs: an aggregation of definitions i am not aware of a consensus definition of knowledge graph. i&# ;ve been discussing this for awhile with liliana giusti serra, and the topic came up again with my fellow organizers of the knowledge graph session at us ts as we prepare for a panel. i&# ;ve proposed the following main features: rdf-compatible, has a defined schema (usually an [&# ;] qotd: doing more requires thinking less by the aid of symbolism, we can make transitions in reasoning almost mechanically by the eye which would otherwise call into play the higher faculties of the brain. &# ;civilization advances by extending the number of important operations that we can perform without thinking about them. operations of thought are like cavalry charges in a battle [&# ;] qotd: sally jackson on how disagreement makes arguments more explicit sally jackson explicates the notion of the &# ;disagreement space&# ; in a new topoi article: &# ;a position that remains in doubt remains in need of defense&# ; &# ; &# ;the most important theoretical consequence of seeing argumentation as a system for management of disagreement is a reversal of perspective on what arguments accomplish. are arguments the means by [&# ;] qotd: working out scientific insights on paper, lavoisier case study &# ;language does do much of our thinking for us, even in the sciences, and rather than being an unfortunate contamination, its influence has been productive historically, helping individual thinkers generate concepts and theories that can then be put to the test. the case made here for the constitutive power of figures [of speech] per se [&# ;] david liebovitz: achieving care transformation by infusing electronic health records with wisdom today i am at the health data analytics summit. the title of the keynote talk is achieving care transformation by infusing electronic health records with wisdom. it&# ;s a delight to hear from a medical informaticist: david m. liebovitz (publications in google scholar), md, facp, chief medical information officer, the university of chicago. he graduated from [&# ;]
warning: "continue" targeting switch is equivalent to "break". did you mean to use "continue "? in /kunden/ _ /jakoblog.de/wp/wp-content/plugins/mendeleyplugin/wp-mendeley.php on line

warning: cannot modify header information - headers already sent by (output started at /kunden/ _ /jakoblog.de/wp/wp-content/plugins/mendeleyplugin/wp-mendeley.php: ) in /kunden/ _ /jakoblog.de/wp/wp-includes/feed-atom.php on line
en &# ; jakoblog das weblog von jakob voß - - t : : z http://jakoblog.de/feed/atom/ wordpress jakob <![cdata[data models age like parents]]> http://jakoblog.de/?p= - - t : : z - - t : : z

denny vrandečić, employed as ontologist at google, noticed that all six of of six linked data applications linked to years ago (iwb, tabulator, disko, marbles, rdfbrowser , and zitgist) have disappeared or changed their calling syntax. this reminded me at a proverb about software and data:

software ages like fish, data ages like wine.

‏
the original form of this saying seems to come from james governor (@monkchips) who in derived it from from an earlier phrase:

hardware is like fish, operating systems are like wine.

the analogy of fishy applications and delightful data has been repeated and explained and criticized several times. i fully agree with the part about software rot but i doubt that data actually ages like wine (i&# ;d prefer whisky anyway). a more accurate simile may be &# ;data ages like things you put into your crowded cellar and then forget about&# ;.

thinking a lot about data i found that data is less interesting than the structures and rules that shape and restrict data: data models, ontologies, schemas, forms etc. how do they age compared with software and data? i soon realized:

data models age like parents.

first they guide you, give good advise, and support you as best as they can. but at some point data begin to rebel against their models. sooner or later parents become uncool, disconnected from current trends, outdated or even embarrassing. eventually you have to accept their quaint peculiarities and live your own life. that&# ;s how standards proliferate. both ontologies and parents ultimately become weaker and need support. and in the end you have to let them go, sadly looking back.

(the analogy could further be extended, for instance data models might be frustrated confronted by how actual data compares to their ideals, but that&# ;s another story)

]]> jakob <![cdata[wikidata documentation on the hackathon in vienna]]> http://jakoblog.de/?p= - - t : : z - - t : : z

at wikimedia hackathon , a couple of volunteers sat together to work on the help pages of wikidata. as part of that wikidata documentation sprint. ziko and me took a look at the wikidata glossary. we identified several shortcomings and made a list of rules how the glossary should look like. the result are the glossary guidelines. where the old glossary partly replicated wikidata:introduction, the new version aims to allow quick lookup of concepts. we already rewrote some entries of the glossary according to these guidelines but several entries are outdated and need to be improved still. we changed the structure of the glossary into a sortable table so it can be displayed as alphabetical list in all languages. the entries can still be translated with the translation system (it took some time to get familiar with this feature).

we also created some missing help pages such as help:wikimedia and help:wikibase to explain general concepts with regard to wikidata. some of these concepts are already explained elsewhere but wikidata needs at least short introductions especially written for wikidata users.

image taken by andrew lih (cc-by-sa)

]]> jakob <![cdata[introduction to phabricator at wikimedia hackathon]]> http://jakoblog.de/?p= - - t : : z - - t : : z

this weekend i participate at wikimedia hackathon in vienna. i mostly contribute to wikidata related events and practice the phrase "long time no see", but i also look into some introductionary talks.

in the late afternoon of day one i attended an introduction to phabricator project management tool given by andré klapper. phabricator was introduced in wikimedia foundation about three years ago to replace and unify bugzilla and several other management tools.

phabricator is much more than an issue tracker for software projects (although it is mainly used for this purpose by wikimedia developers). in summary there are tasks, projects, and teams. tasks can be tagged, assigned, followed,discussed, and organized with milestones and workboards. the latter are kanban-boards like those i know from trello, waffle, and github project boards.

phabricator is open source so you can self-host it and add your own user management without having to pay for each new user and feature (i am looking at you, jira). internally i would like to use phabricator but for fully open projects i don&# ;t see enough benefit compared to using github.

p.s.: wikimedia hackathon is also organized with phabricator. there is also a task for blogging about the event.

]]> jakob <![cdata[some thoughts on iiif and metadata]]> http://jakoblog.de/?p= - - t : : z - - t : : z

yesterday at dini ag kim workshop i martin baumgartner and stefanie rühle gave an introduction to the international image interoperability framework (iiif) with focus on metadata. i already knew that iiif is a great technology for providing access to (especially large) images but i had not have a detailed look yet. the main part of iiif is its image api and i hope that all major media repositories (i am looking at you, wikimedia commons) will implement it. in addition the iiif community has defined a &# ;presentation api&# ;, a &# ;search api&# ;, and an &# ;authentication api&# ;. i understand the need of such additional apis within the iiif community, but i doubt that solving the underlying problems with their own standards (instead of reusing existing standards) is the right way to go. standards should better &# ;do one thing and do it well&# ; (unix philosophy). if images are the &# ;one thing&# ; of iiif, then search and authentication are different matter.

in the workshop we only looked at parts of the presentation api to see where metadata (creator, dates, places, provenance etc. and structural metadata such as lists and hierarchies) could be integrated into iiif. such metadata is already expressed in many other formats such as mets/mods and tei so the question is not whether to use iiif or other metadata standards but how to connect iiif with existing metadata standards. a quick look at the presentation api surprised me to find out that the metadata element is explicitly not intended for additional metadata but only &# ;to be displayed to the user&# ;. the element contains an ordered list of key-value pairs that &# ;might be used to convey the author of the work, information about its creation, a brief physical description, or ownership information, amongst other use cases&# ;. at the same time the standard emphasizes that &# ;there are no semantics conveyed by this information&# ;. hello, mcfly? without semantics conveyed it isn&# ;t information! in particular there is no such thing as structured data (e.g. a list of key-value pairs) without semantics.

i think the design of field metadata in iiif is based on a common misconception about the nature of (meta)data, which i already wrote about elsewhere (sorry, german article &# ; some background in my phd and found by ballsun-stanton).

in a short discussion at twitter rob sanderson (getty) pointed out that the data format of iiif presentation api to describe intellectual works (called a manifest) is expressed in json-ld, so it can be extended by other rdf statements. for instance the field &# ;license&# ; is already defined with dcterms:rights. addition of a field &# ;author&# ; for dcterms:creator only requires to define this field in the json-ld @context of a manifest. after some experimenting i found a possible way to connect the &# ;meaningless&# ; metadata field with json-ld fields:

 {
 "@context": [
 "http://iiif.io/api/presentation/ /context.json",
 { 
 "author": "http://purl.org/dc/terms/creator",
 "bibo": "http://purl.org/ontology/bibo/"
 }
 ],
 "@id": "http://example.org/iiif/book /manifest",
 "@type": ["sc:manifest", "bibo:book"],
 "metadata": [
 {
 "label": "author",
 "property": "http://purl.org/dc/terms/creator",
 "value": "allen smithee"
 },
 { 
 "label": "license",
 "property": "http://purl.org/dc/terms/license", 
 "value": "cc-by . " 
 }
 ],
 "license": "http://creativecommons.org/licenses/by/ . /",
 "author": {
 "@id": "http://www.wikidata.org/entity/q ",
 "label": "allen smithee"
 }
 }

this solution requires an additional element property in the iiif specification to connect a metadata field with its meaning. iiif applications could then enrich the display of metadata fields for instance with links or additional translations. in json-ld some names such as &# ;cc-by . &# ; and &# ;allen smithee&# ; need to be given twice, but this is ok because normal names (in contrast to field names such as &# ;author&# ; and &# ;license&# ;) don&# ;t have semantics.

]]> jakob <![cdata[abbreviated uris with rdfns]]> http://jakoblog.de/?p= - - t : : z - - t : : z

working with rdf and uris can be annoying because uris such as &# ;http://purl.org/dc/elements/ . /title&# ; are long and difficult to remember and type. most rdf serializations make use of namespace prefixes to abbreviate uris, for instance &# ;dc&# ; is frequently used to abbreviate &# ;http://purl.org/dc/elements/ . /&# ; so &# ;http://purl.org/dc/elements/ . /title&# ; can be written as qualified name &# ;dc:title&# ;. this simplifies working with uris, but someone still has to remember mappings between prefixes and namespaces. luckily there is a registry of common mappings at prefix.cc.

a few years ago i created the simple command line tool rdfns and a perl library to look up uri namespace/prefix mappings. meanwhile the program is also available as debian and ubuntu package librdf-ns-perl. the newest version (not included in debian yet) also supports reverse lookup to abbreviate an uri to a qualified name. features of rdfns include:

look up namespaces (as rdf/turtle, rdf/xml, sparql&# ;)

 $ rdfns foaf.ttl foaf.xmlns dbpedia.sparql foaf.json
 
 @prefix foaf:  .
 xmlns:foaf="http://xmlns.com/foaf/ . /"
 prefix dbpedia: 
 "foaf": "http://xmlns.com/foaf/ . /"

expand a qualified name

 $ rdfns dc:title
 
 http://purl.org/dc/elements/ . /title

lookup a preferred prefix

 $ rdfns http://www.w .org/ / /geo/wgs _pos#
 
 geo

create a short qualified name of an url

 $ rdfns http://purl.org/dc/elements/ . /title
 
 dc:title

i use rdf-ns for all rdf processing to improve readability and to avoid typing long uris. for instance catmandu::rdf can be used to parse rdf into a very concise data structure:

 $ catmandu convert rdf --file rdfdata.ttl to yaml

]]> jakob <![cdata[testing command line apps with app::cmd]]> http://jakoblog.de/?p= - - t : : z - - t : : z

this posting has also been published at blogs.perl.org.

ricardo signes&# ; app::cmd has been praised a lot so i gave it a try for my recent command line app. in summary, the module is great although i missed some minor features and documentation (reminder to all: if you miss some feature in a cpan module, don&# ;t create yet another module but try to improve the existing one!). one feature i like a lot is how app::cmd facilitates writing tests for command line apps. after having written a short wrapper around app::cmd::tester my formerly ugly unit tests look very simple and clean. have a look at this example:

 use test::more;
 use app::paia::tester;
 
 new_paia_test;
 
 paia qw(config);
 is stdout, "{}\n";
 is error, undef;
 
 paia qw(config -c x.json --verbose);
 is error, "failed to open config file x.json\n";
 ok exit_code; 
 
 paia qw(config --config x.json --verbose foo bar);
 is output, "# saved config file x.json\n";
 
 paia qw(config foo bar);
 paia qw(config base http://example.org/);
 is exit_code, ;
 is output, '';
 
 paia qw(config);
 is_deeply stdout_json, { 
 base => 'http://example.org/',
 foo => 'bar',
 }, "get full config"
 
 done_paia_test;

the application is called paia &# ; that&# ;s how it called at command line and that&# ;s how it is simply called as function in the tests. the wrapper class (here: app::paia::tester) creates a singleton app::cmd::tester::result object and exports its methods (stdout, stderr, exit_code&# ;). this alone makes the test much more readable. the wrapper further exports two methods to set up a testing environment (new_paia_test) and to finish testing (done_paia_test). in my case the setup creates an empty temporary directory, other applications might clean up environment variables etc. depending on your application you might also add some handy functions like stdout_json to parse the app&# ;s output in a form that can better be tested.

]]> jakob <![cdata[my phd thesis about data]]> http://jakoblog.de/?p= - - t : : z - - t : : z

i have finally received paper copies of my phd thesis &# ;describing data patterns&# ;, published and printed via createspace. the full pdf has already been archived as cc-by-sa, but a paper print may still be nice and more handy (it&# ;s printed as small paperback instead of the large a -pdf). you can get a copy for . € or . € via amazon (isbn - - - ).

i also set up a little website at aboutdata.org. the site contains an html view of the pattern language that i developed as one result of the thesis.

i am sorry for not having written the thesis in pandoc markdown but in latex (source code available at github), so there is no epub/html version.

]]> jakob <![cdata[on the way to a library ontology]]> http://jakoblog.de/?p= - - t : : z - - t : : z

i have been working for some years on specification and implementation of several apis and exchange formats for data used in, and provided by libraries. unfortunately most existing library standards are either fuzzy, complex, and misused (such as marc ), or limited to bibliographic data or authority data, or both. libraries, however, are much more than bibliographic data &# ; they involve library patrons, library buildings, library services, library holdings, library databases etc.

during the work on formats and apis for these parts of library world, patrons account information api (paia) being the newest piece, i found myself more and more on the way to a whole library ontology. the idea of a library ontology started in (now moved to this location) but designing such a broad data model from bottom would surely have lead to yet another complex, impractical and unused library standard. meanwhile there are several smaller ontologies for parts of the library world, to be combined and used as linked open data.

in my opinion, ontologies, rdf, semantic web, linked data and all the buzz is is overrated, but it includes some opportunities for clean data modeling and data integration, which one rarely finds in library data. for this reason i try to design all apis and formats at least compatible with rdf. for instance the document availability information api (daia), created in (and now being slightly redesigned for version . ) can be accessed in xml and in json format, and both can fully be mapped to rdf. other micro-ontologies include:

document service ontology (dso) defines typical document-related services such as loan, presentation, and digitization
simple service status ontology (ssso) defines a service instance as kind of event that connects a service provider (e.g. a library) with a service consumer (e.g. a library patron). ssso further defines typical service status (e.g. reserved, prepared, executed&# ;) and limitations of a service (e.g. a waiting queue or a delay
patrons account information api (paia) will include a mapping to rdf to express basic patron information, fees, and a list of current services in a patron account, based on ssso and dso.
document availability information api (daia) includes a mapping to rdf to express the current availability of library holdings for selected services. see here for the current draft.
a holdings ontology should define properties to relate holdings (or parts of holdings) to abstract documents and editions and to holding institutions.
gbv ontology contains several concepts and relations used in gbv library network that do not fit into other ontologies (yet).
one might further create a database ontology to describe library databases with their provider, extent apis etc. &# ; right now we use the gbv ontology for this purpose. is there anything to reuse instead of creating just another ontology?!

the next step will probably creation of a small holdings ontology that nicely fits to the other micro-ontologies. this ontology should be aligned or compatible with the bibframe initiative, other ontologies such as schema.org, and existing holding formats, without becoming too complex. the german initiative dini-kim has just launched a a working group to define such holding format or ontology.

]]> jakob <![cdata[dead end electronic resource citation (erc)]]> http://jakoblog.de/?p= - - t : : z - - t : : z

tidying up my phd notes, i found this short rant about &# ;electronic resource citation&# ;. i have not used it anywhere, so i publish it here, licensed under cc-by-sa.

electronic resource citation (erc) was introduced by john kunze with a presentation at the international conference on dublin core and metadata applications and with a paper in the journal of digital information, vol. , no ( ). kunze cited his paper in a call for an erc interest group within the dublin core metadata initiative (dcmi) at the perl lib mailing list, giving the following example of an erc:

 erc: kunze, john a. | a metadata kernel for electronic permanence
 | | http://jodi.ecs.soton.ac.uk/articles/v /i /kunze/

an erc is a minimal &# ;kernel&# ; metadata record that consist of four elements: who, what, when and where. in the given example they are:

 who: kunze, john a.
 what: a metadata kernel for electronic permanence
 when: 
 where: http://jodi.ecs.soton.ac.uk/articles/v /i /kunze/

ironically the given url is obsolete, the host &# ;jodi.ecs.soton.ac.uk&# ; does not even exist anymore. the erc is pretty useless if it just uses a fragile url to cite a resource. how about some value that does not change over time, e.g:

 where: journal of digital information, volume issue

as erc is defined as &# ;a location or machine-oriented identifier&# ;, one could also use stable identifiers:

 where: issn - , article no.

both issn and article numbers are much more identifiers then urls. citing an url is more like

 where: at the desk in the little reading room of my library

by the way the current location is http://www.rice.edu/perl lib/archives/ - /msg .html &# ; but who knows whether texas a&# ;m university will still host the journal at this url in years?

there are some interesting ideas in the original erc proposal (different kinds of missing values, temper date values, the four questions etc.), but its specification and implementation are just ridiculous and missing references to current technology (you know that you are doing something wrong in specification if you start to define your own encodings for characters, dates etc. instead of concentrating to your core subject and refering to existing specifications for the rest). the current draft ( ) is a typical example of badly mixing modeling and encoding issues and of loosing touch with existing, established data standards.

in addition to problems at the &# ;low level&# ; of encoding, the &# ;high level&# ; of conceptual modeling lacks appropriate references. what about the relation of erc concepts to models such as frbr and cidoc-crm? why are &# ;who&# ;, &# ;when&# ;, &# ;where&# ;, &# ;what&# ; the important metadata fields (in many cases the most interesting question is &# ;why&# ;)? how about ranganathan&# ;s colon classification with personality, matter, energy, space, and time?

in summary the motivation behind erc contains some good ideas, but its form is misdirected.

]]> jakob <![cdata[access to library accounts for better user experience]]> http://jakoblog.de/?p= - - t : : z - - t : : z

i just stumbled upon readersfirst, a coalition of (public) libraries that call for a better user experience for library patrons, especially to access e-books. the libraries regret that

the products currently offered by e-content distributors, the middlemen from whom libraries buy e-books, create a fragmented, disjointed and cumbersome user experience.

one of the explicit goals of readersfirst is to urge providers of e-content and integrated library systems for systems that allow users to

place holds, check-out items, view availability, manage fines and receive communications within individual library catalogs or in the venue the library believes will serve them best, without having to visit separate websites.

in a summary of the first readersfirst meeting at january , the president of queens library (ny) is cited with the following request:

the reader should be able to look at their library account and see what they have borrowed regardless of the vendor that supplied the ebook.

this goal matches well with my activity at gbv: as part of a project to implement a mobile library app, i designed an api to access library accounts. the patrons account information api (paia) is current being implemented and tested by two independent developers. it will also be used to provide a better user experience in vufind discovery interfaces.

during the research for paia i was surprised by the lack of existing methods to access library patron accounts. some library systems not even provide an internal api to connect to the loan system &# ; not to speak of a public api that could directly be used by patrons and third parties. the only example i could find was york university libraries with a simple, xml-based, read-only api. this lack of public apis to library patron accounts is disappointing, given that its almost ten years after the buzz around web . , service oriented architecture, and mashups. all all major providers of web applications (google, twitter, facebook, stackexchange, github etc.) support access to user accounts via apis.

the patrons account information api will hopefully fill this gap with defined methods to place holds and to view checked out items and fines. papi is agnostic to specific library systems, aligned with similar apis as listed above, and designed with rdf in mind (without any need to bother with rdf, apart from the requirement to use uris as identifiers). feedback and implementations are very welcome!

]]> dshr's blog: chromebook linux update dshr's blog i'm david rosenthal, and this is a place to discuss the work i'm doing in digital preservation. thursday, february , chromebook linux update my three acer c chromebooks running linux are still giving yeoman service, although for obvious reasons i'm not travelling these days. but it is time for an update to 's travels with a chromebook. below the fold, an account of some adventures in sysadmin. battery replacement the battery in c # , which is over six years old, would no longer hold a charge. i purchased a dentsing ap j k replacement battery from amazon. i opened the c , removed the old battery, inserted the new one, closed up the case, and all was well. it was an impressively easy fix. sleeping & waking sometime around last october linux mint switched from kernels in the . series to kernels in the . series. mint uses . series kernels. the . kernels on the c went to sleep properly when the lid closed, and woke properly when the lid opened. the . kernels appeared to go to sleep correctly, but when the lid opened did a cold boot. because this problem happens immediately on wake, and because sleep appears to work correctly, there is no useful information in the logs; this appears to be a very hard problem to diagnose. here is my work-around to use the . . - kernel (the last i installed via updates) on a vanilla linux mint installation: install linux mint . mate edition. add repositories using administration/software sources: deb http://archive.ubuntu.com/ubuntu bionic main restricted universe multiverse deb http://archive.ubuntu.com/ubuntu bionic-updates main restricted universe multiverse deb http://security.ubuntu.com/ubuntu/ bionic-security main restricted universe multiverse install kernel . . - -generic: sudo apt-get install linux-headers- . . - linux-headers- . . - -generic linux-image- . . - -generic linux-modules- . . - -generic linux-modules-extra- . . - -generic edit /etc/default/grub to show the menu of kernels: grub_timeout_style=menu grub_timeout= edit /etc/default/grub so that your most recent choice of kernel becomes the default: grub_savedefault=true grub_default=saved run update-grub after you choose the . . - kernel the first time, it should boot by default, and sleep and wake should work properly. the problem with ehci-pci on wakeup has gone away, there is no need to install the userland files from galliumos. disk & home directory encryption note that you should ideally install linux mint . with full-disk encryption. the release notes explain: the move to systemd caused a regression in ecrypts which is responsible for mounting/unmounting encrypted home directories when you login and logout. because of this issue, please be aware that in mint and newer releases, your encrypted home directory is no longer unmounted on logout: https://bugs.launchpad.net/ubuntu/+source/gnome-session/+bug/ . mint with a full-disk encryption had this problem but i haven't been able to reproduce it with mint and the . . - kernel. home directory encryption works, but will leave its contents decrypted after you log out, rather spoiling the point. touchpad as i described here, the touchpad isn't one of the c 's best features, and it is necessary to disable it while typing, or while using a mouse such as the excellent tracpoint. i use ataraeo's touchpad-indicator, but this doesn't seem to work on mint . out of the box, using x.org's libinput driver. the release notes discuss using the synaptics driver instead. i installed it and, after creating the directory ~/.config/autostart the touchpad-indicator starts on login and works fine. posted by david. at : am labels: linux no comments: post a comment newer post older post home subscribe to: post comments (atom) blog rules posts and comments are copyright of their respective authors who, by posting or commenting, license their work under a creative commons attribution-share alike . united states license. off-topic or unsuitable comments will be deleted. dshr dshr in anwr recent comments full comments blog archive ▼ ( ) ▼ february ( ) talk at berkeley's information access seminar chromebook linux update ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► july ( ) ► june ( ) ► february ( ) ► ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) lockss system has permission to collect, preserve, and serve this archival unit. simple theme. powered by blogger. warning: "continue" targeting switch is equivalent to "break". did you mean to use "continue "? in /kunden/ _ /jakoblog.de/wp/wp-content/plugins/mendeleyplugin/wp-mendeley.php on line jakoblog — das weblog von jakob voß blog about erster expliziter entwurf einer digitalen bibliothek ( ) . märz um : kommentare ich recherchiere (mal wieder) zu digitalen bibliotheken und habe mich gefragt, wann der begriff zum ersten mal verwendet wurde. laut google books taucht (nach aussortieren falsch-positiver treffer) „digital library“ erstmals in einem bericht für das us-außenministerium auf. die bibliographischen daten habe ich bei wikidata eingetragen. der bericht „the need for fundamental research in seismology“ wurde damals erstellt um zu untersuchen wie mit seismischen wellen atomwaffentests erkannt werden können. in anhang legte john gerrard, einer von vierzehn an der studie beteiligten wissenschaftler, auf zwei seiten den bedarf an einem rechenzentrum mit einem ibm rechner dar. da das us-regierungsdokument gemeinfrei ist hier die entsprechenden seiten: bei der geplanten digitalen bibliothek handelt es sich um eine sammlung von forschungsdaten mitsamt wissenschaftlicher software um aus den forschungsdaten neue erkenntnisse zu gewinnen: the following facilities should be available: a computer equivalent to the ibm series, plus necessary peripheral equipment. facilities for converting standard seismograms into digital form. a library of records of earthquakes and explosions in form suitable for machine analysis. a (growing) library of basic programs which have proven useful in investigations of seismic disturbances and related phenomena. … klingt doch ziemlich aktuell, oder? gefallen hat mir auch die beschreibung des rechenzentrums als „open shop“ und der hinweis „nothing can dampen enthusiasm for new ideas quite as effectively as long periods of waiting time“. die bezeichnung „digital library“ bezieht sich in dem text primär auf die sammlung von digitalisierten seimsmogrammen. am ende der empfehlung wird abweichend der begriff „digitized library“ verwendet. dies spricht dafür dass beide begriffe synonym verwendet wurden. interessanterweise bezieht sich „library“ aber auch auf die sammlung von computerprogrammen. ob das empfohlene rechenzentrum mit digitaler bibliothek realisiert wurde konnte ich leider nicht herausfinden (vermutlich nicht). zum autor dr. john gerrard ist mir nicht viel mehr bekannt als dass er als director of data systems and earth science research bei texas instruments (ti) arbeitete. ti wurde als „geophysical service incorporated“ zur seismischen erkundung von erdöllagerstätten gegründet und bekam den regierungsauftrag zur Überwachung von kernwaffentests (projekt vela uniform). an gerrard erinnert sich in diesem interview ein ehemaliger kollege: john gerrard: into digital seismology, and he could see a little bit of the future of digital processing and he talked about how that could be effective in seismology, he was right that this would be important in seismology in birmingham gibt es einen geologen gleichen namens, der ist aber erst geboren. ich vermute dass gerrard bei ti an der entwicklung des texas instruments automatic computer (tiac) beteiligt war, der speziell zur digitalen verarbeitung seismischer daten entwickelt wurde. der einsatz von computern in klassischen bibliotheken kam übrigens erst mit der nächsten rechner-generation: das marc-format wurde in den ern mit dem ibm system/ entwickelt (von henriette avram, die zuvor bei der nsa auch mit ibm gearbeitet hatte). davor gabe es den fiktiven bibliotheks-computer emmarac (angelehnt an eniac und univac) in „eine frau, die alles weiß“ mit katharine hepburn als bibliothekarin und spencer tracy als computervertreter. bis ende der er taucht der begriff „digital library“ bei google books übrigens nur vereinzelt auf. tags: digital library, geschichte kommentare data models age like parents . märz um : keine kommentare denny vrandečić, employed as ontologist at google, noticed that all six of of six linked data applications linked to years ago (iwb, tabulator, disko, marbles, rdfbrowser , and zitgist) have disappeared or changed their calling syntax. this reminded me at a proverb about software and data: software ages like fish, data ages like wine. ‏ the original form of this saying seems to come from james governor (@monkchips) who in derived it from from an earlier phrase: hardware is like fish, operating systems are like wine. the analogy of fishy applications and delightful data has been repeated and explained and criticized several times. i fully agree with the part about software rot but i doubt that data actually ages like wine (i’d prefer whisky anyway). a more accurate simile may be „data ages like things you put into your crowded cellar and then forget about“. thinking a lot about data i found that data is less interesting than the structures and rules that shape and restrict data: data models, ontologies, schemas, forms etc. how do they age compared with software and data? i soon realized: data models age like parents. first they guide you, give good advise, and support you as best as they can. but at some point data begin to rebel against their models. sooner or later parents become uncool, disconnected from current trends, outdated or even embarrassing. eventually you have to accept their quaint peculiarities and live your own life. that’s how standards proliferate. both ontologies and parents ultimately become weaker and need support. and in the end you have to let them go, sadly looking back. (the analogy could further be extended, for instance data models might be frustrated confronted by how actual data compares to their ideals, but that’s another story) tags: data modeling keine kommentare in memoriam ingetraut dahlberg . oktober um : kommentare die informationswissenschaftlerin ingetraut dahlberg, bekannt unter anderem als gründerin der international society for knowledge organization (isko), ist letzte woche im alter von jahren verstorben. meine erste reaktion nach einem angemessenen bedauern war es in wikipedia und in wikidata das sterbedatum einzutragen, was jedoch schon andere erledigt hatten. also stöberte ich etwas im lebenslauf, und legte stattdessen wikidata-items zum mcluhan institute for digital culture and knowledge organization an, dem dahlberg schon zu lebzeiten ihre bibliothek vermacht hat, das aber bereits wieder geschlossen wurde. der ehemalige direktor kim veltman betreibt noch eine webseite zum institut und nennt in seinen memoiren ingetraut dahlberg, douglas engelbart, ted nelson und tim berners lee in einem atemzug. das sollte eigentlich grund genug sein, mich mit der frau zu beschäftigen. wenn ich ehrlich bin war mein verhältnis zu ingetraut dahlberg allerdings eher ein distanziert-ignorantes. ich wusste um ihre bedeutung in der „wissensorganisation-szene“, der ich zwangsläufig auch angehöre, bin ihr aber nur ein oder zwei mal auf isko-tagungen begegnet und hatte auch nie interesse daran mich mehr mit ihr auseinanderzusetzen. als „junger wilder“ schien sie mir immer wie eine person, deren zeit schon lange vorbei ist und deren beiträge hoffnungslos veraltet sind. dass alte ideen auch im rahmen der wissensorganisation keineswegs uninteressant und irrelevant sind, sollte mir durch die beschäftigung mit ted nelson und paul otlet eigentlich klar sein; irgendwie habe ich aber bisher nie einen anknüpfungspunkt zu dahlbergs werk gefunden. wenn ich zurückblicke muss der auslöser für meine ignoranz in meiner ersten begegnung mit vertreter*innen der wissensorganisation auf einer isko-tagung anfang der er jahre liegen: ich war damals noch frischer student der bibliotheks- und informationswissenschaft mit informatik-hintergrund und fand überall spannende themen wie wikipedia, social tagging und ontologien, die prinzipiell alle etwas mit wissensorganisation zu tun hatten. bei der isko fand ich dagegen nichts davon. das internet schien jedenfalls noch sehr weit weg. erschreckend fand ich dabei weniger das fehlen inhaltlicher auseinandersetzung mit den damals neuesten entwicklungen im netz sondern die formale fremdheit: mehrere der beteiligten wissenschaftler*innen hatten nach meiner erinnerung nicht einmal eine email-adresse. menschen, die sich anfang der er jahre ohne email mit information und wissen beschäftigten konnte ich einfach nicht ernst nehmen. so war die isko in meiner ignoranz lange ein relikt, das ähnlich wie die international federation for information and documentation (fid, warum haben die sich eigentlich nicht zusammengetan?) auf tragische weise von der technischen entwicklung überholt wurde. und ingetraut dahlberg stand für mich exemplarisch für dieses ganze scheitern einer zunft. inzwischen sehe ich es etwas differenzierter und bin froh teil dieser kleinen aber feinen fachcommunity zu sein (und wenn die isko endlich auf open access umstellt, werde ich auch meinen publikations-boycott aufgeben). in jedem fall habe ich ingetraut dahlberg unrecht getan und hoffe auf differenziertere auseinandersetzungen mit ihrem werk. tags: nachruf kommentare wikidata documentation on the hackathon in vienna . mai um : kommentare at wikimedia hackathon , a couple of volunteers sat together to work on the help pages of wikidata. as part of that wikidata documentation sprint. ziko and me took a look at the wikidata glossary. we identified several shortcomings and made a list of rules how the glossary should look like. the result are the glossary guidelines. where the old glossary partly replicated wikidata:introduction, the new version aims to allow quick lookup of concepts. we already rewrote some entries of the glossary according to these guidelines but several entries are outdated and need to be improved still. we changed the structure of the glossary into a sortable table so it can be displayed as alphabetical list in all languages. the entries can still be translated with the translation system (it took some time to get familiar with this feature). we also created some missing help pages such as help:wikimedia and help:wikibase to explain general concepts with regard to wikidata. some of these concepts are already explained elsewhere but wikidata needs at least short introductions especially written for wikidata users. image taken by andrew lih (cc-by-sa) tags: wikidata, wmhack kommentare introduction to phabricator at wikimedia hackathon . mai um : kommentar this weekend i participate at wikimedia hackathon in vienna. i mostly contribute to wikidata related events and practice the phrase "long time no see", but i also look into some introductionary talks. in the late afternoon of day one i attended an introduction to phabricator project management tool given by andré klapper. phabricator was introduced in wikimedia foundation about three years ago to replace and unify bugzilla and several other management tools. phabricator is much more than an issue tracker for software projects (although it is mainly used for this purpose by wikimedia developers). in summary there are tasks, projects, and teams. tasks can be tagged, assigned, followed,discussed, and organized with milestones and workboards. the latter are kanban-boards like those i know from trello, waffle, and github project boards. phabricator is open source so you can self-host it and add your own user management without having to pay for each new user and feature (i am looking at you, jira). internally i would like to use phabricator but for fully open projects i don’t see enough benefit compared to using github. p.s.: wikimedia hackathon is also organized with phabricator. there is also a task for blogging about the event. tags: wikimedia, wmhack kommentar some thoughts on iiif and metadata . mai um : keine kommentare yesterday at dini ag kim workshop i martin baumgartner and stefanie rühle gave an introduction to the international image interoperability framework (iiif) with focus on metadata. i already knew that iiif is a great technology for providing access to (especially large) images but i had not have a detailed look yet. the main part of iiif is its image api and i hope that all major media repositories (i am looking at you, wikimedia commons) will implement it. in addition the iiif community has defined a „presentation api“, a „search api“, and an „authentication api“. i understand the need of such additional apis within the iiif community, but i doubt that solving the underlying problems with their own standards (instead of reusing existing standards) is the right way to go. standards should better „do one thing and do it well“ (unix philosophy). if images are the „one thing“ of iiif, then search and authentication are different matter. in the workshop we only looked at parts of the presentation api to see where metadata (creator, dates, places, provenance etc. and structural metadata such as lists and hierarchies) could be integrated into iiif. such metadata is already expressed in many other formats such as mets/mods and tei so the question is not whether to use iiif or other metadata standards but how to connect iiif with existing metadata standards. a quick look at the presentation api surprised me to find out that the metadata element is explicitly not intended for additional metadata but only „to be displayed to the user“. the element contains an ordered list of key-value pairs that „might be used to convey the author of the work, information about its creation, a brief physical description, or ownership information, amongst other use cases“. at the same time the standard emphasizes that „there are no semantics conveyed by this information“. hello, mcfly? without semantics conveyed it isn’t information! in particular there is no such thing as structured data (e.g. a list of key-value pairs) without semantics. i think the design of field metadata in iiif is based on a common misconception about the nature of (meta)data, which i already wrote about elsewhere (sorry, german article – some background in my phd and found by ballsun-stanton). in a short discussion at twitter rob sanderson (getty) pointed out that the data format of iiif presentation api to describe intellectual works (called a manifest) is expressed in json-ld, so it can be extended by other rdf statements. for instance the field „license“ is already defined with dcterms:rights. addition of a field „author“ for dcterms:creator only requires to define this field in the json-ld @context of a manifest. after some experimenting i found a possible way to connect the „meaningless“ metadata field with json-ld fields: { "@context": [ "http://iiif.io/api/presentation/ /context.json", { "author": "http://purl.org/dc/terms/creator", "bibo": "http://purl.org/ontology/bibo/" } ], "@id": "http://example.org/iiif/book /manifest", "@type": ["sc:manifest", "bibo:book"], "metadata": [ { "label": "author", "property": "http://purl.org/dc/terms/creator", "value": "allen smithee" }, { "label": "license", "property": "http://purl.org/dc/terms/license", "value": "cc-by . " } ], "license": "http://creativecommons.org/licenses/by/ . /", "author": { "@id": "http://www.wikidata.org/entity/q ", "label": "allen smithee" } } this solution requires an additional element property in the iiif specification to connect a metadata field with its meaning. iiif applications could then enrich the display of metadata fields for instance with links or additional translations. in json-ld some names such as „cc-by . “ and „allen smithee“ need to be given twice, but this is ok because normal names (in contrast to field names such as „author“ and „license“) don’t have semantics. tags: iiif, metadata keine kommentare ersatzteile aus dem d-drucker . dezember um : kommentar krach, zack, bumm! da liegt die jalousie unten. ein kleinen plastikteil ist abgebrochen, das wäre doch ein prima anwendungsfall für einen d-drucker, oder? schön länger spiele ich mit dem gedanken, einen d-drucker anzuschaffen, kann aber nicht so recht sagen, wozu eigentlich. die herstellung von ersatzteilen aus dem d-drucker scheint mir allerdings eher so ein versprechen zu sein wie der intelligente kühlschrank: theoretisch ganz toll aber nicht wirklich praktisch. es würde mich vermutlich stunden kosten, das passende teil auf diversen plattformen wie thingiverse zu finden oder es mit cad selber zu konstruieren. ohne verlässliche d-modelle bringt also der beste d-drucker nichts, deshalb sind die geräte auch nur ein teil der lösung zur herstellung von ersatzteilen. ich bezweifle sehr dass in naher zukunft hersteller d-modelle ihrer produkte zum download anbieten werden, es sei denn es handelt sich um open hardware. abgesehen von elektronischen bastelprojekten ist das angebot von open-hardware-produkten für den hausgebrauch aber noch sehr überschaubar. dennoch denke ich, dass open hardware, das heisst produkte deren baupläne frei lizensiert zur kostenlosen verfügung stehen, sowie standardisierte bauteile das einzig richtige für den einsatz von d-druckern im hausgebrauch sind. ich werde das problem mit der kaputten jalousie erstmal mit analoger technik angehen und schauen, was ich so an passenden materialien und werkzeugen herumliegen habe. vielleicht hilft ja gaffer tape? tags: d-drucker, maker, open hardware kommentar einfachste projekthomepage bei github . september um : kommentar die einfachste form einer projekthomepage bei github pages besteht aus einer startseite, die lediglich auf das repository verweist. lokal lässt sich eine solche seite so angelegen: . erstellung des neuen, leeren branch gh-pages: git checkout --orphan gh-pages git rm -rf . . anlegen der datei index.md mit folgendem inhalt: --- --- # {{site.github.project_title}} [{{site.github.repository_url}}]({{site.github.repository_url}}#readme). . hinzufügen der datei und push nach github git add index.md git commit -m "homepage" git push origin gh-pages tags: github kommentar abbreviated uris with rdfns . september um : kommentare working with rdf and uris can be annoying because uris such as „http://purl.org/dc/elements/ . /title“ are long and difficult to remember and type. most rdf serializations make use of namespace prefixes to abbreviate uris, for instance „dc“ is frequently used to abbreviate „http://purl.org/dc/elements/ . /“ so „http://purl.org/dc/elements/ . /title“ can be written as qualified name „dc:title„. this simplifies working with uris, but someone still has to remember mappings between prefixes and namespaces. luckily there is a registry of common mappings at prefix.cc. a few years ago i created the simple command line tool rdfns and a perl library to look up uri namespace/prefix mappings. meanwhile the program is also available as debian and ubuntu package librdf-ns-perl. the newest version (not included in debian yet) also supports reverse lookup to abbreviate an uri to a qualified name. features of rdfns include: look up namespaces (as rdf/turtle, rdf/xml, sparql…) $ rdfns foaf.ttl foaf.xmlns dbpedia.sparql foaf.json @prefix foaf: . xmlns:foaf="http://xmlns.com/foaf/ . /" prefix dbpedia: "foaf": "http://xmlns.com/foaf/ . /" expand a qualified name $ rdfns dc:title http://purl.org/dc/elements/ . /title lookup a preferred prefix $ rdfns http://www.w .org/ / /geo/wgs _pos# geo create a short qualified name of an url $ rdfns http://purl.org/dc/elements/ . /title dc:title i use rdf-ns for all rdf processing to improve readability and to avoid typing long uris. for instance catmandu::rdf can be used to parse rdf into a very concise data structure: $ catmandu convert rdf --file rdfdata.ttl to yaml tags: perl, rdf kommentare das wissen der welt . august um : kommentare denny vrandečić, einer der köpfe hinter semantic mediawiki und wikidata, hat eine clevere metrik vorgeschlagen um den erfolg der wikimedia-projekte zu messen. die tätigkeit und damit das ziel der wikimedia-foundation wurde von jimbo wales so ausgedrückt: imagine a world in which every single person on the planet is given free access to the sum of all human knowledge. that’s what we’re doing. in wikiquote wird dieser bekannte ausspruch momentan folgendermaßen übersetzt: „stell dir eine welt vor, in der jeder mensch auf der erde freien zugang zum gesamten menschlichem wissen hat. das ist, was wir machen.“ wie lässt sich nun aber quantifizieren, zu welchem grad das ziel erreicht ist? so wie ich es verstanden (und in meine worte übersetzt) habe, schlägt denny folgendes vor: für jedem menschen auf der welt gibt es theoretisch eine zahl zwischen null und eins, die angibt wieviel vom gesamten wissens der welt („the sum of all human knowledge“) diesem menschen durch wikimedia-inhalte zugänglich ist. der wert lässt sich als prozentzahl des zugänglichen weltwissens interpretieren – da sich wissen aber kaum so einfach messen und vergleichen lässt, ist diese interpretation problematisch. der wert von eins ist utopisch, da wikipedia & co nicht alles wissen der welt enthält. für menschen ohne internet-zugang kann der wert aber bei null liegen. selbst mit zugang zu wikipedia ist die zahl bei jedem menschen eine andere, da nicht alle inhalte in allen sprachen vorhanden sind und weil viele inhalte ohne vorwissen unverständlich und somit praktisch nicht zugänglich sind. die zahlen der individuellen zugänglichkeit des weltwissens lassen sich nun geordnet in ein diagram eintragen, das von links (maximales wissen) nach rechts (kein wissen durch zugänglich) alle menschen aufführt. wie denny an folgendem bild ausführt, kann die wikimedia-community ihrem weg auf verschiedenen wegen näher kommen: ( ) der ausbau von vielen artikeln in einem komplexen spezialgebiet oder einer kleinen sprache kommt nur wenigen menschen zu gute. ( ) stattdessen könnten auch die wichtigsten artikel bzw. themen in sprachen verbessert und ergänzt werden, welche von vielen menschen verstanden werden. ( ) schließlich kann wikimedia auch dafür sorgen, dass mehr menschen einen zugang zu den wikimedia-ihren inhalten bekommen – zum beispiel durch initiativen wie wikipedia zero ich halte die von denny vorgeschlagene darstellung für hilfreich um über das einfache zählen von wikipedia-artikeln hinauszukommen. wie er allerdings selber zugibt, gibt es zahlreiche offene fragen da sich die tatsächlichen zahlen der verfügbarkeit von wissen nicht einfach ermitteln lassen. meiner meinung nach liegt ein grundproblem darin, dass sich wissen – und vor allem das gesamte wissen der menschheit – nicht quantifizieren lässt. es ist auch irreführend davon auszugehen, dass die wikimedia-produkte wissen sammeln oder enthalten. möglicherweise ist dieser irrtum für die metrik egal, nicht aber für das was eigentlich gemessen werden soll (zugänglichkeit des wissens der welt). falls wikimedia an einem unverstelltem blick auf die frage interessiert ist, wieviel des wissens der menschheit durch ihre angebote den menschen zugänglich gemacht wird, könnte es helfen mal einige philosophen und philosophinnen zu fragen. ganz im ernst. mag sein (und so vermute ich mit meinem abgebrochenen philosophie-studium), dass am ende lediglich deutlich wird, warum dass ganze wikimedia-projekt nicht zu realisieren ist; selbst erkenntnisse über mögliche gründe dieses scheitern wären aber hilfreich. vermutlich ist es aber zu verpönt, philosophen ernsthaft um rat zu fragen oder die verbliebenen philosophen beschäftigen sich lieber mit anderen fragen. p.s: eine weitere relevante disziplin zur beantwortung der frage wieviel wissen der welt durch wikipedia & co der menschheit zugänglich gemacht wird, ist die pädagogik, aber da kenne ich mich noch weniger aus als mit der philosophie. tags: freie inhalte, wikipedia, wissensordnung kommentare nächste seite » neueste beiträge erster expliziter entwurf einer digitalen bibliothek ( ) data models age like parents in memoriam ingetraut dahlberg wikidata documentation on the hackathon in vienna introduction to phabricator at wikimedia hackathon neueste kommentare will taking dht at increase penis size bei abbreviated uris with rdfns http://asikgapleqq.com/ bei dublin core conference started thekitchenconnection-nc.com bei suchmaschinenoptimierung á la insm http://juarabola .com bei suchmaschinenoptimierung á la insm http://thebibble.org/natexuphilptq bei suchmaschinenoptimierung á la insm themen api archivierung atom bibliothek bibliothekswissenschaft bibsonomy daia data modeling digital library feed freie inhalte gbv humor identifier katalog katalog . librarything literatur mashup medien metadata microformats musik oai open access openstreetmap perl pica politik rdf seealso semantic web soa software standards suchmaschine tagging veranstaltung web . webservices widget wikimedia wikipedia wikis Überwachungsstaat blogroll planet biblioblog . planet code lib planet wikimedia (de) feeds siehe auch powered by wordpress with theme based on pool theme and silk icons. entries and comments feeds. valid xhtml and css. ^top^ dshr's blog: effort balancing and rate limits dshr's blog i'm david rosenthal, and this is a place to discuss the work i'm doing in digital preservation. thursday, january , effort balancing and rate limits catalin cimpanu reports on yet another crime wave using bitcoin in as bitcoin price surges, ddos extortion gangs return in force: in a security alert sent to its customers and shared with zdnet this week, radware said that during the last week of and the first week of , its customers received a new wave of ddos extortion emails. extortionists threatened companies with crippling ddos attacks unless they got paid between and bitcoins ($ , to $ , ) ... the security firm believes that the rise in the bitcoin-to-usd price has led to some groups returning to or re-prioritizing ddos extortion schemes. and dan goodin reports on the latest technique the ddos-ers are using in ddosers are abusing microsoft rdp to make attacks more powerful: as is typical with many authenticated systems, rdp responds to login requests with a much longer sequence of bits that establish a connection between the two parties. so-called booter/stresser services, which for a fee will bombard internet addresses with enough data to take them offline, have recently embraced rdp as a means to amplify their attacks, security firm netscout said. the amplification allows attackers with only modest resources to strengthen the size of the data they direct at targets. the technique works by bouncing a relatively small amount of data at the amplifying service, which in turn reflects a much larger amount of data at the final target. with an amplification factor of . to , gigabytes-per-second of requests directed at an rdp server will deliver roughly gbps to the target. i don't know why it took me so long to figure it out, but reading goodin's post i suddenly realized that techniques we described in impeding attrition attacks in p p systems, a follow-up to our award-winning sosp paper on the architecture of the lockss system, can be applied to preventing systems from being abused by ddos-ers. below the fold, brief details. among the lockss system's defenses against abuse are two relevant to ddos prevention, rate limits and effort balancing. rate limits i've written before about the importance of rate limits, quoting paul vixie: every reflection-friendly protocol mentioned in this article is going to have to learn rate limiting. this includes the initial tcp three-way handshake, icmp, and every udp-based protocol. in rare instances it's possible to limit one's participation in ddos reflection and/or amplification with a firewall, but most firewalls are either stateless themselves, or their statefulness is so weak that it can be attacked separately. the more common case will be like dns [response rate limiting], where deep knowledge of the protocol is necessary for a correctly engineered rate-limiting solution applicable to the protocol. the rdp server being used to ddos sees a flood of authentication requests whose source address has been spoofed to be the target of the ddos. this isn't what they'd see from a real user, so the rdp server should rate-limit sending authentication responses to a client to a reasonable rate for a real client. this would be helpful, but it isn't enough. because the ddos-ers use a large number of systems to mount an attack, even a fairly low rate of reponses can be harmful. effort balancing in our paper we wrote: effort balancing. if the effort needed by a requester to procure a service from a supplier is less than the effort needed by the supplier to furnish the requested service, then the system can be vulnerable to an attrition attack that consists simply of large numbers of ostensibly valid service requests. we can use provable effort mechanisms such as memory-bound functions to inflate the cost of relatively “cheap” protocol operations by an adjustable amount of provably performed but otherwise useless effort. by requiring that at each stage of a multi-step protocol exchange the requester has invested more effort in the exchange than the supplier, we raise the cost of an attrition strategy that defects part-way through the exchange. this effort balancing is applicable not only to consumed resources such as computations performed, memory bandwidth used or storage occupied, but also to resource commitments. for example, if an adversary peer issues a cheap request for service and then defects, he can cause the supplier to commit resources that are not actually used and are only released after a timeout (e.g., syn floods). the size of the provable effort required in a resource reservation request should reflect the amount of effort that could be performed by the supplier with the resources reserved for the request. vixie also noted the economic requirement: engineering economics requires that the cost in cpu, memory bandwidth, and memory storage of any new state added for rate limiting be insignificant compared with an attacker's effort. the reason rdp can be used to amplify a ddos attack is that, as goodin wrote: rdp responds to login requests with a much longer sequence of bits that establish a connection between the two parties. the obvious application of effort balancing would be to require that rdp's login requests be padded with additional bytes to make them longer than the login reponse. thus the rdp server would act to attenuate the attack, not amplify it. this would satisfy vixie's goal: attenuation also has to be a first-order goal—we must make it more attractive for attackers to send their packets directly to their victims than to bounce them off a ddos attenuator. the protocol could specify that the padding bytes not be random, but be computed from the login request parameters by some algorithm making them relatively expensive to generate but cheap to verify (cf. proof-of-work). this would not significantly impact legitimate clients, who issue login requests infrequently, but would increase the cost of using the rdp server to disguise the source of the attack. posted by david. at : am labels: bitcoin, networking, security no comments: post a comment newer post older post home subscribe to: post comments (atom) blog rules posts and comments are copyright of their respective authors who, by posting or commenting, license their work under a creative commons attribution-share alike . united states license. off-topic or unsuitable comments will be deleted. dshr dshr in anwr recent comments full comments blog archive ▼ ( ) ► february ( ) ▼ january ( ) effort balancing and rate limits isp monopolies the bitcoin "price" two million page views! the new oldweb.today ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► july ( ) ► june ( ) ► february ( ) ► ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) lockss system has permission to collect, preserve, and serve this archival unit. simple theme. powered by blogger. jodischneider.com/blog reading, technology, stray thoughts blog about categories argumentative discussions books and reading computer science firefox future of publishing higher education information ecosystem information quality lab news intellectual freedom ios: ipad, iphone, etc. library and information science math old newspapers phd diary programming random thoughts reviews scholarly communication semantic web social semantic web social web uncategorized search paid graduate hourly research position at uiuc for spring december rd, by jodi jodi schneider’s information quality lab (http://infoqualitylab.org) seeks a graduate hourly student for a research project on bias in citation networks. biased citation benefits authors in the short-term by bolstering grants and papers, making them more easily accepted. however, it can have severe negative consequences for scientific inquiry. our goal is to find quantitative measures of network structure that can indicate the existence of citation bias. this job starts january , . pay depending on experience (master’s students start at $ /hour). optionally, the student can also take a graduate independent study course (generally - credits is or info ). apply on handshake responsibilities will include: assist in the development of algorithms to simulate an unbiased network carry out statistical significance tests for candidate network structure measures attend weekly meetings assist with manuscript and grant preparation required skills proficiency in python or r demonstrated ability to systematically approach a simulation or modeling problem statistical knowledge, such as developed in a course on mathematical statistics and probability (e.g. stat statistics and probability i https://courses.illinois.edu/schedule/ /spring/stat/ ) preferred skills knowledge of stochastic processes experience with simulation knowledge of random variate generation and selection of input probability distribution knowledge of network analysis may have taken classes such as stat stochastic processes (https://courses.illinois.edu/schedule/ /spring/stat/ ) or ie advanced topics in stochastic processes & applications (https://courses.illinois.edu/schedule/ /fall/ie/ ) more information: https://ischool.illinois.edu/people/jodi-schneider http://infoqualitylab.org application deadline: monday december th. apply on handshake with the following application materials: resume transcript – such as free university of illinois academic history from banner self-service (https://apps.uillinois.edu, click “registration & records”, “student records and transcripts”, “view academic history”, choose “web academic history”) cover letter: just provide short answers to the following two questions: ) why are you interested in this particular project? ) what past experience do you have that is related to this project? tags: citation bias, jobs, network analysis, statistical modeling posted in information quality lab news | comments ( ) avoiding long-haul air travel during the covid- pandemic october th, by jodi i would not recommend long-haul air travel at this time. an epidemiological study of a . hour flight from the middle east to ireland concluded that groups ( people), traveling from continents in four groups, who used separate airport lounges, were likely infected in flight. the flight had % occupancy ( passengers/ seats; crew) and took place in summer . (note: i am not an epidemiologist.) the study (published open access): murphy nicola, boland máirín, bambury niamh, fitzgerald margaret, comerford liz, dever niamh, o’sullivan margaret b, petty-saphon naomi, kiernan regina, jensen mette, o’connor lois. a large national outbreak of covid- linked to air travel, ireland, summer . euro surveill. ; ( ):pii= . https://doi.org/ . / - .es. . . . irish news sites including rte and the irish times also covered the paper. figure from “a large national outbreak of covid- linked to air travel, ireland, summer ” https://doi.org/ . / - .es. . . . caption in original “passenger seating diagram on flight, ireland, summer (n= passengers)” “numbers on the seats indicate the flight groups – .” the age of the flight cases ranged from to years with a median age of years. twelve of flight cases and almost three quarters ( / ) of the non-flight cases were symptomatic. after the flight, the earliest onset of symptoms occurred days after arrival, and the latest case in the entire outbreak occurred days after the flight. of symptomatic flight cases, symptoms reported included cough (n = ), coryza (n = ), fever (n = ) and sore throat (n = ), and six reported loss of taste or smell. no symptoms were reported for one flight case. a mask was worn during the flight by nine flight cases, not worn by one (a child), and unknown for three. murphy nicola, boland máirín, bambury niamh, fitzgerald margaret, comerford liz, dever niamh, o’sullivan margaret b, petty-saphon naomi, kiernan regina, jensen mette, o’connor lois. a large national outbreak of covid- linked to air travel, ireland, summer . euro surveill. ; ( ):pii= . https://doi.org/ . / - .es. . . . (notes to figure caption) “it is interesting that four of the flight cases were not seated next to any other positive case, had no contact in the transit lounge, wore face masks in-flight and would not be deemed close contacts under current guidance from the european centre for disease prevention and control (ecdc) [ ].” murphy nicola, boland máirín, bambury niamh, fitzgerald margaret, comerford liz, dever niamh, o’sullivan margaret b, petty-saphon naomi, kiernan regina, jensen mette, o’connor lois. a large national outbreak of covid- linked to air travel, ireland, summer . euro surveill. ; ( ):pii= . https://doi.org/ . / - .es. . . . “the source case is not known. the first two cases in group became symptomatic within h of the flight, and covid- was confirmed in three, including an asymptomatic case from this group in region a within days of the flight. thirteen secondary cases and one tertiary case were later linked to these cases. two cases from flight group were notified separately in region a with one subsequent secondary family case, followed by three further flight cases notified from region b in two separate family units (flight groups and ). these eight cases had commenced their journey from the same continent and had some social contact before the flight. the close family member of a group case seated next to the case had tested positive abroad weeks before, and negative after the flight. flight group was a household group of which three cases were notified in region c and one case in region d. these cases had no social or airport lounge link with groups or pre-flight and were not seated within two rows of them. their journey origin was from a different continent. a further case (flight group ) had started the journey from a third continent, had no social or lounge association with other cases and was eated in the same row as passengers from group . three household contacts and a visitor of flight group became confirmed cases. one affected contact travelled to region e, staying in shared accommodation with others; of these became cases (attack rate %) notified in regions a, b, c, d, e and f, with two cases of quaternary spread.” murphy nicola, boland máirín, bambury niamh, fitzgerald margaret, comerford liz, dever niamh, o’sullivan margaret b, petty-saphon naomi, kiernan regina, jensen mette, o’connor lois. a large national outbreak of covid- linked to air travel, ireland, summer . euro surveill. ; ( ):pii= . https://doi.org/ . / - .es. . . . “in-flight transmission is a plausible exposure for cases in group and group given seating arrangements and onset dates. one case could hypothetically have acquired the virus as a close household contact of a previous positive case, with confirmed case onset date less than two incubation periods before the flight, and symptom onset in the flight case was h after the flight. in-flight transmission was the only common exposure for four other cases (flight groups and ) with date of onset within four days of the flight in all but the possible tertiary case. this case from group developed symptoms nine days after the flight and so may have acquired the infection in-flight or possibly after the flight through transmission within the household.” murphy nicola, boland máirín, bambury niamh, fitzgerald margaret, comerford liz, dever niamh, o’sullivan margaret b, petty-saphon naomi, kiernan regina, jensen mette, o’connor lois. a large national outbreak of covid- linked to air travel, ireland, summer . euro surveill. ; ( ):pii= . https://doi.org/ . / - .es. . . . “genomic sequencing for cases travelling from three different continents strongly supports the epidemiological transmission hypothesis of a point source for this outbreak. the ability of genomics to resolve transmission events may increase as the virus evolves and accumulates greater diversity [ ].” murphy nicola, boland máirín, bambury niamh, fitzgerald margaret, comerford liz, dever niamh, o’sullivan margaret b, petty-saphon naomi, kiernan regina, jensen mette, o’connor lois. a large national outbreak of covid- linked to air travel, ireland, summer . euro surveill. ; ( ):pii= . https://doi.org/ . / - .es. . . . authors note that a large percentage of the flight passengers were infected: “we calculated high attack rates, ranging plausibly from . % to . % despite low flight occupancy and lack of passenger proximity on-board.” murphy nicola, boland máirín, bambury niamh, fitzgerald margaret, comerford liz, dever niamh, o’sullivan margaret b, petty-saphon naomi, kiernan regina, jensen mette, o’connor lois. a large national outbreak of covid- linked to air travel, ireland, summer . euro surveill. ; ( ):pii= . https://doi.org/ . / - .es. . . . among the reasons for the uncertainty of this range is that “ flight passengers could not be contacted and were consequently not tested.” (a twelfth passenger “declined testing”.) there is also some inherent uncertainty due to incubation period and possibility of “transmission within the household”, especially after the flight; authors note that “exposure possibilities for flight cases include in-flight, during overnight transfer/pre-flight or unknown acquisition before the flight.” beyond the people on the flight, cases spread to several social groups, across “six of the eight different health regions (regions a–h) throughout the republic of ireland”. flight groups and started their travel from one continent; flight group from another; flight group from a third continent. figure from “a large national outbreak of covid- linked to air travel, ireland, summer ” https://doi.org/ . / - .es. . . . caption in original: “diagram of chains of transmission, flight-related covid- cases, ireland, summer (n= )” tags: air travel, attack rate, covid- , covid , epidemiology, flights, flying, ireland, middle east, pandemic posted in random thoughts | comments ( ) paid undergraduate research position at uiuc for fall & spring august th, by jodi university of illinois undergraduates are encouraged to apply for a position in my lab. i particularly welcome applications from students in the new ischool bs/is degree or in the university-wide informatics minor. while i only have paid position open, i also supervise unpaid independent study projects. dr. jodi schneider and the information quality lab seek undergraduate research assistants for % remote work. past students have published research articles, presented posters, earned independent study credit, james scholar research credit, etc. one paid position in news analytics/data science for assessing the impact of media polarization on public health emergencies, funded by the cline center for advanced research in the social sciences. ( hrs/week at $ . /hour + possible independent study – % remote work). covid- news analytics: we seek to understand how public health emergencies are reported and to assess the polarization and politicization of the u.s. news coverage. you will be responsible for testing and improving search parameters, investigating contextual information such as media bias and media circulation, using text mining and data science, and close reading of sample texts. you will work closely with a student who has worked on the opioid crisis – see the past work following poster (try the link twice – you have to log in with an illinois netid): https://compass g.illinois.edu/webapps/discussionboard/do/message?action=list_messages&course_id=_ _ &nav=discussion_board&conf_id=_ _ &forum_id=_ _ &message_id=_ _ applications should be submitted here: https://forms.illinois.edu/sec/ deadline: pm central time sunday august , tags: covid , data science, health controversies, jobs, media polarization, news analytics, research experiences for undergraduates, undergraduate research posted in information quality lab news | comments ( ) #shutdownstem #strike blacklives #shutdownacademia june th, by jodi i greatly appreciated receiving messages from senior people about their participation in the june th #shutdownstem #strike blacklives #shutdownacademia. in that spirit, i am sharing my email bounce message for tomorrow, and the message i sent to my research lab. email bounce: i am not available by email today: this june th is a day of action about understanding and addressing racism, and its impact on the academy, and on stem. -jodi email to my research lab wednesday is a day of action about understanding and addressing racism, and its impact on the academy, and on stem. i strongly encourage you to use tomorrow for this purpose. specifically, i invite you to think about what undoing racism – moving towards antiracism – means, and what you can do. one single day, by itself, will not cure racism; but identifying what we can do on an ongoing basis, and taking those actions day after day – that can and will have an impact. and, if racism is vivid in your daily life, make #shutdownstem a day of rest. if tomorrow doesn’t suit, i encourage you to reserve a day over the course of the next week, to replace your everyday duties. what does taking this time actually mean? it means scheduling a dedicated block of time to learn more; rescheduling meetings; shutting down your email; reading books and articles and watching videos; and taking time to reflect on recent events and the stress that they cause every single person in our community. what am i doing personally? i’ve cancelled meetings tomorrow, and set an email bounce. i will spend part of the day to think more seriously about what real antiracist action looks like from my position, as a white female academic. this week i will also be using time to re-read white fragility, to finish dreamland burning (a ya novel about the tulsa race riot), and to investigate how to bring bystander training to the ischool. i will also be thinking about the relationship of racism to other forms of oppression – classism, sexism, homophobia, transphobia, xenophobia. if you are looking for readings of your own, i can point to a list curated by an anti-racism task force: https://idea.illinois.edu/education for basic information, #shutdownstem #strike blacklives #shutdownacademia website: https://www.shutdownstem.com physicists’ particles for justice: https://www.particlesforjustice.org -jodi tags: #shutdownacademia, #shutdownstem, #strike blacklives, email posted in random thoughts | comments ( ) qotd: storytelling in protest and politics march th, by jodi i recently read francesca polletta‘s book it was like a fever: storytelling in protest and politics ( , university of chicago press). i recommend it! it will appeal to researchers interested in topics such as narrative, strategic communication, (narrative) argumentation, or epistemology (here, of narrative). parts may also interest activists. the book’s case studies are drawn from the student nonviolent coordinating committee (sncc) (chapters & ); online deliberation about the / memorial (listening to the city, summer ) (chapter ); women’s stories in law (including, powerfully, battered women who had killed their abusers, and the challenges in making their stories understandable) (chapter ); references to martin luther king by african american congressmen (in the congressional record) and by “leading back political figures who were not serving as elected or appointed officials” (chapter ). several are extended from work polletta previously published from through (see page xiii for citations). the conclusion—”conclusion: folk wisdom and scholarly tales” (pages - )—takes up several topics, starting with canonicity, interpretability, ambivalence. i especially plan to go back to the last two sections: “scholars telling stories” (pages - )—about narrative and storytelling in analysts’ telling of events—and “towards a sociology of discursive forms” (pages - )—about investigating the beliefs and conventions of narrative and its institutional conventions (and relating those to conventions of other “discursive forms” such as interviews). these set forward a research agenda likely useful to other scholars interested in digging in further. these are foreshadowed a bit in the introduction (“why stories matter”) which, among other things, sets out the goal of developing “a sociology of storytelling”. a few quotes i noted—may give you the flavor of the book: page : “but telling stories also carries risks. people with unfamiliar experiences have found those experiences assimilated to canonical plot lines and misheard as a result. conventional expectations about how stories work, when they are true, and when they are appropriate have also operated to diminish the impact of otherwise potent political stories. for the abused women whom juries disbelieved because their stories had changed in small details since their first traumatized [p ] call to police, storytelling has not been especially effective. nor was it effective for the citizen forum participants who did not say what it was like to search fruitlessly for affordable housing because discussions of housing were seen as the wrong place in which to tell stories.” pages - : “so which is it? is narrative fundamentally subversive or hegemonic? both. as a rhetorical form, narrative is equipped to puncture reigning verities and to uphold them. at times, it seems as if most of the stories in circulation are subtly or not so subtly defying authorities; at others as if the most effective storytelling is done by authorities. to make it more complicated, sometimes authorities unintentionally undercut their own authority when they tell stories. and even more paradoxically, undercutting their authority by way of a titillating but politically inconsequential story may actually strengthen it. dissenters, for their part, may find their stories misread in ways that support the very institutions that are challenging….”for those interested in the relations between storytelling, protest, and politics, this all suggests two analytical tasks. one is to identify the features of narrative that allow it to [p ] achieve certain rhetorical effects. the other is to identify the social conditions in which those rhetorical effects are likely to be politically consequential. the surprise is that scholars of political processes have devoted so little attention to either task.” pages - – “so institutional conventions of storytelling influence what people can do strategically with stories. in the previously pages, i have described the narrative conventions that operate in legal adjudication, media reporting, television talk shows, congressional debate, and public deliberation. sociolinguists have documented such conventions in other settings: in medical intake interviews, for example, parole hearings, and jury deliberations. one could certainly generate a catalogue of the institutional conventions of storytelling. to some extent, those conventions reflect the peculiarities of the institution as it has developed historically. they also serve practical functions; some explicit, others less so. i have argued that the lines institutions draw between suitable and unsuitable occasions for storytelling or for certain kinds of stories serve to legitimate the institution.” [specific examples follow] ….”as these examples suggest, while institutions have different conventions of storytelling, storytelling does some of the same work in many institutions. it does so because of broadly shared assumptions about narrative’s epistemological status. stories are generally thought to be more affecting by less authoritative than analysis, in part because narrative is associated with women rather than men, the private sphere rather than the public one, and custom rather than law. of course, conventions of storytelling and the symbolic associations behind them are neither unitary nor fixed. nor are they likely to be uniformly advantageous for those in power and disadvantageous for those without it. narrative’s alignment [ ] along the oppositions i noted is complex. for example, as i showed in chapter , americans’ skepticism of expert authority gives those telling stories clout. in other words, we may contrast science with folklore (with science seen as much more credible), but we may also contrast it with common sense (with science seen as less credible). contrary to the lamentation of some media critics and activists, when disadvantaged groups have told personal stories to the press and on television talk shows, they have been able to draw attention not only to their own victimization but to the social forces responsible for it.“ tags: congressional record, francesca polletta, listening to the city, martin luther king, narrative, qotd, sncc, storytelling, strategic communication, student nonviolent coordinating committee posted in argumentative discussions, books and reading | comments ( ) knowledge graphs: an aggregation of definitions march rd, by jodi i am not aware of a consensus definition of knowledge graph. i’ve been discussing this for awhile with liliana giusti serra, and the topic came up again with my fellow organizers of the knowledge graph session at us ts as we prepare for a panel. i’ve proposed the following main features: rdf-compatible, has a defined schema (usually an owl ontology) items are linked internally may be a private enterprise dataset (e.g. not necessarily openly available for external linking) or publicly available covers one or more domains below are some quotes. i’d be curious to hear of other definitions, especially if you think there’s a consensus definition i’m just not aware of. “a knowledge graph consists of a set of interconnected typed entities and their attributes.” jose manuel gomez-perez, jeff z. pan, guido vetere and honghan wu. “enterprise knowledge graph: an introduction.” in exploiting linked data and knowledge graphs in large organisations. springer. part of the whole book: http://link.springer.com/ . / - - - - “a knowledge graph is a structured dataset that is compatible with the rdf data model and has an (owl) ontology as its schema. a knowledge graph is not necessarily linked to external knowledge graphs; however, entities in the knowledge graph usually have type information, defined in its ontology, which is useful for providing contextual information about such entities. knowledge graphs are expected to be reliable, of high quality, of high accessibility and providing end user oriented information services.” boris villazon-terrazas, nuria garcia-santa, yuan ren, alessandro faraotti, honghan wu, yuting zhao, guido vetere and jeff z. pan . “knowledge graphs: foundations”. in exploiting linked data and knowledge graphs in large organisations. springer. part of the whole book: http://link.springer.com/ . / - - - - “the term knowledge graph was coined by google in , referring to their use of semantic knowledge in web search (“things, not strings”), and is recently also used to refer to semantic web knowledge bases such as dbpedia or yago. from a broader perspective, any graph-based representation of some knowledge could be considered a knowledge graph (this would include any kind of rdf dataset, as well as description logic ontologies). however, there is no common definition about what a knowledge graph is and what it is not. instead of attempting a formal definition of what a knowledge graph is, we restrict ourselves to a minimum set of characteristics of knowledge graphs, which we use to tell knowledge graphs from other collections of knowledge which we would not consider as knowledge graphs. a knowledge graph mainly describes real world entities and their interrelations, organized in a graph. defines possible classes and relations of entities in a schema. allows for potentially interrelating arbitrary entities with each other. covers various topical domains.” paulheim, h. ( ). knowledge graph refinement: a survey of approaches and evaluation methods. semantic web, ( ), - . http://www.semantic-web-journal.net/system/files/swj .pdf “isi’s center on knowledge graphs research group combines artificial intelligence, the semantic web, and database integration techniques to solve complex information integration problems. we leverage general research techniques across information-intensive disciplines, including medical informatics, geospatial data integration and the social web.” http://usc-isi-i .github.io/home/ just as i was “finalizing” my list to send to colleagues, i found a poster all about definitions: ehrlinger, l., & wöß, w. ( ). towards a definition of knowledge graphs. semantics (posters, demos, success), . http://ceur-ws.org/vol- /paper .pdf its table : selected definitions of knowledge graph has the following definitions (for citations see that paper) “a knowledge graph (i) mainly describes real world entities and their interrelations, organized in a graph, (ii) defines possible classes and relations of entities in a schema, (iii) allows for potentially interrelating arbitrary entities with each other and (iv) covers various topical domains.” paulheim [ ] “knowledge graphs are large networks of entities, their semantic types, properties, and relationships between entities.” journal of web semantics [ ] “knowledge graphs could be envisaged as a network of all kind things which are relevant to a specific domain or to an organization. they are not limited to abstract concepts and relations but can also contain instances of things like documents and datasets.” semantic web company [ ] “we define a knowledge graph as an rdf graph. an rdf graph consists of a set of rdf triples where each rdf triple (s, p, o) is an ordered set of the following rdf terms: a subjects∈u∪b,apredicatep∈u,andanobjectu∪b∪l. anrdftermiseithera uri u ∈ u, a blank node b ∈ b, or a literal l ∈ l.” färber et al. [ ] “[…] systems exist, […], which use a variety of techniques to extract new knowledge, in the form of facts, from the web. these facts are interrelated, and hence, recently this extracted knowledge has been referred to as a knowledge graph.” pujara et al. [ ] “a knowledge graph is a graph that models semantic knowledge, where each node is a real-world concept, and each edge represents a relationship between two concepts” fang, y., kuan, k., lin, j., tan, c., & chandrasekhar, v. ( ). object detection meets knowledge graphs. https://oar.a-star.edu.sg/jspui/handle/ / “things not strings” – google https://googleblog.blogspot.com/ / /introducing-knowledge-graph-things-not.html tags: knowledge graph, knowledge representation, quotations posted in information ecosystem, semantic web | comments ( ) qotd: doing more requires thinking less december st, by jodi by the aid of symbolism, we can make transitions in reasoning almost mechanically by the eye which would otherwise call into play the higher faculties of the brain. …civilization advances by extending the number of important operations that we can perform without thinking about them. operations of thought are like cavalry charges in a battle — they are strictly limited in number, they require fresh horses, and must only be made at decisive moments. one very important property for symbolism to possess is that it should be concise, so as to be visible at one glance of the eye and be rapidly written. – whitehead, a.n. ( ). an introduction to mathematics, chapter , “the symbolism of mathematics” (page in this version) ht to santiago nuñez-corrales (illinois page for santiago nuñez-corrales, linkedin for santiago núñez-corrales) who used part of this quote in a conceptual foundations group talk, nov . from my point of view, this is why memorizing multiplication tables is not now irrelevant; why new words for concepts are important; and underlies a lot of scientific advancement. tags: cavalry, modes of thought, qotd, symbolism posted in information ecosystem, random thoughts | comments ( ) qotd: sally jackson on how disagreement makes arguments more explicit june th, by jodi sally jackson explicates the notion of the “disagreement space” in a new topoi article: “a position that remains in doubt remains in need of defense” “the most important theoretical consequence of seeing argumentation as a system for management of disagreement is a reversal of perspective on what arguments accomplish. are arguments the means by which conclusions are built up from established premises? or are they the means by which participants drill down from disagreements to locate how it is that they and others have arrived at incompatible positions? a view of argumentation as a process of drilling down from disagreements suggests that arguers themselves do not simply point to the reasons they hold for a particular standpoint, but sometimes discover where their own beliefs come from, under questioning by others who do not share their beliefs. a logical analysis of another’s argument nearly always involves first making the argument more explicit, attributing more to the author than was actually said. this is a familiar enough problem for analysts; my point is that it is also a pervasive problem for participants, who may feel intuitively that something is seriously wrong in what someone else has said but need a way to pinpoint exactly what. getting beliefs externalized is not a precondition for argument, but one of its possible outcomes.” from sally jackson’s reason-giving and the natural normativity of argumentation. the original treatment of disagreement space is cited to a book chapter revising an issa paper , somewhat harder to get one’s hands on. p , sally jackson. reason-giving and the natural normativity of argumentation. topoi. online first. http://doi.org/ . /s - - - [↩] p , sally jackson. reason-giving and the natural normativity of argumentation. topoi. online first. http://doi.org/ . /s - - - [↩] sally jackson. reason-giving and the natural normativity of argumentation. topoi. online first. http://doi.org/ . /s - - - [↩] jackson s ( ) “virtual standpoints” and the pragmatics of conversational argument. in: van eemeren fh, grootendorst r, blair ja, willard ca (eds) argument illuminated. international centre for the study of argumentation, amsterdam, pp. – [↩] tags: argumentation, argumentation norms, disagreement space posted in argumentative discussions | comments ( ) qotd: working out scientific insights on paper, lavoisier case study july th, by jodi …language does do much of our thinking for us, even in the sciences, and rather than being an unfortunate contamination, its influence has been productive historically, helping individual thinkers generate concepts and theories that can then be put to the test. the case made here for the constitutive power of figures [of speech] per se supports the general point made by f.l. holmes in a lecture addressed to the history of science society in . a distinguished historian of medicine and chemistry, holmes based his study of antoine lavoisier on the french chemist’s laboratory notebooks. he later examined drafts of lavoisier’s published papers and discovered that lavoisier wrote many versions of his papers and in the course of careful revisions gradually worked out the positions he eventually made public (holmes, ). holmes, whose goal as a historian is to reconstruct the careful pathways and fine structure of scientific insights, concluded from his study of lavoisier’s drafts we cannot always tell whether a thought that led him to modify a passage, recast an argument, or develop an alternative interpretation occurred while he was still engaged in writing what he subsequently altered, or immediately afterward, or after some interval during which he occupied himself with something else; but the timing is, i believe, less significant than the fact that the new developments were consequences of the effort to express ideas and marshall supporting information on paper ( ). – page xi of rhetorical figures in science by jeanne fahnestock, oxford university press, . she is quoting frederich l. holmes. . scientific writing and scientific discovery. isis : - . doi: . / as moore summarizes, lavoisier wrote at least six drafts of the paper over a period of at least six months. however, his theory of respiration did not appear until the fifth draft. clearly, lavoisier’s writing helped him refine and understand his ideas. moore, randy. language—a force that shapes science. journal of college science teaching . ( ): . http://www.jstor.org/stable/ (which i quoted in a review i wrote recently) fahnestock adds: “…holmes’s general point [is that] there are subtle interactions ‘between writing, thought, and operations in creative scientific activity’ ( ).” tags: lavoisier, revision, rhetoric of science, scientific communication, scientific writing posted in future of publishing, information ecosystem, scholarly communication | comments ( ) david liebovitz: achieving care transformation by infusing electronic health records with wisdom may st, by jodi today i am at the health data analytics summit. the title of the keynote talk is achieving care transformation by infusing electronic health records with wisdom. it’s a delight to hear from a medical informaticist: david m. liebovitz (publications in google scholar), md, facp, chief medical information officer, the university of chicago. he graduated from university of illinois in electrical engineering, making this a timely talk as the engineering-focused carle illinois college of medicine gets going. david liebovitz started with a discussion of the data problems — problem lists, medication lists, family history, rules, results, notes — which will be familiar to anyone using ehrs or working with ehr data. he draws attention also to the human problems — both in terms of provider “readiness” (e.g. their vision for population-level health) as well as about “current expectations”. (an example of such an expectation is a “main clinician satisfier” he closed with: u chicago is about to turn on outbound faxing from the ehr!) he mentioned also the importance of resilience. he mentioned customizing systems as a risk when the vendor makes upstream changes (this is not unique to healthcare but a threat to innovation and experimentation with information systems in other industries.) still, in managing the ehr, there is continual optimization, scored based on a number of factors. he mentioned: safety quality/patient experience regulatory/legal financial usability/productivity availability of alternative solutions as well as weighting for old requests. he emphasized the complexity of healthcare in several ways: “nobody knew that healthcare could be so complicated.” – potus showing the medicare readmissions adjustment factors pharmacy pricing, an image (showing kickbacks among other things) from “prices that are too high”, chapter , the healthcare imperative: lowering costs and improving outcomes: workshop series summary ( ) national academies press doi: . / an image from “prices that are too high”, chapter , the healthcare imperative: lowering costs and improving outcomes: workshop series summary ( ) icosystem’s diagram of the complexity of the healthcare system icosystem – complexity of the healthcare system another complexity is the modest impact of medical care compared to other factors such as the impact of socioeconomic and political context on equity in health and well-being (see the who image below). for instance, there is a large impact of health behaviors, which “happen in larger social contexts.” (see the relative contribution of multiple determinants to health, august , , health policy briefs) solar o, irwin a. a conceptual framework for action on the social determinants of health. social determinants of health discussion paper (policy and practice). given this complexity, david liebovitz stresses that we need to start with the right model, “simultaneously improving population health, improving the patient experience of care, and reducing per capita cost”. (see stiefel m, nolan k. a guide to measuring the triple aim: population health, experience of care, and per capita cost. ihi innovation series white paper. cambridge, massachusetts: institute for healthcare improvement; ). table from stiefel m, nolan k. a guide to measuring the triple aim: population health, experience of care, and per capita cost. ihi innovation series white paper. cambridge, massachusetts: institute for healthcare improvement; . given the modest impact of medical care, and of data, he suggests that we should choose the right outcomes. david liebovitz says that “not enough attention has been paid to usability”; i completely agree and suggest that information scientists, human factors engineeers, and cognitive ergonomists help mainstream medical informaticists fill this gap. he put up jakob nielsen’s usability heuristics for user interface design a vivid example is whether a patient’s resuscitation preferences are shown (which seems to depend on the particular ehr screen): the system doesn’t highlight where we are in the system. for providers, he says user control and freedom are very important. he suggests that there are only a few key tasks. a provider should be able to do any of these things wherever they are in the chart: put a note order something send a message similarly, ehr should support recognition (“how do i admit a patient again?”) rather than requiring recall. meanwhile, on the decision support side he highlights the (well-known) problems around interruptions by saying that speed is everything and changing direction is much easier than stopping. here he draws on some of his own work, describing what he calls a “diagnostic process aware workflow” david liebovitz. next steps for electronic health records to improve the diagnostic process. diagnosis ( ) - . doi: . /dx- - can we predict x better? yes, he says (for instance pointing to table of “can machine-learning improve cardiovascular risk prediction using routine clinical data?” and its machine learning analysis of over , patients, based on variables chosen from previous guidelines and expert-informed selection–generating further support for aspects such as aloneness, access to resources, socio-economic status). but what’s really needed, he says, is to: predict the best next medical step, iteratively predict the best next lifestyle step, iteratively (and what to do about genes and epigenetic measures?) he shows an image of “all of our planes in the air” from flightaware, drawing the analogy that we want to work on “optimal patient trajectories” — predicting what are the “turbulent events” to avoid”. this is not without challenges. he points to three: data privacy (he suggests google deepmind and healthcare in an age of algorithms. powles, j. & hodson, h. health technol. ( ). doi: . /s - - - two sorts of mismatches between the current situation and where we want to go: for instance the source of data being from finance certain basic current clinician needs (e.g. that a main clinician satisfier is that uchicago is soon to turn on outbound faxing from their ehr — and that an ongoing source of dissatisfaction: managing volume of inbound faxes.) he closes suggesting that we: finish the basics address key slices of the spectrum descriptive/prescriptive begin the prescriptive journey: impact one trajectory at a time. tags: data analytics, electronic health records, healthcare systems, medical informatics posted in information ecosystem | comments ( ) « older entries recent posts paid graduate hourly research position at uiuc for spring avoiding long-haul air travel during the covid- pandemic paid undergraduate research position at uiuc for fall & spring #shutdownstem #strike blacklives #shutdownacademia qotd: storytelling in protest and politics monthly december october august june march meta log in valid xhtml xfn wordpress wordpress powers jodischneider.com/blog. layers theme designed by jai pandya. indigenous knowledge centres | torres strait island regional council skip to content go to front page search the site torres strait island regional council facebook page search form search home your council community grant program who we are media releases tenders & opportunities planning & decision making laws & obligations engagement & advocacy publications work with us financial services our work aged care animal management asset maintenance building child care disaster management environmental health health & wellbeing housing indigenous knowledge centres land management native title parks & gardens pests & diseases seaports, airports and roads torres strait islander flag waste management water management divisional administration our communities our geography governance history travelling to our communities art centres torres strait history accommodation in our communities culture, protocols & heritage celebrations & holidays contact us contact info contact us form complaints bills & payments youmpla voice an autonomous, prosperous and sustainable community and council you are here home / our work indigenous knowledge centres indigenous knowledge centres - warraber.jpg tsirc's indigenous knowledge centres (ikc's) are a joint partnership with queensland's state library. the ikc partnership is guided by our community values, social identity, ownership and authority. our indigenous knowledge centres endeavour to deliver new technology, literacy and learning programs to empower our communities through shared learning experiences. we work with communities to preserve local knowledge and culture and heritage, to keep our culture strong for generations. indigenous knowledge centre services tsirc indigenous knowledge centres provide free services to the community including: entry to a centre a minimum of hours per week of centre services (including outreach services delivered outside a physical centre) staff to help you use a centre lending, reference and information services as required by your community internet access to library and information services access to the internet through public computers or devices, or via wi-fi services if you use your own device services and programs that promote knowledge, culture, language, preservation of materials, emerging technologies and literacy to your community equitable access to centre resources and programs, including customised programs for community members with special needs. sitemap privacy news & events artists design on our website torres strait weather publications scheme © torres strait island regional council find us on libraries and digital humanities · website for the adho libraries and dh sig adho sig where digital humanities and libraries meet home about lib + dh on github © . all rights reserved. welcome! the libraries and digital humanities special interest group (lib&dh sig) was established in by the international alliance of digital humanities organizations adho. with conveners from five different countries, the goals of this sig are: to offer advice and support to librarians engaged in digital humanities pursuits; to advocate for initiatives of interest and benefit to both libraries and digital humanities; to document how librarians and library-based units meet the many challenges, and take advantage of the many opportunities, of dh librarianship; to provide information about available resources and opportunities (e.g., training, funding) that encourage collaboration between dh scholars in a variety of roles, especially in libraries; to showcase the work of librarians engaged in the digital humanities; and to promote librarians’ perspectives and skills in the greater dh community worldwide. recent posts call for abstracts: dh & lib dh pre-conference “libraries as research partner in digital humanities” adho's new special interest group: libraries and dh! older newer biograd - google slides javascript isn't enabled in your browser, so this file can't be opened. enable and reload. some powerpoint features can't be displayed in google slides and will be dropped if you make changes view details biograd present share sign in the version of the browser you are using is no longer supported. please upgrade to a supported browser.dismiss file edit view help accessibility debug unsaved changes to drive see new changes accessibility view only html view of the presentation none recent uploads tagged code lib recent uploads tagged code lib img_ img_ img_ img_ img_ img_ img_ img_ img_ img_ img_ img_ img_ img_ img_ img_ img_ img_ img_ img_ use cases, software architecture and deployment patterns arkisto arkisto why arkisto case studies paradisec uts data grants uts cultural data use cases standards storage packaging identifiers tools data description data discovery data import presentations use cases, software architecture and deployment patterns there are multiple use cases for arkisto which we will document in the abstract, in addition to the specific case studies we’re already working on. due to its standards-based and extensible nature, arkisto can realise the goal of making data fair (findable, accessible, interoperable, re-usable). the (mythical) minimal arkisto platform the simplest possible arkisto platform deployment would be a repository with some objects in it. no portal, no data ingest or preservation service, eg: that might not seem very useful in itself but a standards-based repository is a great candidate for data preservation - it puts the i (interoperable) in fair data! it can also provide a basis for re-activation via services that can reuse the data by making findable and accessible. because of arkisto’s use of standards, the repository is the core to which services can be added. adding a web portal to make the data findable - the f in fair data - just add one of the arkisto portal tools - this requires some configuration but significantly less than building a service from scratch. data ingest pathways but how does data get into an ocfl repository? there are several patterns in production, in development and planned. a snapshot repository exporting a repository to ro-crate can be a matter of writing a script to interrogate a database, convert static files, or otherwise traverse an existing dataset. this pattern is used by the snapshot of expertnation - where we were given an xml file exported from heurist and used a script to convert that data to the ro-crate format. this ro-crate can in turn be deposited in a repository - in this case the uts data repository - and served via a portal, preserving the data while at the same time making it accessible. a growing cultural collection in paradisec, research teams add collections and items using bespoke tools that were written for the archive. this is transitioning to a model where datasets will be described using arkisto tools, particularly describo. field data capture data from sensors in the field is often streamed directly to some kind of database with or without portal services and interfaces. there are multiple possible arkisto deployment patterns in this situation. where practical, the uts eresearch team aims to take an approach that first keeps copies of any raw data files and preserves those. the team then builds databases and discovery portals from the raw data, although this is not always possible. this diagram shows an approximation of one current scenario we’re working with at uts: we are working to supply the research team in question with an ocfl repository, add a portal, and create additional data-capture pathways for genomics data and potentially more sensors. we could then to move on to adding analytical tools, such as a dashboard that shows plots of sensor readings. analytical and housekeeping tools so far on this page we have covered simplified views of arkisto deployment patterns with the repository at the centre, adding discovery portals and giving examples of data-acquisition pipelines (just scratching the surface of the possibilities). these things in themselves have value: making sure data is well described and as future proof as possible are vitally important but what can we do with data? ocfl + ro-crate tools having data in a standard-stack, with ocfl used to lay-out research data objects and ro-crate to describe them, means that it is possible to write simple programs that can interrogate a repository. that is, you don’t have to spend time understanding the organisation of each dataset. the same idea underpins arkisto’s web-portal tools: standardization reduces the cost of building. validators and preservation tools: there are not many of these around yet, but members of the arkisto community as well as the broader ocfl and ro-crate communities are working on these; expect to see preservation tools that work over ocfl repositories in particular. brute-force scripts: for moderate-sized repositories, it is possible to write a scripts to examine every object in a repository and to run analytics. for instance, it would be possible to visualise a number of years’ worth of sensor readings from a small number of sensors or to look for the geographical distribution of events in historical datasets. we’re working on several such use-cases at uts at the moment. adding databases and other indexes for larger-scale use, visiting every object in a repository can be inefficient. in these cases, using an index means that an analysis script can request all the data in a particular time-window or of a certain type - or any other query that the index supports. the arkisto portal tools are built around general purpose indexes that can do full-text and facetted searching, with the potential to support either human or machine interfaces. while the index engines used in our current portals are based on full-text search and metadata, we expect others to be built as needed by disciplines using, for example, sql databases or triple stores. analysis tool integration above, we have looked at how people or machines can access an arkisto platform deployment by querying the repository, either directly or via an index. however, there is a much larger opportunity in being able to integrate arkisto deployments with other tools in the research landscape. to take one example, text analysis is in great demand across a very wide range of disciplines. this hypothetical scenario shows the potential for a researcher to use a web portal to locate datasets which contain text and then send the whole results set to a an analysis platform, in this case an interactive jupyter notebook. arkisto already allows re-use of visualisation tools and viewers that can be embedded directly in a portal. we are planning a “widget store” that will enable researchers to choose and deploy a number of add-ons to the basic portal. institutional and discipline repositories one of the major use case deployment patterns for arkisto is to underpin an institutional data repository / archive function, see the uts data repository for an established example. in this use case, data is ingested into the repository via a research data management system which talks to the ocfl repository, not the portal. there is a clear separation of concerns: the portal’s job is to provide controlled access and search services via an index, the ocfl repository keeps version controlled data on disc, and the research data management system handles deposit of data. manual archiving at uts the research data management system in use is redbox - an open source platform for managing data across the research process from research data management planning (via research data management plans (rdmps)) to archiving and re-use of data. redbox has services for provisioning and/or tracking research workspaces, which are sites where research data collection and management. all of the data acquisition scenarios described above would qualify as research workspaces, as do file-shares on institutional storage or share-sync services such as cloudstor, as well as survey platforms, project management and version control systems such as gitlab and github. fetching data from research workspaces soon at uts researchers will be able to use the research data management system to select a workspace for archiving, add metadata as appropriate, and the system will deposit the data for them. publishing data the uts institutional data repository actually has two parts: an internal behind-the-firewall archive, with access control - to ensure that only authorized people can access data - and an external data portal for publicly accessible data. this architecture reduces the risk of data breaches by not allowing access through the firewall to sensitive or confidential data until secure tools are available to allow extra-institutional access. researchers can select a subset of an archived dataset to be published, or publish an entire dataset. a “bot” “notices” that a new public dataset is available and copies it to the public repository, where it will be indexed and made available through the data portal. note: there is no monolithic “repository application” that mediates all interactions with the file-base ocfl store but a set of services which operate independently. this does mean that processes must be in place to ensure that there is not file-contention, with two bits of software trying to update an object at the same time. none ~ , organizations downloaded backdoor planted by cozy bear hackers | ars technica skip to main content biz & it tech science policy cars gaming & culture store forums subscribe close navigate store subscribe videos features reviews rss feeds mobile site about ars staff directory contact us advertise with ars reprints filter by topic biz & it tech science policy cars gaming & culture store forums settings front page layout grid list site theme black on white white on black sign in comment activity sign up or login to join the discussions! stay logged in | having trouble? sign up to comment and more sign up in-the-wild — ~ , organizations downloaded backdoor planted by cozy bear hackers russia-backed hackers use supply chain attack to infect public and private organizations. dan goodin - dec , : pm utc enlarge getty images reader comments with posters participating share this story share on facebook share on twitter share on reddit about , organizations around the world downloaded network management tools that contained a backdoor that a nation state used to install malware in organizations the used the software, the tools provider, solarwinds, said on monday. the disclosure from austin, texas-based solarwinds, came a day after the us government revealed a major security breach hitting federal agencies and private companies. the us departments of treasury, commerce, and homeland security departments were among the federal agencies on the receiving end of hacks that gave access to email and other sensitive resources, reuters reported. federal agencies using the software were instructed on sunday to disconnect systems that run the software and perform a forensic analysis of their networks. further reading premiere security firm fireeye says it was breached by nation-state hackers security firm fireeye, which last week disclosed a serious breach of its own network, said that hackers backed by a nation-state compromised a solarwinds software update mechanism and then used it to infect selected customers who installed a backdoored version of the company’s orion network management tool. the backdoor infected customers who installed an update from march to june of this year, solarwinds said in a document filed on monday with the securities and exchange commission. the implant “was introduced as a result of a compromise of the orion software build system and was not present in the source code repository of the orion products,” monday's filing said. solarwinds, which said it has about , orion customers, put the number of affected customers at about , . stealing the master keys several factors made orion an ideal stepping stone into networks coveted by russia-backed hackers, who over the past decade have become one of the most formidable threats to us cyber security. mike chapple, a teaching professor of it, analytics, and operations at the university of notre dame, said the tool is widely used to manage routers, switches, and other network devices inside large organizations. the level of privileged access coupled with the number of networks exposed made orion the perfect tool for the hackers to exploit. “solarwinds by its nature has very privileged access to other parts of your infrastructure,” chapple, a former computer scientist at the national security agency, said in an interview. “you can think of solarwinds as having the master keys to your network, and if you’re able to compromise that type of tool, you’re able to use those types of keys to gain access to other parts of the network. by compromising that, you have a key basically to unlock the network infrastructure of a large number of organizations.” advertisement further reading russian hackers hit us government using widespread supply chain attack the hacks are part of what the federal government and officials from fireeye, microsoft, and other private companies said was a widespread espionage campaign that a sophisticated threat actor was carrying out through a supply chain attack. in blog post fireeye published sunday night, the company said it uncovered a global intrusion campaign that used the backdoored solarwinds’ update mechanism as an initial entryway “into the networks of public and private organizations through the software supply chain.” publications—including the washington post and the new york times—cited unnamed government officials saying cozy bear, a hacking group believed to be part of the russian federal security service (fsb) was behind the compromises. “based on our analysis, we have now identified multiple organizations where we see indications of compromise dating back to the spring of , and we are in the process of notifying those organizations,” fireeye officials wrote. “our analysis indicates that these compromises are not self-propagating; each of the attacks require meticulous planning and manual interaction. our ongoing investigation uncovered this campaign, and we are sharing this information consistent with our standard practice.” in a separate post also published sunday night, fireeye added: “fireeye has uncovered a widespread campaign, that we are tracking as unc . the actors behind this campaign gained access to numerous public and private organizations around the world. they gained access to victims via trojanized updates to solarwind’s orion it monitoring and management software. this campaign may have begun as early as spring and is currently ongoing. post compromise activity following this supply chain compromise has included lateral movement and data theft. the campaign is the work of a highly skilled actor and the operation was conducted with significant operational security.” fireeye went on to say that a digitally signed component of the orion framework contained a backdoor that communicates with hacker-controlled servers. the backdoor, planted in the windows dynamic link library file solarwinds.orion.core.businesslayer.dll, was written to remain stealthy, both by remaining dormant for a couple weeks and then blending in with legitimate solarwinds data traffic. fireeye researchers wrote: the trojanized update file is a standard windows installer patch file that includes compressed resources associated with the update, including the trojanized solarwinds.orion.core.businesslayer.dll component. once the update is installed, the malicious dll will be loaded by the legitimate solarwinds.businesslayerhost.exe or solarwinds.businesslayerhostx .exe (depending on system configuration). after a dormant period of up to two weeks, the malware will attempt to resolve a subdomain of avsvmcloud[.]com. the dns response will return a cname record that points to a command and control (c ) domain. the c traffic to the malicious domains is designed to mimic normal solarwinds api communications. the list of known malicious infrastructure is available on fireeye’s github page. burrowing in further the orion backdoor, which fireeye is calling sunburst and microsoft calls solorigate, gave the hackers the limited but crucial access to internal network devices. the hackers then used other techniques to burrow further. according to microsoft, the hackers then stole signing certificates that allowed them to impersonate any of a target’s existing users and accounts through the security assertion markup language. typically abbreviated as saml, the xml-based language provides a way for identity providers to exchange authentication and authorization data with service providers. advertisement microsoft’s advisory stated: an intrusion through malicious code in the solarwinds orion product. this results in the attacker gaining a foothold in the network, which the attacker can use to gain elevated credentials. microsoft defender now has detections for these files. also, see solarwinds security advisory. an intruder using administrative permissions acquired through an on-premises compromise to gain access to an organization’s trusted saml token-signing certificate. this enables them to forge saml tokens that impersonate any of the organization’s existing users and accounts, including highly privileged accounts. anomalous logins using the saml tokens created by a compromised token-signing certificate, which can be used against any on-premises resources (regardless of identity system or vendor) as well as against any cloud environment (regardless of vendor) because they have been configured to trust the certificate. because the saml tokens are signed with their own trusted certificate, the anomalies might be missed by the organization. using highly privileged accounts acquired through the technique above or other means, attackers may add their own credentials to existing application service principals, enabling them to call apis with the permission assigned to that application. supply chain attacks are among the hardest to counter because they rely on software that's already trusted and widely distributed. solarwinds' monday-morning filing suggests that cozy bear hackers had the ability to infect the networks about , of the company’s customers. it’s not yet clear how many of those eligible users were actually hacked. the department of homeland security’s cybersecurity infrastructure and infrastructure security agency has issued an emergency directive instructing federal agencies that use solarwinds products to analyze their networks for signs of compromise. fireeye’s post here lists a variety of signatures and other indicators admins can use to detect infections. promoted comments me ars tribunus militum jump to post uberdoward wrote: so... no integrity checks for the updates, hmm? seems like a sha hash check would have alerted this sooner... comes from the internet? it's not trusted until verified. are we sure it wasn't something ... internal? maybe i missed it but in the articles i've read so far it's not % clear to me if this was someone with the keys injecting their own updates from rando internet ... or if the bad code came ... from solarwinds traditional update processes... (whole ton of new questions then). posts | registered / / mistergrumps ars scholae palatinae et subscriptor jump to post i received this today at : am eastern: quote: we have just been made aware our systems experienced a highly sophisticated, manual supply chain attack on solarwinds® orion® platform software builds for versions . through . . . we have been advised this attack was likely conducted by an outside nation state and intended to be a narrow, extremely targeted, and manually executed incident, as opposed to a broad, system-wide attack. at this time, we are not aware of an impact to our solarwinds msp products including rmm and n-central. if you own a solarwinds orion product, we recommend you visit http://www.solarwinds.com/securityadvisory for more detailed information. if you have any immediate questions, please contact customer support at - - - or swisupport@solarwinds.com. security and trust in our software are the foundation of our commitment to our customers. thank you for your continued patience and partnership as we continue to work through this issue. thank you, john pagliuca | president | solarwinds msp posts | registered / / reader comments with posters participating share this story share on facebook share on twitter share on reddit dan goodin dan is the security editor at ars technica, which he joined in after working for the register, the associated press, bloomberg news, and other publications. email dan.goodin@arstechnica.com // twitter @dangoodin advertisement you must login or create an account to comment. channel ars technica ← previous story next story → related stories sponsored stories powered by today on ars store subscribe about us rss feeds view mobile site contact us staff advertise with us reprints newsletter signup join the ars orbital transmission mailing list to get weekly updates delivered to your inbox. sign me up → cnmn collection wired media group © condé nast. all rights reserved. use of and/or registration on any portion of this site constitutes acceptance of our user agreement (updated / / ) and privacy policy and cookie statement (updated / / ) and ars technica addendum (effective / / ). ars may earn compensation on sales from links on this site. read our affiliate link policy. your california privacy rights | do not sell my personal information the material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of condé nast. ad choices dshr's blog: blockchain briefing for dod dshr's blog i'm david rosenthal, and this is a place to discuss the work i'm doing in digital preservation. tuesday, july , blockchain briefing for dod i was asked to deliver blockchain: what's not to like? version . to a department of defense conference-call. i took the opportunity to update the talk, and expand it to include some of the "additional material" from the original, and from the podcast. below the fold, the text of the talk with links to the sources. the yellow boxes contain material that was on the slides but was not spoken. [slide ] it’s one of these things that if people say it often enough it starts to sound like something that could work, sadhbh mccarthy i'd like to thank jen snow for giving me the opportunity to talk about blockchain technology and cryptocurrencies. the text of my talk with links to the sources is up on my blog, so you don't need to take notes. there's been a supernova of hype about them. almost everything positive you have heard is paid advertising, and should be completely ignored. why am i any more credible? first, i'm retired. no-one is paying me to speak, and i have no investments in cryptocurrencies or blockchain companies. [slide ] this is not to diminish nakamoto's achievement but to point out that he stood on the shoulders of giants. indeed, by tracing the origins of the ideas in bitcoin, we can zero in on nakamoto's true leap of insight—the specific, complex way in which the underlying components are put together. bitcoin's academic pedigree, arvind narayanan and jeremy clark second, i've been writing skeptically about cryptocurrencies and blockchain technology for more than five years. what are my qualifications for such a long history of pontification? nearly sixteen years ago, about five years before satoshi nakamoto published the bitcoin protocol, a cryptocurrency based on a decentralized consensus mechanism using proof-of-work, my co-authors and i won a "best paper" award at the prestigious sosp workshop for a decentralized consensus mechanism using proof-of-work. it is the protocol underlying the lockss system. the originality of our work didn't lie in decentralization, distributed consensus, or proof-of-work. all of these were part of the nearly three decades of research and implementation leading up to the bitcoin protocol, as described by arvind narayanan and jeremy clark in bitcoin's academic pedigree. our work was original only in its application of these techniques to statistical fault tolerance; nakamoto's only in its application of them to preventing double-spending in cryptocurrencies. we're going to start by walking through the design of a system to perform some function, say monetary transactions, storing files, recording reviewers' contributions to academic communication, verifying archival content, whatever. my goal is to show you how the pieces fit together in such a way that the problems the technology encounters in practice aren't easily fixable; they are inherent in the underlying requirements. being of a naturally suspicious turn of mind, you don't want to trust any single central entity, but instead want a decentralized system. you place your trust in the consensus of a large number of entities, which will in effect vote on the state transitions of your system (the transactions, reviews, archival content, ...). you hope the good entities will out-vote the bad entities. in the jargon, the system is trustless (a misnomer). techniques using multiple voters to maintain the state of a system in the presence of unreliable and malign voters were first published in the byzantine generals problem by lamport et al in . alas, byzantine fault tolerance (bft) requires a central authority to authorize entities to take part. in the blockchain jargon, it is permissioned. you would rather let anyone interested take part, a permissionless system with no central control. [slide ] in the case of blockchain protocols, the mathematical and economic reasoning behind the safety of the consensus often relies crucially on the uncoordinated choice model, or the assumption that the game consists of many small actors that make decisions independently. the meaning of decentralization, vitalik buterin, co-founder of ethereum the security of your permissionless system depends upon the assumption of uncoordinated choice, the idea that each voter acts independently upon its own view of the system's state. if anyone can take part, your system is vulnerable to sybil attacks, in which an attacker creates many apparently independent voters who are actually under his sole control. if creating and maintaining a voter is free, anyone can win any vote they choose simply by creating enough sybil voters. [slide ] from a computer security perspective, the key thing to note ... is that the security of the blockchain is linear in the amount of expenditure on mining power, ... in contrast, in many other contexts investments in computer security yield convex returns (e.g., traditional uses of cryptography) ... analogously to how a lock on a door increases the security of a house by more than the cost of the lock. the economic limits of bitcoin and the blockchain, eric budish, booth school, university of chicago so creating and maintaining a voter has to be expensive. permissionless systems can defend against sybil attacks by requiring a vote to be accompanied by a proof of the expenditure of some resource. this is where proof-of-work comes in; a concept originated by cynthia dwork and moni naor in . in a bft system, the value of the next state of the system is that computed by the majority of the nodes. in a proof-of-work system such as bitcoin, the value of the next state of the system is that computed by the first node to solve a puzzle. there is no guarantee that any other node computed that value; bft is a consensus system whereas bitcoin-type systems select a winning node. proof-of-work is a random process, but at scale the probability of being selected is determined by how quickly you can compute hashes. the idea is that the good voters will spend more on hashing power, and thus compute more useless hashes, than the bad voters. [slide ] the blockchain trilemma much of the innovation in blockchain technology has been aimed at wresting power from centralised authorities or monopolies. unfortunately, the blockchain community’s utopian vision of a decentralised world is not without substantial costs. in recent research, we point out a ‘blockchain trilemma’ – it is impossible for any ledger to fully satisfy the three properties shown in [the diagram] simultaneously ... in particular, decentralisation has three main costs: waste of resources, scalability problems, and network externality inefficiencies. the economics of blockchains, markus k brunnermeier & joseph abadi, princeton brunnermeir and abadi's blockchain trilemma shows that a blockchain has to choose at most two of the following three attributes: correctness decentralization cost-efficiency obviously, your system needs the first two, so the third has to go. running a voter (mining in the jargon) in your system has to be expensive if the system is to be secure. no-one will do it unless they are rewarded. they can't be rewarded in "fiat currency", because that would need some central mechanism for paying them. so the reward has to come in the form of coins generated by the system itself, a cryptocurrency. to scale, permissionless systems need to be based on a cryptocurrency; the system's state transitions will need to include cryptocurrency transactions in addition to records of files, reviews, archival content, whatever. your system needs names for the parties to these transactions. there is no central authority handing out names, so the parties need to name themselves. as proposed by david chaum in they can do so by generating a public-private key pair, and using the public key as the name for the source or sink of each transaction. [slide ] we created a small bitcoin wallet, placed it on images in our honeyfarm, and set up monitoring routines to check for theft. two months later our monitor program triggered when someone stole our coins. this was not because our bitcoin was stolen from a honeypot, rather the graduate student who created the wallet maintained a copy and his account was compromised. if security experts can't safely keep cryptocurrencies on an internet-connected computer, nobody can. if bitcoin is the "internet of money," what does it say that it cannot be safely stored on an internet connected computer? risks of cryptocurrencies, nicholas weaver, u.c. berkeley in practice this is implemented in wallet software, which stores one or more key pairs for use in transactions. the public half of the pair is a pseudonym. unmasking the person behind the pseudonym turns out to be fairly easy in practice. the security of the system depends upon the user and the software keeping the private key secret. this can be difficult, as nicholas weaver's computer security group at berkeley discovered when their wallet was compromised and their bitcoins were stolen. [slide ] -yr bitcoin "price" history the capital and operational costs of running a miner include buying hardware, power, network bandwidth, staff time, etc. bitcoin's volatile "price", high transaction fees, low transaction throughput, and large proportion of failed transactions mean that almost no legal merchants accept payment in bitcoin or other cryptocurrency. thus one essential part of your system is one or more exchanges, at which the miners can sell their cryptocurrency rewards for the "fiat currency" they need to pay their bills. who is on the other side of those trades? the answer has to be speculators, betting that the "price" of the cryptocurrency will increase. thus a second essential part of your system is a general belief in the inevitable rise in "price" of the coins by which the miners are rewarded. if miners believe that the "price" will go down, they will sell their rewards immediately, a self-fulfilling prophesy. over time, permissionless blockchains require an inflow of speculative funds at an average rate greater than the current rate of mining rewards if the "price" is not to collapse. to maintain bitcoin's price at $ k would require an inflow of $ k/hour, or about $ b from now until the next reward halving around may th . [slide ] ether miners / / can we really say that the uncoordinated choice model is realistic when % of the bitcoin network’s mining power is well-coordinated enough to show up together at the same conference? the meaning of decentralization, vitalik buterin in order to spend enough to be secure, say $ k/hour, you need a lot of miners. it turns out that a third essential part of your system is a small number of “mining pools”. a year ago bitcoin had the equivalent of around m antminer s s, and a block time of minutes. each s , costing maybe $ k, could expect a reward about once every years. it would be obsolete in about a year, so only in would ever earn anything. to smooth out their income, miners join pools, contributing their mining power and receiving the corresponding fraction of the rewards earned by the pool. these pools have strong economies of scale, so successful cryptocurrencies end up with a majority of their mining power in - pools. each of the big pools can expect a reward every hour or so. these blockchains aren’t decentralized, but centralized around a few large pools. at multiple times in one mining pool controlled more than % of the bitcoin mining power. at almost all times since - pools have controlled the majority of the bitcoin mining power. currently two of them, with . % of the power, are controlled by bitmain, the dominant supplier of mining asics. with the advent of mining-as-a-service, % attacks have become endemic among the smaller alt-coins. the security of a blockchain depends upon the assumption that these few pools are not conspiring together outside the blockchain; an assumption that is impossible to verify in the real world (and by murphy's law is therefore false). [slide ] since then there have been other catastrophic bugs in these smart contracts, the biggest one in the parity ethereum wallet software ... the first bug enabled the mass theft from "multisignature" wallets, which supposedly required multiple independent cryptographic signatures on transfers as a way to prevent theft. fortunately, that bug caused limited damage because a good thief stole most of the money and then returned it to the victims. yet, the good news was limited as a subsequent bug rendered all of the new multisignature wallets permanently inaccessible, effectively destroying some $ m in notional value. this buggy code was largely written by gavin wood, the creator of the solidity programming language and one of the founders of ethereum. again, we have a situation where even an expert's efforts fell short. risks of cryptocurrencies, nicholas weaver, u.c. berkeley in practice the security of a blockchain depends not merely on the security of the protocol itself, but on the security of both the core software, and the wallets and exchanges used to store and trade its cryptocurrency. this ancillary software has bugs, such as last september's major vulnerability in bitcoin core, the parity wallet fiasco, the routine heists using vulnerabilities in exchange software, and the wallet that was sending user's pass-phrases to the google spell-checker over http. who doesn't need their pass-phrase spell-checked? recent game-theoretic analysis suggests that there are strong economic limits to the security of cryptocurrency-based blockchains. to guarantee safety, the total value of transactions in a block needs to be less than the value of the block reward, which kind of spoils the whole idea. your system needs an append-only data structure to which records of the transactions, files, reviews, archival content, whatever are appended. it would be bad if the miners could vote to re-write history, undoing these records. in the jargon, the system needs to be immutable (another misnomer). [slide ] merkle tree (source) the necessary data structure for this purpose was published by stuart haber and w. scott stornetta in . a company using their technique has been providing a centralized service of securely time-stamping documents for nearly a quarter of a century. it is a form of merkle or hash tree, published by ralph merkle in . for blockchains it is a linear chain to which fixed-size blocks are added at regular intervals. each block contains the hash of its predecessor; a chain of blocks. the blockchain is mutable, it is just rather hard to mutate it without being detected, because of the merkle tree’s hashes, and easy to recover, because there are lots of copies keeping stuff safe. but this is a double-edged sword. immutability makes systems incompatible with the gdpr, and immutable systems to which anyone can post information will be suppressed by governments. [slide ] btc transaction fees cryptokitties’ popularity exploded in early december and had the ethereum network gasping for air. ... ethereum has historically made bold claims that it is able to handle unlimited decentralized applications ... the crypto-kittie app has shown itself to have the power to place all network processing into congestion. ... at its peak [cryptokitties] likely only had about , daily users. neopets, a game to which cryptokitties is often compared, once had as many as million users. how crypto-kitties disrupted the ethereum network, open trading network a user of your system wanting to perform a transaction, store a file, record a review, whatever, needs to persuade miners to include their transaction in a block. miners are coin-operated; you need to pay them to do so. how much do you need to pay them? that question reveals another economic problem, fixed supply and variable demand, which equals variable "price". each block is in effect a blind auction among the pending transactions. so lets talk about cryptokitties, a game that bought the ethereum blockchain to its knees despite the bold claims that it could handle unlimited decentralized applications. how many users did it take to cripple the network? it was far fewer than non-blockchain apps can handle with ease; cryptokitties peaked at about k users. neopets, a similar centralized game, peaked at about , times as many. cryptokitties average "price" per transaction spiked % between november and december as the game got popular, a major reason why it stopped being popular. the same phenomenon happened during bitcoin's price spike around the same time. cryptocurrency transactions are affordable only if no-one wants to transact; when everyone does they immediately become un-affordable. nakamoto's bitcoin blockchain was designed only to support recording transactions. it can be abused for other purposes, such as storing illegal content. but it is likely that you need additional functionality, which is where ethereum's "smart contracts" come in. these are fully functional programs, written in a javascript-like language, embedded in ethereum's blockchain. they are mainly used to implement ponzi schemes, but they can also be used to implement initial coin offerings, games such as cryptokitties, and gambling parlors. further, in on-chain vote buying and the rise of dark daos philip daian and co-authors show that "smart contracts" also provide for untraceable on-chain collusion in which the parties are mutually pseudonymous. [slide ] ico returns the first big smart contract, the dao or decentralized autonomous organization, sought to create a democratic mutual fund where investors could invest their ethereum and then vote on possible investments. approximately % of all ethereum ended up in the dao before someone discovered a reentrancy bug that enabled the attacker to effectively steal all the ethereum. the only reason this bug and theft did not result in global losses is that ethereum developers released a new version of the system that effectively undid the theft by altering the supposedly immutable blockchain. risks of cryptocurrencies, nicholas weaver, u.c. berkeley "smart contracts" are programs, and programs have bugs. some of the bugs are exploitable vulnerabilities. research has shown that the rate at which vulnerabilities in programs are discovered increases with the age of the program. the problems caused by making vulnerable software immutable were revealed by the first major "smart contract". the decentralized autonomous organization (the dao) was released on th april , but on th may dino mark, vlad zamfir, and emin gün sirer posted a call for a temporary moratorium on the dao, pointing out some of its vulnerabilities; it was ignored. three weeks later, when the dao contained about % of all the ether in circulation, a combination of these vulnerabilities was used to steal its contents. the loot was restored by a "hard fork", the blockchain's version of mutability. since then it has become the norm for "smart contract" authors to make them "upgradeable", so that bugs can be fixed. "upgradeable" is another way of saying "immutable in name only". [slide ] security researchers from srlabs revealed that a large chunk of the ethereum client software that runs on ethereum nodes has not yet received a patch for a critical security flaw the company discovered earlier this year. "according to our collected data, only two thirds of nodes have been patched so far," said karsten nohl, ... "the parity ethereum has an automated update process - but it suffers from high complexity and some updates are left out," nohl said. all of these issues put all ethereum users at risk, and not just the nodes running unpatched versions. the number of unpatched notes may not be enough to carry out a direct % attack, but these vulnerable nodes can be crashed to reduce the cost of a % attack on ethereum, currently estimated at around $ , per hour. ... "the patch gap signals a deep-rooted mistrust in central authority, including such any authority that can automatically update software on your computer." a large chunk of ethereum clients remain unpatched catalin cimpanu it isn't just the "smart contracts" that need to be upgradeable, it is the core software for the blockchain. bugs and vulnerabilities are inevitable. if you trust a central authority to update your software automatically, or if you don't but you think others do, what is the point of a permissionless blockchain? [slide ] permissionless systems trust: the core developers of the blockchain software not to write bugs. the developers of your wallet software not to write bugs. the developers of the exchanges not to write bugs. the operators of the exchanges not to manipulate the markets or to commit fraud. the developers of your upgradeable "smart contracts" not to write bugs. the owners of the smart contracts to keep their secret key secret. the owners of the upgradeable smart contracts to avoid losing their secret key. the owners and operators of the dominant mining pools not to collude. the operators of miners to apply patches in a timely manner. the speculators to provide the funds needed to keep the “price” going up. users' ability to keep their secret key secret. users’ ability to avoid losing their secret key. other users not to transact when you want to. so, this is the list of people your permissionless system has to trust if it is going to work as advertised over the long term. you started out to build a trustless, decentralized system but you have ended up with: a trustless system that trusts a lot of people you have every reason not to trust. a decentralized system that is centralized around a few large mining pools that you have no way of knowing aren’t conspiring together. an immutable system that either has bugs you cannot fix, or is not immutable a system whose security depends on it being expensive to run, and which is thus dependent upon a continuing inflow of funds from speculators. a system whose coins are convertible into large amounts of "fiat currency" via irreversible pseudonymous transactions, which is thus an irresistible target for crime. if the “price” keeps going up, the temptation for your trust to be violated is considerable. if the "price" starts going down, the temptation to cheat to recover losses is even greater. maybe it is time for a re-think. suppose you give up on the idea that anyone can take part and accept that you have to trust a central authority to decide who can and who can’t vote. you will have a permissioned system. the first thing that happens is that it is no longer possible to mount a sybil attack, so there is no reason running a node need be expensive. you can use bft to establish consensus, as ibm’s hyperledger, the canonical permissioned blockchain system plans to. you need many fewer nodes in the network, and running a node just got way cheaper. overall, the aggregated cost of the system got orders of magnitude cheaper. now there is a central authority it can collect “fiat currency” for network services and use it to pay the nodes. no need for cryptocurrency, exchanges, pools, speculators, or wallets, so much less temptation for bad behavior. [slide ] permissioned systems trust: the central authority. the software developers. the owners and operators of the nodes. the secrecy of a few private keys. this is now the list of entities you trust. trusting a central authority to determine the voter roll has eliminated the need to trust a whole lot of other entities. the permissioned system is more trustless and, since there is no need for pools, the network is more decentralized despite having fewer nodes. [slide ] faults replicas a byzantine quorum system of size could achieve better decentralization than proof-of-work mining at a much lower resource cost. decentralization in bitcoin and ethereum networks, adem efe gencer, soumya basu, ittay eyal, robbert van renesse and emin gün sirer how many nodes does your permissioned blockchain need? the rule for bft is that f + nodes can survive f simultaneous failures. that's an awful lot fewer than you need for a permissionless proof-of-work blockchain. what you get from bft is a system that, unless it encounters more than f simultaneous failures, remains available and operating normally. the problem with bft is that if it encounters more than f simultaneous failures, the state of the system is irrecoverable. if you want a system that can be relied upon for the long term you need a way to recover from disaster. successful permissionless blockchains have lots of copies keeping stuff safe, so recovering from a disaster that doesn't affect all of them is manageable. [slide ] source so in addition to implementing bft you need to back up the state of the system each block time, ideally to write-once media so that the attacker can't change it. but if you're going to have an immutable backup of the system's state, and you don't need continuous uptime, you can rely on the backup to recover from failures. in that case you can get away with, say, replicas of the blockchain in conventional databases, saving even more money. i've shown that, whatever consensus mechanism they use, permissionless blockchains are not sustainable for very fundamental economic reasons. these include the need for speculative inflows and mining pools, security linear in cost, economies of scale, and fixed supply vs. variable demand. proof-of-work blockchains are also environmentally unsustainable. the top cryptocurrencies are estimated to use as much energy as the netherlands. this isn't to take away from nakamoto's ingenuity; proof-of-work is the only consensus system shown to work well for permissionless blockchains. the consensus mechanism works, but energy consumption and emergent behaviors at higher levels of the system make it unsustainable. [slide ] mentions in s&p quarterlies still new to nyc, but i met this really cool girl. energy sector analyst or some such. four dates in, she uncovers my love for bitcoin. completely ghosted. zack voell s&p companies are slowly figuring out that there is no there there in blockchains and cryptocurrencies, and they're not the only ones: so if both permissionless and permissioned blockchains are fatally flawed, and experts in both cryptography and economics have been saying so for many years, how come they are generally perceived as huge successes? the story starts in the early s with david chaum. his work on privacy was an early inspiration for the cypherpunks. many of the cypherpunks were libertarians, so the idea of money not controlled by governments was attractive. but chaum's pioneering digicash was centralized, a fatal flaw in their eyes. it would be two decades before the search for a practical decentralized cryptocurrency culminated with nakamoto's bitcoin. [slide ] bitcoin failed at every one of nakamoto's aspirations here. the price is ridiculously volatile and has had multiple bubbles; the unregulated exchanges (with no central bank backing) front-run their customers, paint the tape to manipulate the price, and are hacked or just steal their user's funds; and transaction fees and the unreliability of transactions make micropayments completely unfeasible. david gerard a parallel but less ideological thread was the idea that the business model for the emerging internet was micropayments. this was among the features nakamoto touted for bitcoin in early , despite the idea having been debunked by clay shirky in and andrew odlyzko in . in fact, none of nakamoto's original goals worked out in practice. but nakamoto was not just extremely clever in the way he assembled the various component technologies into a cryptocurrency, he also had exceptionally good timing. his paper was posted on st october , and met three related needs: just days earlier, on th september lehman brothers had gone bankrupt, precipitating the global financial crisis (the gfc). the gfc greatly increased the demand for flight capital in china. mistaking pseudonymity for anonymity, vendors and customers on the dark web found bitcoin a reassuring means of exchange. a major reason bitcoin was attractive to the libertarian cypherpunks was that many were devotees of the austrian economics cult. because there would only ever be million bitcoin, they believed that, like gold, the price would inevitably increase. consider a currency whose price is doomed to increase. it is a mechanism for transferring wealth from later adopters, called suckers, to early adopters, called geniuses. and the cypherpunks were nothing if not early adopters of technology. sure enough, a few of the geniuses turned into "whales", hodl-ing the vast majority of the bitcoin. the gini coefficient of cryptocurrencies is an interesting research question; it is huge but probably less than nouriel roubini's claim of . . the whales needed to turn large amounts of cryptocurrency in their wallets into large numbers in a bank account denominated in "fiat currency". to do this they needed to use an exchange to sell cryptocurrency to a sucker for dollars, and then transfer the dollars from the exchange's bank account into their bank account. [slide ] we’ve had banking hiccups in the past, we’ve just always been able to route around it or deal with it, open up new accounts, or what have you … shift to a new corporate entity, lots of cat and mouse tricks. phil potter of the bitfinex exchange. fowler opened numerous u.s.-based business bank accounts at several different banks, and in opening and using these accounts fowler and yosef falsely represented to those banks that the accounts would be primarily used for real estate investment transactions even though fowler and yosef knew that the accounts would be used, and were in fact used, by fowler, yosef and others to transmit funds on behalf of an unlicensed money transmitting business related to the operation of cryptocurrency exchanges. us vs. reginald fowler and ravid yosef for the exchange to have a bank account, it had to either conform to or evade the "know your customer/anti-money laundering" laws. the whole point of cryptocurrencies is to avoid dealing with banks and laws such as kyc/aml, so most exchanges chose to evade kyc/aml by a cat-and-mouse game of fraudulent accounts. once the banks caught on to the cat-and-mouse game, most exchanges could not trade cryptocurrency for fiat currency. to continue, they needed a "stablecoin", a cryptocurrency fixed against the us dollar as a substitute for actual dollars. the guys behind bitfinex, one of the sketchier exchanges, invented tether. they claimed their usdt was backed one-for-one by usd, promising an audit would confirm this. but before an audit appeared they fired their auditors. earlier this year, after the new york attorney general sued them, they claimed it was % backed by usd (except when they accidentally create billion usdt), and revealed an m usd hole in bitfinex' accounts. [slide ] "approximately % of this volume is fake and/or non-economic in nature, and that the real market for bitcoin is significantly smaller, more orderly, and more regulated than commonly understood." bitwise asset management's detailed comments to the sec about btc/usdt trading on unregulated exchanges. according to blockchain.info, about $ m worth of bitcoin was traded on friday on the main dollar-based exchanges. which sounds decent until you notice that about $ bn worth of tether was traded on friday, according to coinmarketcap. jemima kelly, ft alphaville there were many usdt exchanges, and competition was intense. customers wanted the exchange with the highest volume for their trades, so these exchanges created huge volumes of wash trades to inflate their volume. around % of all cryptocurrency trades are fake. [slide ] "an upset mt. gox creditor analyses the data from the bankruptcy trustee’s sale of bitcoins. he thinks he’s demonstrated incompetent dumping by the trustee — but actually shows that a “market cap” made of million btc can be crashed by selling , btc, over months, at market prices, which suggests there is no market." david gerard. because there was so little real trading between cryptocurrencies and usd, trades of the size the whales needed would crash the price. it was thus necessary to pump the price before even part of their hodlings could be dumped on the suckers. [slide ] p&ds have dramatic short-term impacts on the prices and volumes of most of the pumped tokens. in the first seconds after the start of a p&d, the price increases by % on average, trading volume increases times, and the average -second absolute return reaches %. a quick reversal begins seconds after the start of the p&d. ... for an average p&d, investors make one bitcoin (about $ , ) in profit, approximately one-third of a token’s daily trading volume. the trading volume during the minutes before the pump is % of the total volume during the minutes after the pump. this implies that an average trade in the first minutes after a pump has a % chance of trading against these insiders and on average they lose more than % ( %* %). cryptocurrency pump-and-dump schemes tao li, donghwa shin and baolian wang off-chain collusion among cryptocurrency traders allows for extremely profitable pump-and-dump schemes, especially given the thin trading in "alt-coins". but the major pumps, such as the one currently under way, come from the creation of huge volumes of usdt, in this case about one billion usdt per month. [slide ] issuance of usdt [in april] there were about $ billion worth of tethers on the market. since then, tether has gone on a frenzied issuance spree. in the month of may, the stablecoin company pumped out $ billion worth of tethers into the market. and this month, it is on track for another $ billion. currently, there are roughly $ . billion worth of tethers sloshing around in the bitcoin markets. whether this money is backed by real dollars is anyone's guess. amy castor who would believe that pushing a billion "dollars" a month that can only be used to buy cryptocurrency into the market might cause people to buy cryptocurrency and drive the price up? if we believe bitfinex that the % of usdt that isn't usd is in cryptocurrencies, that might provide a motive for a massive pump to recover, say, m usd in losses. i want to end by talking about a technology with important implications for software supply chain security that looks like, but isn't a blockchain. [slide ] a green padlock (with or without an organization name) indicates that: you are definitely connected to the website whose address is shown in the address bar; the connection has not been intercepted. the connection between firefox and the website is encrypted to prevent eavesdropping. how do i tell if my connection to a website is secure? mozilla how do i know that i'm talking to the right web site? because there's a closed padlock icon in the url bar, right? the padlock icon appears when the browser has verified that the connection to the url in the url bar supplied a certificate for the site in question carrying a signature chain ending in one of the root certificates the browser trusts. browsers come with a default list of root certificates from certificate authorities (cas). my current firefox browser trusts root certificates from unique organizations, among them foreign governments but not the us government. some of the organizations whose root certificates my browser trusts are known to have abused this trust, allowing miscreants to impersonate sites, spy on users and sign malware so it appears to be coming from, for example, microsoft or apple. [slide ] a crucial technical property of the https authentication model is that any ca can sign certificates for any domain name. in other words, literally anyone can request a certificate for a google domain at any ca anywhere in the world, even when google itself has contracted one particular ca to sign its certificate. security collapse in the https market, axel arnbak et al for example, google discovered that "symantec cas have improperly issued more than , certificates". but browsers still trust symantec cas; their market share is so large the web would collapse if they didn't. as things stand, clients have no way of knowing whether the root of trust for a certificate, say for the library of congress, is the one the library intended, or a spoof from some ca in turkey or china. in google started work on an approach based on ronald reagan's "trust but verify" paradigm, called certificate transparency (ct). the basic idea is to accompany the certificate with a hash of the certificate signed by a trusted third party, attesting that the certificate holder told the third party that the certificate with that hash was current. thus in order to spoof a service, an attacker would have to both obtain a fraudulent certificate from a ca, and somehow persuade the third party to sign a statement that the service had told them the fraudulent certificate was current. clearly this is: more secure than the current situation, which requires only compromising a ca, and: more effective than client-only approaches, which can detect that a certificate has changed but not whether the change was authorized. clients now need two lists of trusted third parties, the cas and the sources of ct attestations. the need for these trusted third parties is where the blockchain enthusiasts would jump in and claim (falsely) that using a blockchain would eliminate the need for trust. in the real world it isn't feasible to solve the problem of untrustworthy cas by eliminating the need for trust. ct's approach instead is to provide a mechanism by which breaches of trust, both by the cas and by the attestors, can be rapidly and unambiguously detected. this can be done because: certificate owners obtain attestations from multiple sources, who are motivated not to conspire. clients can verify these multiple attestations. the attestors publish merkle trees of their attestations, which can be verified by their competitors. [slide ] each log operates independently. each log gets its content directly from the cas, not via replication from other logs. each log contains a subset of the total information content of the system. there is no consensus mechanism operating between the logs, so it cannot be abused by, for example, a % attack. monitoring and auditing is asynchronous to web content delivery, so denial of service against the monitors and auditors cannot prevent clients obtaining service. certificate transparency david s. h. rosenthal how do i know i'm running the right software, and no-one has implanted a backdoor? right now, there is no equivalent of ct for the signatures that purport to verify software downloads, and this is one reason for the rash of software supply chain attacks. the open source community has a long-standing effort to use ct-like techniques not merely to enhance the reliability of the signatures on downloads, but more importantly to verify that a binary download was compiled from the exact source code it claims to represent. the reason this project is taking a long time is that it is woefully under-funded, and it is a huge amount of work. it depends on ensuring that the build process for each package is reproducible, so that given the source code and the build specification, anyone can run the build and generate bit-for-bit identical results. to give you some idea of how hard this is, the uk government has been working with huawei since to make their router software builds reproducible so they know the binaries running in the uk's routers match the source code huawei disclosed. huawei expects to finish this program in . with a few million dollars in funding, in a couple of years the open source community could finish making the major linux distributions reproducible and implement ct-like assurance that the software you were running matched the source code in the repositories, with no hidden backdoors. i would think this would be something the dod would be interested in. thank you for your attention, i'm ready for questions. posted by david. at : am labels: bitcoin comments: david. said... the topics of the questions i remember were: ) use of cryptocurrency for money laundering and terrorism funding. ) enforcement actions by governments. ) use of blockchain technology by major corporations. ) pr for libertarian politics by cryptocurrency hodlers. ) relative security of decentralized vs. centralized blockchains. i will add shortly comments addressing them, with links to sources. july , at : am david. said... ) use of cryptocurrency for money laundering and terrorism funding. in general it is a bad idea to commit crimes using an immutable public blockchain. pseudonymous blockchains such as bitcoin's require extremely careful op-sec if the pseudonym is not to be linked to web trackers and cookies (see, for example, when the cookie meets the blockchain: privacy risks of web payments via cryptocurrencies by steven goldfeder et al). there are cryptocurrencies with stronger privacy features, such as zcash and monero. these are more popular among malefactors than bitcoin. but turning cryptocurrencies into fiat currency with which to buy your lamborghini while remaining anonymous faces difficulties. users of exchanges that observe kyc/aml, such as coinbase, will need to explain the source of funds to the tax authorities. the irs recently sent letters to coinbase users reminding them of their obligation to report the gains and losses on every single transaction. north korea is reputed to be very active in stealing cryptocurrency via exchange hacks and other techniques. ) enforcement actions by governments. see my post regulating cryptocurrencies and the comments to it. ) use of blockchain technology by major corporations. see, for example, blockchain for international development: using a learning agenda to address knowledge gaps by john burg, christine murphy, & jean paul pétraud. and this, from david gerard: "bundesbank and deutsche boerse try settlements on the blockchain. you’ll be amazed to hear that it was slower and more expensive. “despite numerous tests of blockchain-based prototypes, a real breakthrough in application is missing so far.” but at least it “in principle fulfilled all basic regulatory features for financial transactions.” july , at : pm david. said... ) pr for libertarian politics by cryptocurrency hodlers. john mcafee is running for us president. see also laurie penny's must-read four days trapped at sea with crypto’s nouveau riche. ) relative security of decentralized vs. centralized blockchains. as i described above, at scale anything claiming to be a "decentralized blockchain" isn't going to be decentralized. economic forces will have centralized it around a small number of mining pools. see decentralization in bitcoin and ethereum networks by adem efe gencer, soumya basu, ittay eyal, robbert van renesse and emin gün sirer. its security will depend upon those pools not conspiring together, among many other things (slide ). centralized, permissioned blockchains have fewer vulnerabilities, but their central authority is a single point of failure. iirc the questioner used the phrase " % secure". no networked computer system is ever % secure. ) i seem to remember also a question on pump-and-dump schemes. the current pump is via tether. social capital has a series explaining tether and the "stablecoin" scam: * pumps, spoofs and boiler rooms * tether, part one: the stablecoin dream * tether, part two: pokedex * tether, part three: crypto island july , at : pm david. said... north korea took $ billion in cyberattacks to fund weapons program: u.n. report by michelle nichols reports that: "north korea has generated an estimated $ billion for its weapons of mass destruction programs using “widespread and increasingly sophisticated” cyberattacks to steal from banks and cryptocurrency exchanges, according to a confidential u.n. report seen by reuters on monday." august , at : pm david. said... timothy b. lee debunks the idea of bitcoin for purchases in i tried to pay with bitcoin at a mexico city bar—it didn’t go well: "so we gave up and paid with a conventional credit card. after leaving the bar, i sent off an email to the support address listed on my receipt. the next morning, i got a response: "transactions under [ , pesos] are taking a day to two, in the course of today they will reach the wallet." i finally got my bitcoins around pm." the bar is called bitcoin embassy: "does bitcoin embassy pay its employees in bitcoin? "i always tell them i can pay you in bitcoin if you want to, but they don't want to," ortiz says." august , at : am david. said... clare duffy reports that the fed is getting into the real-time payments business: "the fed announced monday that it will develop a real-time payment service called "fednow" to help move money around the economy more quickly. it's the kind of government service that companies and consumers have been requesting for years — one that already exists in other countries. the service could also compete with solutions already developed in the private sector by big banks and tech companies. the fed itself is not setting up a consumer bank, but it has always played a behind-the-scenes role facilitating the movement of money between banks and helping to verify transactions. this new system would help cut down on the amount of time between when money is deposited into an account and when it is available for use. fednow would operate all hours and days of the week, with an aim to launch in or . " "real-time payments" are something that enthusiasts see as a competitive edge for cryptocurrencies against fiat currencies. this is strange, for two reasons: ) in most countries except the us, instantaneous inter-bank transfers have been routine for years. but the enthusiasts are so ignorant of the way the world outside the us works that they don't know this. similar ignorance was evident in the way facebook thought that libra would "bank the unbanked" in the third world. ) cryptocurrency transfers are not in practice real-time. bitcoin users are advised to wait block times (one hour) before treating a transaction as confirmed. august , at : pm david. said... jemima kelly's when bitcoin bros talk cryptography provides an excellent example of the hype surrounding cryptocurrencies. anthony pompliano, a "crypto fund manager" who "has over half his net worth in bitcoin" was talking (his book) to cnbc and: "when one of the cnbc journalists put it to pomp that just because bitcoin is scarce that doesn’t necessarily make it valuable, as “there are a lot of things that are scarce that nobody cares about”, pomp said: of course. look, if you don’t believe in bitcoin, you’re essentially saying you don’t believe in cryptography. have a watch for yourself here (and count the seconds it takes for the others to recover from his comment, around the . mark):" the video is here. august , at : pm david. said... more on the fed's real-time payment proposal in the fed is going to revamp how americans pay for things. big banks aren’t happy from mit technology review. august , at : am david. said... trail of bits has released: "findings from the full final reports for twenty-three paid security audits of smart contract code we performed, five of which have been kept private. the public audit reports are available online, and make informative reading. we categorized all smart-contract related findings from these reports" the bottom line is that smart contracts are programs, and programs have bugs. using current automated tools can find some but not all of them. august , at : pm david. said... brenna smith's the evolution of bitcoin in terrorist financing makes interesting and somewhat scary reading: "terrorists’ early attempts at using cryptocurrencies were filled with false starts and mistakes. however, terrorists are nothing if not tenacious, and through these mistakes, they’ve grown to have a highly sophisticated understanding of blockchain technology. this investigation outlines the evolution of terrorists’ public bitcoin funding campaigns starting from the beginning and ending with the innovative solutions various groups have cooked up to make the technology work in their favor." august , at : pm david. said... larry cermak has a twitter thread that starts: "it’s now obvious that icos were a massive bubble that's unlikely to ever see a recovery. the median ico return in terms of usd is - % and constantly dropping. let's look at some data!" hat tip to david gerard. august , at : pm david. said... the abstract for the european central bank's in search for stability in crypto-assets: are stablecoins the solution? reads: "stablecoins claim to stabilise the value of major currencies in the volatile crypto-asset market. this paper describes the often complex functioning of different types of stablecoins and proposes a taxonomy of stablecoin initiatives. to this end it relies on a novel framework for their classification, based on the key dimensions that matter for crypto-assets, namely: (i) accountability of issuer, (ii) decentralisation of responsibilities, and (iii) what underpins the value of the asset. the analysis of different types of stablecoins shows a trade-off between the novelty of the stabilisation mechanism used in an initiative (from mirroring the traditional electronic money approach to the alleged introduction of an “algorithmic central bank”) and its capacity to maintain a stable market value. while relatively less innovative stablecoins could provide a solution to users seeking a stable store of value, especially if legitimised by the adherence to standards that are typical of payment services, the jury is still out on the potential future role of more innovative stablecoins outside their core user base." august , at : am david. said... david gerard writes: "tethers as erc- tokens on the ethereum blockchain are so popular that they’re flooding ethereum with transactions, and clogging the blockchain — “yesterday i had to wait and half hours for a standard transfer to go through.” ethereum is the world computer, as long as you don’t try to use it for any sort of real application. another million tethers were also printed today." september , at : pm david. said... david gerard has been researching libra, and has two posts up on the topic. today's is switzerland’s guidance on stablecoins — what it means for facebook’s libra: "libra will need to register as a bank and as a payment provider (a money transmitter). it probably won’t need to register as a collective investment scheme for retail investors. finma notes explicitly: “the highest international anti-money laundering standards would need to be ensured throughout the entire ecosystem of the project” — and that libra in particular requires an “internationally coordinated approach.” so the effective consequence is that libra will be a coin for well-documented end users in highly regulated rich countries, and not so available in poorer ones." yesterday's was your questions about facebook libra — as best as we can answer them as yet (my emphasis): "as i write this, calibra.com, the big splash page for calibra, doesn’t work in firefox — only in chrome. this is how companies behave toward products they don’t really take seriously. facebook also forgot to buy the obvious typo, colibra.com — which is a domain squatter holding page." facebook is under mounting anti-trust pressure, both in the us and elsewhere, and it is starting to look like cost-of-doing-business fines are no longer the worst that can happen. my take on libra is that facebook is floating it as a bargaining chip - in the inevitable negotiations on enforcement measures facebook can sacrifice libra to protect more valuable assets. september , at : pm david. said... claire jones and izabella kaminska's libra is imperialism by stealth points out that, in practice, currency-backed stablecoins like libra and ( % of) tether are tied to the us dollar. argentina and zimbabwe are just two examples showing how bad an idea dollarizing your economy is: "a common criticism against dollarisation (and currency blocs) is that they are a form of neocolonialism, handing global powers -- whether they are states or tech behemoths -- another means of exercising control over more vulnerable players. stablecoins backed by dollar assets are part of the same problem, which is why we believe their adoption in places like argentina would constitute imperialism by stealth." september , at : am david. said... dan goodin writes about a statement from the us treasury announcing sanctions against north korean hacking groups: "north korean hacking operations have also targeted virtual asset providers and cryptocurrency exchanges, possibly in an attempt to obfuscate revenue streams used to support the countries weapons programs. the statement also cited industry reports saying that the three north korean groups likely stole about $ million in cryptocurrency from five exchanges in asia between january and september . news agencies including reuters have cited a united nations report from last month that estimated north korean hacking has generated $ billion for the country’s weapons of mass destruction programs." september , at : pm david. said... tether slammed as “part-fraud, part-pump-and-dump, and part-money laundering” by jemima kelly suggests some forthcoming increase in transparency about tether: "a class-action lawsuit was filed against tether, bitfinex (a sister crypto exchange), and a handful of others. the suit was made public on monday, having been filed on saturday in court of the southern district of new york by vel freedman and kyle roche. notably, they are the same lawyers who recently (and successfully) sued craig wright on behalf of ira kleiman." october , at : am david. said... the abstract of cryptodamages: monetary value estimates of the air pollution and human health impacts of cryptocurrency mining by goodkind et al reads: "cryptocurrency mining uses significant amounts of energy as part of the proof-of-work time-stamping scheme to add new blocks to the chain. expanding upon previously calculated energy use patterns for mining four prominent cryptocurrencies (bitcoin, ethereum, litecoin, and monero), we estimate the per coin economic damages of air pollution emissions and associated human mortality and climate impacts of mining these cryptocurrencies in the us and china. results indicate that in , each $ of bitcoin value created was responsible for $ . in health and climate damages in the us and $ . in china. the similar value in china relative to the us occurs despite the extremely large disparity between the value of a statistical life estimate for the us relative to that of china. further, with each cryptocurrency, the rising electricity requirements to produce a single coin can lead to an almost inevitable cliff of negative net social benefits, absent perpetual price increases. for example, in december , our results illustrate a case (for bitcoin) where the health and climate change “cryptodamages” roughly match each $ of coin value created. we close with discussion of policy implications." october , at : am david. said... ian allison's foreign exchange giant cls admits: no, we don’t need a blockchain for that starts: "blockchain technology is nice to have, but it’s hardly a must for rewiring the global financial markets. so says alan marquard, chief strategy and development officer at cls group, the global utility for settling foreign exchange trades, owned by the largest banks active in that market. nearly a year ago, it went live with clsnet, touted as “the first global fx market enterprise application running on blockchain in production,” with megabanks goldman sachs, morgan stanley, and bank of china (hong kong) on board. clsnet was built on hyperledger fabric, the enterprise blockchain platform developed by ibm. but a blockchain was not the obvious solution for netting down high volumes of fx trades in currencies, marquard said recently." october , at : pm david. said... preston byrne's fear and loathing on the blockchain: leibowitz et al. v. ifinex et al. is a must-read summary of the initial pleadings in the civil case just filed against tether and bitfinex. byrne explains how the risks for the defendants are different from earlier legal actions: "being a civil case, protections bitfinex might be able to rely on in other contexts, such as the fourth amendment in any criminal action, arguing that the martin act doesn't confer jurisdiction over bitfinex's activities, or arguing that an administrative subpoena served on it by the new york attorney general is overbroad, won't apply here. discovery has the potential to be broader and deeper than bitfinex has shown, to date, that it is comfortable with. the consequence of defaulting could be financially catastrophic. the burden of proof is lower, too, than it would be with a criminal case (balance of probabilities rather than beyond a reasonable doubt)." october , at : pm david. said... david gerard provides some good advice: "if you’re going to do crimes, don’t do them on a permanent immutable public ledger of all transactions — and especially, don’t do crimes reprehensible enough that everyone gets together to come after you." from the chainalysis blog: "today, the department of justice announced the shutdown of the largest ever child pornography site by amount of material stored, along with the arrest of its owner and operator. more than site users across countries have also been arrested so far. most importantly, as of today, at least minors were identified and rescued from their abusers as a result of this investigation. u.s. attorney jessie k. liu put it best: “children around the world are safer because of the actions taken by u.s. and foreign law enforcement to prosecute this case and recover funds for victims.” commenting on the investigation itself, irs-criminal investigations chief don fort mentioned the importance of the sophisticated tracing of bitcoin transactions in order to identify the administrator of the website. we’re proud to say that chainalysis products provided assistance in this area, helping investigators analyze the website’s cryptocurrency transactions that ultimately led to the arrests. ... when law enforcement shut down the site, they siezed over terabytes of child pornography, making it one of the largest siezures of its kind. the site had . million bitcoin addresses registered. between and , the site received nearly $ , worth of bitcoin across thousands of individual transactions." october , at : pm david. said... tim swanson has updated his post from august entitled how much electricity is consumed by bitcoin, bitcoin cash, ethereum, litecoin, and monero? which concluded as much as the netherlands. in have pow blockchains become less resource intensive? he concludes that: "in aggregate, based on the numbers above, these five pow coins likely consume between . billion kwh and . billion kwh annually. that’s somewhere around switzerland on the low end to finland or pakistan near the upper end. it is likely much closer to the upper bound because the calculations above all assumed little energy loss ‘at the wall’ when in fact there is often % or more energy loss depending on the setup. this is a little lower than last year, where we used a similar method and found that these pow networks may consume as much resources as the netherlands. why the decline? all of it is due to the large decline in coin prices over the preceding time period. again, miners will consume resources up to the value of a block reward wherein the marginal cost to mine equals the marginal value of the coin (mc=mv)." october , at : pm david. said... what's blockchain actually good for, anyway? for now, not much by gregory barber has a wealth of examples of blockchain hype fizzling but, being a journalist, he can't bring himself to reach an actual conclusion: "“decentralized” projects represent a tiny portion of corporate blockchain efforts, perhaps percent, says apolline blandin, a researcher at the cambridge centre for alternative finance. the rest take shortcuts. so-called permissioned blockchains borrow ideas and terms from bitcoin, but cut corners in the name of speed and simplicity. they retain central entities that control the data, doing away with the central innovation of blockchains. blandin has a name for those projects: “blockchain memes.” hype and lavish funding fueled many such projects. but often, the same applications could be built with less-splashy technology. as the buzzwords wear off, some have begun asking, what’s the point?" november , at : pm david. said... patrick mckenzie's tether: the story so far is now the one-stop go-to explainer for tether and bitfinex: "a friend of mine, who works in finance, asked me to explain what tether was. short version: tether is the internal accounting system for the largest fraud since madoff. read on for the long version." you need to follow his advice. november , at : pm david. said... a lone bitcoin whale likely fueled surge, study finds by matthew leising and matt robinson reports on an update to 's is bitcoin really un-tethered?: "one entity on the cryptocurrency exchange bitfinex appears capable of sending the price of bitcoin higher when it falls below certain thresholds, according to university of texas professor john griffin and ohio state university’s amin shams. griffin and shams, who have updated a paper they first published in , say the transactions rely on tether, a widely used digital token that is meant to hold its value at $ ." november , at : pm david. said... today's news emphasizes that using "trustless" systems requires trusting a lot more than just the core software. first, dan goodin's official monero website is hacked to deliver currency-stealing malware: "the supply-chain attack came to light on monday when a site user reported that the cryptographic hash for a command-line interface wallet downloaded from the site didn't match the hash listed on the page. over the next several hours, users discovered that the miss-matching hash wasn't the result of an error. instead, it was an attack designed to infect getmonero users with malware. site officials later confirmed that finding." second, david gerard reports that canada’s einstein exchange shuts down — and all the money’s gone: "yet another canadian crypto exchange goes down — this time, the einstein exchange in vancouver, british columbia, canada, which was shut down by the british columbia securities commission on november. yesterday, november, the news came out that there’s nothing left at all — the money and cryptos are gone." november , at : pm david. said... jonathan syu tweets: "the whole decentralization experiment is really just an attempt to figure out what's the maximum amount of control i can maintain over a system without having any legal accountability over what happens to it." november , at : pm david. said... electronic voting machines mean you can't trust the result of the election, but there are even worse ways to elect politicians. one is internet voting. but among the very worst is to use (drum-roll) blockchain technology! to understand why it is so bad, read what we don’t know about the voatz “blockchain” internet voting system by david jefferson, duncan buell, kevin skogland, joe kiniry and joshua greenbaum. the list of unknowns covers ten pages, and every one should disqualify the system from use. november , at : pm david. said... nathaniel rich's ponzi schemes, private yachts, and a missing $ million in crypto: the strange tale of quadriga is a great illustration of the kind of people you have to trust to use trustless cryptocurrencies. the subhead reads: "when canadian blockchain whiz gerald cotten died unexpectedly last year, hundreds of millions of dollars in investor funds vanished into the crypto ether. but when the banks, the law, and the forces of reddit tried to track down the cash, it turned out the young mogul may not have been who he purported to be." it is a fascinating story - go read it. november , at : am david. said... among the people you have to trust to use a trustless system are the core developers. people like virgil griffith, a core ethereum developer. read about him in david gerard's virgil griffith arrested over north korea visit — engineer arrogance, but on the blockchain and see whether you think he's trustworthy. december , at : am david. said... i'm shocked, shocked to find illegal activities going on here!. celia wan reports that: "there are over $ million worth of illicit activities conducted via xrp, and a large portion of these activities are scams and ponzi schemes, elliptic, a blockchain analytics startup, found in its research. the uk-based startup announced on wednesday that it can now track the transactions of xrp, marking the th digital assets the firm supports. according to elliptic co-founder tom robinson, the firm is the first to have transaction monitoring capacity for xrp and it has already identified several hundred xrp accounts related to illegal activities." december , at : pm david. said... jemima kelley's when is a blockchain startup not a blockchain startup? recounts yet another company discovering that permissioned blockchains are just an inefficient way to implement things you can do with a regular database: "it’s awkward when you set up a business around a technology that you reckon is going to disrupt global finance so you name your business after said technology, send your ceo on speaking tours to evangelise about said technology, but then decide that said technology isn’t going to do anything useful for you, isn’t it? digital asset was one of the pioneers of blockchain-in-finance, with the idea that it could make clearing and trade settlement sexy again (well ok maybe not but at least make it faster and more efficient). its erstwhile ceo blythe masters, a glamorous former jp executive who was credited with/blamed for pioneering credit-default swaps, trotted around the globe telling people that blockchain was “email for money”. ... what is unique about this “blockchain start-up” is that although it is designed to work with blockchain platforms, it can actually work with any old database." december , at : am david. said... more dents in the credibility of the btc "price" from yashu gola's how a whale crashed bitcoin to sub-$ , overnight: "bitcoin lost billions of dollars worth of valuation within a -minutes timeframe as a chinese cryptocurrency scammer allegedly liquidated its steal via over-the-counter markets. the initial sell-off by plustoken caused a domino effect, causing mass liquidations. plustoken, a fraud scheme that duped investors of more than $ bn, dumped huge bitcoin stockpiles from its anonymous accounts, according to chainalysis." december , at : am david. said... plustoken scammers didn’t just steal $ + billion worth of cryptocurrency. they may also be driving down the price of bitcoin from chainalysis has the details of the plustoken scammers use of the huboi exchange to cash out the winnings from their ponzi scheme. and there is more where that came from: "they’ve cashed out at least , of that initial , eth, while the other , has been sitting untouched in a single ethereum wallet for months. the flow of the , stolen bitcoin is more complicated. so far, roughly , of it has been cashed out." december , at : pm david. said... in china electricity crackdown sparks concerns, paul muir reports on yet another way in which bitcoin mining is centralized: "a recent crackdown in china on bitcoin miners who were using electricity illegally – about , machines were seized in hebei and shanxi provinces – raises concerns about the danger of so much of the leading cryptocurrency’s hash rate being concentrated in the totalitarian country, according to a crypto industry observer. ... however, he pointed out that just four regions in china account for % of the world’s hash rate and siachen alone is responsible for %. therefore, if china decides to shut down network access, it could be very problematic." december , at : am david. said... catalin cimpanu's chrome extension caught stealing crypto-wallet private keys is yet another example of the things you have to trust to work in the "trustless" world of cryptocurrencies: "a google chrome extension was caught injecting javascript code on web pages to steal passwords and private keys from cryptocurrency wallets and cryptocurrency portals. the extension is named shitcoin wallet (chrome extension id: ckkgmccefffnbbalkmbbgebbojjogffn), and was launched last month, on december ." january , at : am david. said... in blockchain, all over your face, jemima kelly writes: "enter the “blockchain creme” from cosmetique bio naturelle suisse (translation: swiss organic natural cosmetics). we thought it must be a joke when we first heard about it ... but yet here it is, being sold on the actual internet:" january , at : am david. said... adriana hamacher's is this the end of malta's reign as blockchain island? reports on reality breaking in on malta's blockchain hype: "malta’s technicolor blockchain dream has turned an ominous shade of grey. last weekend, prime minister joseph muscat—chief architect of the tiny mediterranean island’s pioneering policies in blockchain, gaming and ai—was obliged to step down amid the crisis surrounding the murder of investigative journalist daphne caruana galizia." it appears that the government's response to her criticisms was to put a large bomb in her car: "anonymous maltese blogger bugm, one of many determined to bring caruana galizia’s killers to justice, believes that an aggressively pro-blockchain policy was seized upon by the government to distract attention from the high-profile murder investigation that ensued." the whole article is worth a read. january , at : am david. said... jill carlson's trust no one. not even a blockchain is a skeptical response to emily parker’s credulous the truth is all there is. but in focusing on "garbage in, garbage out" carlson is insufficiently skeptical about the immutability of blockchains. january , at : am david. said... david canellis reports that bitcoin gold hit by % attacks, $ k in cryptocurrency double-spent: "malicious cryptocurrency miners took control of bitcoin btc gold‘s blockchain recently to double-spend $ , worth of btg. bad actors assumed a majority of the network‘s processing power (hash rate) to re-organize the blockchain twice between thursday and friday last week: the first netted attackers , btg ($ , ), and the second roughly , btg ($ , ). cryptocurrency developer james lovejoy estimates the miners spent just $ , to perform each of the attacks, based on prices from hash rate marketplace nicehash. this marks the second and third times bitcoin gold has suffered such incidents in two years." mutating immutability can be profitable. investing $ . k to get $ k is a , % return in days. find me another investment with that rate of return! january , at : pm david. said... john nugée's what libra means for money creation points out two big problems that libra, or any private stablecoin system, would have for the economy: "the introduction of libra threatens to split the banking sector’s currently unified balance sheet, by moving a significant proportion of customer deposits (that is, the banking system’s liabilities) to the digital currency issuers, while leaving customer loans and overdrafts (the banking system’s assets) with the banks. the inevitable result of this would be to force the banks to reduce the asset side of their balance sheet to match the reduced liability side – in other words reduce their loans. this would almost certainly lead to a major credit squeeze, which would be highly damaging to economic activity." and: "it is by no means clear that such a private sector payment system would be cheaper to operate than the existing bank-based system even on the narrow point of cost per transaction, particularly if, as seems probable, one digital currency soon becomes dominant to the exclusion of competitors. but there is the wider issue of whether society is advantaged by big tech creaming off yet more money from the economy into an unaccountable, untaxable and often overseas behemoth." february , at : am david. said... yet another company doing real stuff that started out enthusiastic finds out they don't need a blockchain: "we have run several pocs integrating blockchain technology but we so far decided to run our core services without blockchain technology. meaning, the solutions that we are already providing are working fine without dlt." february , at : pm david. said... drug dealer loses codes for € . m bitcoin accounts by conor lally reports that btc are frozen in wallets for which the codes have been lost. presumably there is a fairly constant average background rate at which btc are frozen in this way, in effect being destroyed. btc are being created by the block reward, which is decreasing. eventually, the rate of creation will fall below the rate of freezing and the universe of usable btc will shrink, driving the value of the remaining btc "to the moon" february , at : am david. said... david gerard summarizes libra: "libra can only work if libra can evade regulation — and simultaneously, that no system that libra’s competing with can evade regulation. and regulators would have to let libra do this, for some reason. i’m not convinced." march , at : am david. said... trolly mctrollface's theory about the backing for tether is worth considering: "we all know miners need real cash to pay their electricity bills. they could sell their rewards on exchanges - which someone (cough tether cough) would have to buy, to prevent $btc from crashing. but when everyone knows that everyone knows, something different happens. tether has real money, because bitfinex has real money from sheering muppets dumb enough to trade on its exchange. but why would they buy miners' bitcoins, when they can loan them the money instead, and get a death grip on their balls? tether is secured by these loans, not cash. in any case, tether would be secured by loans, not cash. nobody keeps $ b in cash in a bank account, especially not the kind of bank that would accept tether's pedo laundromat money. too much credit risk. ... what if, ... you called up bitcoin miners, and offered them a lifeline, promising them to pay their electricity bills, in exchange of a small favour - a promise they won't sell their bitcoin for a while? let's put these bitcoins in escrow, or, in finance words, let's offer miners a loan secured by their bitcoins. miners are happy because they can pay their bills without going through all the trouble of selling their rewards, while tether is happy because, well, when someone owes you a lot of money, you have a metaphorical gun to his head." hat tip to david gerard, who writes: "in just six weeks, tether’s circulation has doubled to billion usdt! gosh, those stablecoins sure are popular! ... don’t believe those tether conspiracy theorists who think that . billion tethers being pumped into the crypto markets since march has anything to do with keeping the bitcoin price pumped" may , at : am david. said... jemima kelly's goldman sachs betrays bitcoin reports that goldman has seen the light: "we believe that a security whose appreciation is primarily dependent on whether someone else is willing to pay a higher price for it is not a suitable investment for our clients." may , at : am david. said... crimes on a public immutable ledger are risky, as three alleged perpetrators of the twitter hack discovered: "three individuals have been charged today for their alleged roles in the twitter hack that occurred on july , , the us department of justice has announced. ... the cyber crimes unit “analyzed the blockchain and de-anonymized bitcoin transactions allowing for the identification of two different hackers. this case serves as a great example of how following the money, international collaboration, and public-private partnerships can work to successfully take down a perceived anonymous criminal enterprise,” agent jackson said." july , at : pm david. said... implausibly good opsec is necessary if you're committing crimes on an immutable public ledger. tim cushing's fbi used information from an online forum hacking to track down one of the hackers behind the massive twitter attack reveals how mason john sheppard was exposed as one of the perpetrators when his purchase of a video game username sent bitcoin to address zsdvpv rkdiqn v v w fdqvk pdf . the fbi found this in a public database resulting from the compromise of an on-line forum: "available on various websites since approximately april . on or about april , , the fbi obtained a copy of this database. the fbi found that the database included all public forum postings, private messages between users, ip addresses, email addresses, and additional user information. also included for each user was a list of the ip addresses that user used to log into the service along with a corresponding date and timestamp." august , at : am david. said... alex de vries' bitcoin’s energy consumption is underestimated: a market dynamics approach shows that: "most of the currently used methods to estimate bitcoin’s energy demand are still prone to providing optimistic estimates. this happens because they apply static assumptions in defining both market circumstances (e.g. the price of available electricity) as well as the subsequent behavior of market participants. in reality, market circumstances are dynamic, and this should be expected to affect the preferences of those participating in the bitcoin mining industry. the various choices market participants make ultimately determines the amount of resources consumed by the bitcoin network. it will be shown that, when starting to properly consider the previous dynamics, even a conservative estimate of the bitcoin network’s energy consumption per september ( ) would be around . twh annually (comparable to a country like belgium)" tip of the hat to david gerard. august , at : pm david. said... in the tether whitepaper and you, cas piancey goes through the tether "white paper" with a fine-tooth comb: "since many tether defenders are intent on making arguments about the actual promises tether has committed to (ie; “tether isn’t a cryptocurrency!” “tether doesn’t need to be fully backed!”), it felt like the right time to run through as much of the tether whitepaper as possible. hopefully, through a long-form analysis of the whitepaper (which hasn’t been updated), we can come to conclusions about what promises tether has kept, and what promises tether has broken." one guess as to how many it has kept! hat tip to david gerard. october , at : pm david. said... in ibm blockchain is a shell of its former self after revenue misses, job cuts: sources ian allison reports that reality has dawned at ibm: "ibm has cut its blockchain team down to almost nothing, according to four people familiar with the situation. job losses at ibm escalated as the company failed to meet its revenue targets for the once-fêted technology by % this year, according to one of the sources. “ibm is doing a major reorganization,” said a source at a startup that has been interviewing former ibm blockchain staffers. “there is not really going to be a blockchain team any longer. most of the blockchain people at ibm have left.” ibm’s blockchain unit missed its revenue targets by a wide margin for two years in a row, said a second source. expectations for enterprise blockchain were too high, they said, adding that ibm “didn’t really manage to execute, despite doing a lot of announcements.” a spokesperson for ibm denied the claims." february , at : pm post a comment newer post older post home subscribe to: post comments (atom) blog rules posts and comments are copyright of their respective authors who, by posting or commenting, license their work under a creative commons attribution-share alike . united states license. off-topic or unsuitable comments will be deleted. dshr dshr in anwr recent comments full comments blog archive ► ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ▼ ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ▼ july ( ) blockchain briefing for dod boeing's corporate suicide carl malamud's text mining project not to pick on toyota the eff vs. dmca section finn brunton's "digital cash" the web is a low-trust society ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► july ( ) ► june ( ) ► february ( ) ► ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) lockss system has permission to collect, preserve, and serve this archival unit. simple theme. powered by blogger. dshr's blog: talk at berkeley's information access seminar dshr's blog i'm david rosenthal, and this is a place to discuss the work i'm doing in digital preservation. friday, february , talk at berkeley's information access seminar once again cliff lynch invited me to give a talk to the information access seminar at uc berkeley's ischool. preparation time was limited because these days i'm a full-time grandparent so the talk, entitled securing the digital supply chain summarizes and updates two long posts from two years ago: certificate transparency securing the software supply chain the abstract was: the internet is suffering an epidemic of supply chain attacks, in which a trusted supplier of content is compromised and delivers malware to some or all of their clients. the recent solarwinds compromise is just one glaring example. this talk reviews efforts to defend digital supply chains. below the fold, the text of the talk with links to the sources. solarwinds, and many other recent system and network compromises have been supply chain attacks. these are extremely efficient, because unlike one-at-a-time attacks such as phishing, they provide a built-in mass deployment mechanism. a single compromise of solarwinds infected at least , networks. clearly, the vendors' security practices, and their vendors' security practices, and so on ad infinitum are important, but the sad truth is that current digital supply chain technologies are incapable of mitigating the inevitable security lapses along the chain. this talk reviews the efforts to defend supply chains that deliver digital content, such as software. but lets start with a simpler case, web pages. web page supply chain how do i know that i'm talking to the right web site? because there's a closed padlock icon in the url bar, right? [slide ] mozilla says: a green padlock (with or without an organization name) indicates that: you are definitely connected to the website whose address is shown in the address bar; the connection has not been intercepted. the connection between firefox and the website is encrypted to prevent eavesdropping. nb - this is misleading! the padlock icon appears when the browser has validated that the connection to the url in the url bar supplied a certificate for the site in question carrying a signature chain ending in one of the root certificates the browser trusts. browsers come with a default list of root certificates from certificate authorities (cas). my firefox trusts certificates from different organizations including, for example, amazon and google, but also chunghwa telecom co., ltd. and the dutch government. why is this list a problem? the browser trusts all of them equally. the browser trusts cas that the cas on the list delegate trust to. back in , the eff found more than organizations that internet explorer and firefox trusted. commercial cas on the list, and cas they delegate to, have regularly been found to be issuing false or insecure certificates. [slide ] one of these trusted organizations is the internet security research group, a not-for-profit organization hosted by the linux foundation and sponsored by many organizations including mozilla and the eff, which has greatly improved the information hygiene of the web through a program called let's encrypt. this has provided over million web sites with free certificates carrying a signature chain rooted in a certificate that almost all browsers trust. my blog's certificate is one of them, as you can see by clicking on the padlock icon. [slide ] barysevich identified four such sellers of counterfeit certificates since . two of them remain in business today. the sellers offered a variety of options. in , one provider calling himself c@t advertised certificates that used a microsoft technology known as authenticode for signing executable files and programming scripts that can install software. c@t offered code-signing certificates for macos apps as well. ... "in his advertisement, c@t explained that the certificates are registered under legitimate corporations and issued by comodo, thawte, and symantec—the largest and most respected issuers," dan goodin one-stop counterfeit certificate shops for all your malware-signing needs abuse of the trust users place in cas is routine: in one case, a prominent dutch ca (diginotar) was compromised and the hackers were able to use the ca’s system to issue fake ssl certificates. the certificates were used to impersonate numerous sites in iran, such as gmail and facebook, which enabled the operators of the fake sites to spy on unsuspecting site users. ... more recently, a large u.s.-based ca (trustwave) admitted that it issued subordinate root certificates to one of its customers so the customer could monitor traffic on their internal network. subordinate root certificates can be used to create ssl certificates for nearly any domain on the internet. although trustwave has revoked the certificate and stated that it will no longer issue subordinate root certificates to customers, it illustrates just how easy it is for cas to make missteps and just how severe the consequences of those missteps might be. in sennheiser provided another example: the issue with the two headsetup apps came to light earlier this year when german cyber-security firm secorvo found that versions . , . , and . installed two root certification authority (ca) certificates into the windows trusted root certificate store of users' computers but also included the private keys for all in the senncomcckey.pem file. certificates depend on public-key cryptography, which splits keys into public/private key pairs. private keys can decrypt text encrypted by the public key, and vice versa. the security of the system depends upon private keys being kept secret. this poses two problems: as the sennheiser example shows, it is easy for the private keys to leak. another common way for them to leak is for a server to be compromised. for the server to be able to verify its identity, and thus unlock the padlock, the private key needs to be stored on the server in cleartext. so an intruder can steal it to impersonate the server. there is no alarm bell or notification to the owner or affected users when a private key leaks. so, as in the sennheiser case, the attacker may be able to use it unimpeded for a long time, until some security researcher notices some anomaly. catalin cimpanu continues: in a report published today, secorvo researchers published proof-of-concept code showing how trivial would be for an attacker to analyze the installers for both apps and extract the private keys. making matters worse, the certificates are also installed for mac users, via headsetup macos app versions, and they aren't removed from the operating system's trusted root certificate store during current headsetup updates or uninstall operations. ... sennheiser's snafu ... is not the first of its kind. in , lenovo shipped laptops with a certificate that exposed its private key in a scandal that became known as superfish. dell did the exact same thing in in a similarly bad security incident that became known as edellroot. cimpanu also reports on a more recent case: under the guise of a "cybersecurity exercise," the kazakhstan government is forcing citizens in its capital of nur-sultan (formerly astana) to install a digital certificate on their devices if they want to access foreign internet services. once installed, the certificate would allow the government to intercept all https traffic made from users' devices via a technique called mitm (man-in-the-middle). this type of “mistake” allows attackers to impersonate any web site to affected devices. cas are supposed to issue three grades of certificate based on increasingly rigorous validation: domain validated (dv) certificates verify control over the dns entries, email and web content of the specified domain. they can be issued via automated processes, as with let's encrypt. organization validated (ov) certificates are supposed to verify the legal entity behind the dv-level control of the domain, but in practice are treated the same as dv certificates. extended validation (ev) certificates require "verification of the requesting entity's identity by a certificate authority (ca)". verification is supposed to be an intrusive, human process. [slide ] source but, as can be seen from the advert, the extended verification process is far from fool-proof. this lack of trustworthiness of cas should not be a surprise. six years ago security collapse in the https market, a fascinating analysis of the (lack of) security on the web from an economic rather than a technical perspective by axel arnbak et al from amsterdam and delft universities showed that cas lack incentives to be trustworthy. they write: information asymmetry prevents buyers from knowing what cas are really doing. buyers are paying for the perception of security, a liability shield, and trust signals to third parties. none of these correlates verifiably with actual security. given that ca security is largely unobservable, buyers’ demands for security do not necessarily translate into strong security incentives for cas. negative externalities of the weakest-link security of the system exacerbate these incentive problems. the failure of a single ca impacts the whole ecosystem, not just that ca’s customers. all other things being equal, these interdependencies undermine the incentives of cas to invest, as the security of their customers depends on the efforts of all other cas. the reason for the weakest-link is: a crucial technical property of the https authentication model is that any ca can sign certificates for any domain name. in other words, literally anyone can request a certificate for a google domain at any ca anywhere in the world, even when google itself has contracted one particular ca to sign its certificate. this "technical property" is actually important, it is what enables a competitive market of cas. symantec in particular has exploited it wholesale: google's investigation revealed that over a span of years, symantec cas have improperly issued more than , certificates. ... they are a major violation of the so-called baseline requirements that major browser makers impose of cas as a condition of being trusted by major browsers. but symantec has suffered no effective sanctions because they are too big to fail: symantec's repeated violations underscore one of the problems google and others have in enforcing terms of the baseline requirements. when violations are carried out by issuers with a big enough market share they're considered too big to fail. if google were to nullify all of the symantec-issued certificates overnight, it might cause widespread outages. my firefox still trusts symantec root certificates. because google, mozilla and others prioritize keeping the web working over keeping it secure, deleting misbehaving big cas from trust lists won't happen. when mozilla writes: you are definitely connected to the website whose address is shown in the address bar; the connection has not been intercepted. they are assuming a world of honest cas that isn't this world. if you have the locked padlock icon in your url bar, you are probably talking to the right web site, but there is a chance you aren't. [slide ] recent data from anti-phishing company phishlabs shows that percent of all phishing sites in the third quarter of bore the padlock security icon next to the phishing site domain name as displayed in a browser address bar. that’s up from percent just one year ago, and from percent in the second quarter of . brian krebs half of all phishing sites now have the padlock building on earlier work by wendlandt et al, moxie marlinspike, the eff and others, in google started work on an approach specified in rfc , and called certificate transparency (ct). the big difference from earlier efforts, which didn't require cooperation from website owners and cas, was that google's did require cooperation and they had enough leverage to obtain it: [slide ] google's certificate transparency project fixes several structural flaws in the ssl certificate system, which is the main cryptographic system that underlies all https connections. these flaws weaken the reliability and effectiveness of encrypted internet connections and can compromise critical tls/ssl mechanisms, including domain validation, end-to-end encryption, and the chains of trust set up by certificate authorities. if left unchecked, these flaws can facilitate a wide range of security attacks, such as website spoofing, server impersonation, and man-in-the-middle attacks. certificate transparency helps eliminate these flaws by providing an open framework for monitoring and auditing ssl certificates in nearly real time. specifically, certificate transparency makes it possible to detect ssl certificates that have been mistakenly issued by a certificate authority or maliciously acquired from an otherwise unimpeachable certificate authority. it also makes it possible to identify certificate authorities that have gone rogue and are maliciously issuing certificates. certificate transparency the basic idea is to accompany the certificate with a hash of the certificate signed by a trusted third party, attesting that the certificate holder told the third party that the certificate with that hash was current. thus in order to spoof a service, an attacker would have to both obtain a fraudulent certificate from a ca, and somehow persuade the third party to sign a statement that the service had told them the fraudulent certificate was current. clearly this is: more secure than the current situation, which requires only compromising a ca, and: more effective than client-only approaches, which can detect that a certificate has changed but not whether the change was authorized. ct also requires participation from browser manufacturers: in order to improve the security of extended validation (ev) certificates, google chrome requires certificate transparency (ct) compliance for all ev certificates issued after jan . clients now need two lists of trusted third parties, the cas and the sources of ct attestations. the need for these trusted third parties is where the blockchain enthusiasts would jump in and claim (falsely) that using a blockchain would eliminate the need for trust. but ct has a much more sophisticated approach, ronald reagan's "trust, but verify". in the real world it isn't feasible to solve the problem of untrustworthy cas by eliminating the need for trust. ct's approach instead is to provide a mechanism by which breaches of trust, both by the cas and by the attestors, can be rapidly and unambiguously detected. [slide ] source here is a brief overview of how ct works to detect breaches of trust. the system has the following components: logs, to which cas report their current certificates, and from which they obtain attestations, called signed certificate timestamps (scts), that owners can attach to their certificates. clients can verify the signature on the sct, then verify that the hash it contains matches the certificate. if it does, the certificate was the one that the ca reported to the log, and the owner validated. it is envisaged that there will be tens but not thousands of logs; chrome currently trusts logs operated by organizations. each log maintains a merkle tree data structure of the certificates for which it has issued scts. monitors, which periodically download all newly added entries from the logs that they monitor, verify that they have in fact been added to the log, and perform a series of validity checks on them. they also thus act as backups for the logs they monitor. auditors, which use the merkle tree of the logs they audit to verify that certificates have been correctly appended to the log, and that no retroactive insertions, deletions or modifications of the certificates in the log have taken place. clients can use auditors to determine whether a certificate appears in a log. if it doesn't, they can use the sct to prove that the log misbehaved. in this way, auditors, monitors and clients cooperate to verify the correct operation of logs, which in turn provides clients with confidence in the [certificate,attestation] pairs they use to secure their communications. although the process works if certificate owners each obtain their scts from only one log, they should get them from multiple logs and send a random selection of their scts to each client to improve robustness. note the key architectural features of ct: [slide ] certificate transparency architecture: the certificate data is held by multiple independent services. they get the data directly from the source, not via replication from other services. clients access the data from a random selection of the services. there is an audit process continually monitoring the services looking for inconsistencies. these are all also features of the protocol underlying the lockss digital preservation system, published in . in both cases, the random choice among a population of independent services makes life hard for attackers. if they are to avoid detection, they must compromise the majority of the services, and provide correct information to auditors while providing false information to victims. looking at the list of logs chrome currently trusts, it is clear that almost all are operated by cas themselves. assuming that each monitor at each ca is monitoring some of the other logs as well as the one it operates, this does not represent a threat, because misbehavior by that ca would be detected by other cas. a ca's monitor that was tempted to cover up misbehavior by a different ca's log it was monitoring would risk being "named and shamed" by some other ca monitoring the same log, just as the misbehaving ca would be "named and shamed". it is important to observe that, despite the fact that cas operate the majority of the ct infrastructure, its effectiveness in disciplining cas is not impaired. all three major cas have suffered reputational damage from recent security failures, although because they are "too big to fail" this hasn't impacted their business much. however, as whales in a large school of minnows it is in their interest to impose costs (for implementing ct) and penalties (for security lapses) on the minnows. note that google was sufficiently annoyed with symantec's persistent lack of security that it set up its own ca. the threat that their business could be taken away by the tech oligopoly is real, and cooperating with google may have been the least bad choice. because these major corporations have an incentive to pay for the ct infrastructure, it is sustainable in a way that a market of separate businesses, or a permissionless blockchain supported by speculation in a cryptocurrency would not be. fundamentally, if applications such as ct attempt to provide absolute security they are doomed to fail, and their failures will be abrupt and complete. it is more important to provide the highest level of security compatible with resilience, so that the inevitable failures are contained and manageable. this is one of the reasons why permissionless blockchains, subject to % attacks, and permissioned blockchains, with a single, central point of failure, are not suitable. software supply chain [slide ] when the mass compromise came to light last month, microsoft said the hackers also stole signing certificates that allowed them to impersonate any of a target’s existing users and accounts through the security assertion markup language. typically abbreviated as saml, the xml-based language provides a way for identity providers to exchange authentication and authorization data with service providers. the full impact of the recent compromise of solarwind's orion network management software will likely never be known, it affected at least , networks, including microsoft's and: the treasury department, the state department, the commerce department, the energy department and parts of the pentagon it was not detected by any of the us government's network monitoring systems, but by fireeye, a computer security company that was also a victim. but for a mistake by the attackers at fireeye it would still be undetected. it was an extremely sophisticated attack, which has rightfully gained a lot of attention. to understand how defenses against attacks like this might work, it is first necessary to understand how the supply chain that installs and updates the software on your computer works. i'll use apt, the system used by debian linux and its derivatives, as the example. a system running debian or another apt-based linux distribution runs software it received in "packages" that contain the software files, and metadata that includes dependencies. their hashes can be verified against those in a release file, signed by the distribution publisher. packages come in two forms, source and compiled. the source of a package is signed by the official package maintainer and submitted to the distribution publisher. the publisher verifies the signature and builds the source to form the compiled package, whose hashes are then included in the release file. the signature on the source package verifies that the package maintainer approves this combination of files for the distributor to build. the signature on the release file verifies that the distributor built the corresponding set of packages from approved sources and that the combination is approved for users to install. [slide ] there are thus two possible points of entry for an attacker: they could compromise the developer, so that the signed source code files received by the distributor contained malware (type a), or they could compromise the distributor, so that the package whose hash was in the signed release file did not reflect the signed source code, but contained malware (type b). an example of a type a attack occurred in november . dan goodin reported that: the malicious code was inserted in two stages into event-stream, a code library with million downloads that's used by fortune companies and small startups alike. in stage one, version . . , published on september , included a benign module known as flatmap-stream. stage two was implemented on october when flatmap-stream was updated to include malicious code that attempted to steal bitcoin wallets and transfer their balances to a server located in kuala lumpur. how were the attackers able to do this? goodin explains: according to the github discussion that exposed the backdoor, the longtime event-stream developer no longer had time to provide updates. so several months ago, he accepted the help of an unknown developer. the new developer took care to keep the backdoor from being discovered. besides being gradually implemented in stages, it also narrowly targeted only the copay wallet app. the malicious code was also hard to spot because the flatmap-stream module was encrypted. all that was needed to implement this type a attack was e-mail and github accounts, and some social engineering. dan goodin describes a simple type b attack in new supply chain attack uses poisoned updates to infect gamers’ computers: in a nutshell, the attack works this way: on launch, nox.exe sends a request to a programming interface to query update information. the bignox api server responds with update information that includes a url where the legitimate update is supposed to be available. eset speculates that the legitimate update may have been replaced with malware or, alternatively, a new filename or url was introduced. malware is then installed on the target’s machine. the malicious files aren’t digitally signed the way legitimate updates are. that suggests the bignox software build system isn’t compromised; only the systems for delivering updates are. the malware performs limited reconnaissance on the targeted computer. the attackers further tailor the malicious updates to specific targets of interest. [slide ] source the solarwinds attackers tried but failed to penetrate the network of crowdstrike, another computer security company. sunspot: an implant in the build process, crowdstrike's analysis of the attack, reveals the much greater sophistication of this type b attack. once implanted in solarwinds' build system: sunspot runs once a second scanning for instances of msbuild.exe, the tool used to build the target software. if sunspot finds an msbuild.exe, it next locates the directory in which the build is running. then sunspot checks whether what is being built is the target software. if it is, sunspot checks whether the target source file has changed. if it hasn't, sunspot carefully substitutes the modified source file for the target source file. sunspot waits until the build completes, then carefully restores the target source file and erases the traces of its work. solarwinds forensic timeline shows that the attackers penetrated their network in september , and a month later tested sunspot by injecting test code into the next release of orion. an improved sunspot was deployed from february to june, when it was removed having successfully compromised the orion release with the production malware. no-one noticed until december, when fireeye spotted suspicious activity on their internal network and traced it to orion. microsoft's analysis reveals a lot more sophistication of the attacker's operations once they had penetrated the network: [slide ] each cobalt strike dll implant was prepared to be unique per machine and avoided at any cost overlap and reuse of folder name, file name, export function names, c domain/ip, http requests, timestamp, file metadata, config, and child process launched. this extreme level of variance was also applied to non-executable entities, such as wmi persistence filter name, wmi filter query, passwords used for -zip archives, and names of output log files. how could software supply chains be enhanced to resist these attacks? in an important paper entitled software distribution transparency and auditability, benjamin hof and georg carle from tu munich: describe how apt works to maintain up-to-date software on clients by distributing signed packages. review previous efforts to improve the security of this process. propose to enhance apt's security by layering a system similar to certificate transparency (ct) on top. detail the operation of their systems' logs, auditors and monitors, which are similar to ct's in principle but different in detail. describe and measure the performance of an implementation of their layer on top of apt using the trillian software underlying some ct implementations. their system's ct-like logs contain the hashes of both the source and the binaries of each version of each package, and ensure that attackers would be detected if they, for example, create a short-lived version containing malware for a specific victim. it certainly defeats a significant class of attacks but, alas, does not address either the solarwinds or the event-stream attacks. as regards the solarwinds attack, there are two important "missing pieces" in their system, and all the predecessors. each is the subject of a separate effort: [slide ] reproducible builds. bootstrappable compilers. suppose solarwinds had been working in hof and carle's system. they would have signed their source code, built it, and signed the resulting binaries. the attackers would have arranged that the source that was built was not the source that solarwinds signed, but solarwinds would not have known that. so the signatures on both the unmodified source and the modified binaries would appear valid in the logs, but the binaries would be malign. the problem is that the connection between the source and the binaries rests on an assumption that the distributor's build environment has not been compromised - i.e. no type b attack. as with the multiple logs of ct, what is needed is multiple independent builds of the signed source. unless all of the independent build environments are compromised, a compromised build will differ from the others because it contains malware. this is a great idea, but in practice it is very hard to achieve for both technical and organizational reasons: the first technical reason is that in general, building the same source twice results in different binaries. compiler and linker output typically contains timestamps, temporary file names, and other sources of randomness. the build system needs to be reproducible. the second technical reason is that, in order to be reproducible, the multiple independent builds have to use the same build environment. so each of the independent build environments will have the same vulnerabilities, allowing for the possibility that the attacker could compromise them all. the organizational reason is that truly independent builds can only be done in an open source environment in which anyone, and in particular each of the independent builders, can access the source code. to enable binaries to be securely connected to their source, a reproducible builds effort has been under way for more than years. debian project lead chris lamb's -minute talk think you're not a target? a tale of developers ... provides an overview of the problem and the work to solve it using three example compromises: alice, a package developer who is blackmailed to distribute binaries that don't match the public source (a type a attack). bob, a build farm sysadmin whose personal computer has been compromised, leading to a compromised build toolchain in the build farm that inserts backdoors into the binaries (a type b attack). carol, a free software enthusiast who distributes binaries to friends. an evil maid attack has compromised her laptop. as lamb describes, eliminating all sources of irreproducibility from a package is a painstaking process because there are so many possibilities. they include non-deterministic behaviors such as iterating over hashmaps, parallel builds, timestamps, build paths, file system directory name order, and so on. the work started in with % of debian packages building reproducibly. currently, around % of the debian packages for the amd and arm architectures are now reproducible. that is good, but % coverage is really necessary to provide security. [slide ] way back in , paul karger and roger schell discovered a devastating attack against computer systems. ken thompson described it in his classic speech, "reflections on trusting trust." basically, an attacker changes a compiler binary to produce malicious versions of some programs, including itself. once this is done, the attack perpetuates, essentially undetectably. thompson demonstrated the attack in a devastating way: he subverted a compiler of an experimental victim, allowing thompson to log in as root without using a password. the victim never noticed the attack, even when they disassembled the binaries -- the compiler rigged the disassembler, too. in , bruce schneier summarized the message of perhaps the most famous of acm's annual turing award lectures. in this attack, the compromised build environment inserts malware even though it is building the unmodified source code. unlike the solarwinds attack, the signatures testifying that the binaries are the output of building the signed source code are correct. [slide ] source this is the motivation for the bootstrappable builds project, whose goal is to create a process for building a complete toolchain starting from a "seed" binary that is simple enough to be certified "by inspection". recently, they achieved a major milestone. starting from a tiny "seed" binary, they were able to create a working tinycc compiler for the arm architecture. starting from tinycc, it is possible to build the entire gnucc toolchain and thus, in principle, a working linux. there is clearly a long way still to go to a bootstrapped full toolchain proof against type b attacks. the event-stream attack can be thought of as the organization-level analog of a sybil attack on a peer-to-peer system. creating an e-mail identity is almost free. the defense against sybil attacks is to make maintaining and using an identity in the system expensive. as with proof-of-work in bitcoin, the idea is that the white hats will spend more (compute more useless hashes) than the black hats. even this has limits. eric budish's analysis shows that, if the potential gain from an attack on a blockchain is to be outweighed by its cost, the value of transactions in a block must be less than the block reward. would a similar defense against "sybil" type a attacks on the software supply chain be possible? there are a number of issues: the potential gains from such attacks are large, both because they can compromise very large numbers of systems quickly (event-stream had m downloads), and because the banking credentials, cryptocurrency wallets, and other data these systems contain can quickly be converted into large amounts of cash. thus the penalty for mounting an attack would have to be an even larger amount of cash. package maintainers would need to be bonded or insured for large sums, which implies that distributions and package libraries would need organizational structures capable of enforcing these requirements. bonding and insurance would be expensive for package maintainers, who are mostly unpaid volunteers. there would have to be a way of paying them for their efforts, at least enough to cover the costs of bonding and insurance. thus users of the packages would need to pay for their use, which means the packages could neither be free, nor open source. which would make implementing the reproducible builds and bootstrapped compilers needed to defend against type b attacks extremely difficult. the foss (free open source software) movement will need to find other ways to combat sybil attacks, which will be hard if the reward for a successful attack greatly exceeds the cost of mounting it. adequately rewarding maintainers for their essential but under-appreciated efforts is a fundamental problem for foss. it turns out that this talk is timely. two days ago, eric brewer, rob pike et al from google posted know, prevent, fix: a framework for shifting the discussion around vulnerabilities in open source, an important and detailed look at the problem of vulnerabilities in open source and what can be done to reduce them. their summary is: it is common for a program to depend, directly or indirectly, on thousands of packages and libraries. for example, kubernetes now depends on about , packages. open source likely makes more use of dependencies than closed source, and from a wider range of suppliers; the number of distinct entities that need to be trusted can be very high. this makes it extremely difficult to understand how open source is used in products and what vulnerabilities might be relevant. there is also no assurance that what is built matches the source code. taking a step back, although supply-chain attacks are a risk, the vast majority of vulnerabilities are mundane and unintentional—honest errors made by well-intentioned developers. furthermore, bad actors are more likely to exploit known vulnerabilities than to find their own: it’s just easier. as such, we must focus on making fundamental changes to address the majority of vulnerabilities, as doing so will move the entire industry far along in addressing the complex cases as well, including supply-chain attacks. the bulk of their post addresses improvements to the quality of the development process, with three goals: know about the vulnerabilities in your software prevent the addition of new vulnerabilities, and fix or remove vulnerabilities. then, in a section entitled prevention for critical software they specifially address the security of the development process and thus the two types of supply chain attacks we have been discussing. they write: this is a big task, and currently unrealistic for the majority of open source. part of the beauty of open source is its lack of constraints on the process, which encourages a wide range of contributors. however, that flexibility can hinder security considerations. we want contributors, but we cannot expect everyone to be equally focused on security. instead, we must identify critical packages and protect them. such critical packages must be held to a range of higher development standards, even though that might add developer friction. [slide ] define criteria for “critical” open source projects that merit higher standards no unilateral changes to critical software require code review for critical software changes to critical software require approval by two independent parties authentication for participants in critical software for critical software, owners and maintainers cannot be anonymous strong authentication for contributors of critical software a federated model for identities notification for changes in risk transparency for artifacts trust the build process their goals for the "higher development standards" include identifying the important packages that require higher standards, implementing review and signoff of changes by at least two independent developers, "transparency for artifacts", by which they mean reproducible builds, and "trust the build process" which implies a bootstrappable toolchain. they acknowledge that these are very aggressive goals, because in many ways they cut against the free-wheeling development culture of open source that has sparked its remarkable productivity. if google were to persuade other major corporations to put significant additional resources of money and manpower into implementing them they would likely succeed. absent this, the additional load on developers will likely cause resistance. posted by david. at : pm labels: security no comments: post a comment older post home subscribe to: post comments (atom) blog rules posts and comments are copyright of their respective authors who, by posting or commenting, license their work under a creative commons attribution-share alike . united states license. off-topic or unsuitable comments will be deleted. dshr dshr in anwr recent comments full comments blog archive ▼ ( ) ▼ february ( ) talk at berkeley's information access seminar chromebook linux update ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► july ( ) ► june ( ) ► february ( ) ► ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) lockss system has permission to collect, preserve, and serve this archival unit. simple theme. powered by blogger. none none code lib skip to main content toggle navigation home schedule speakers sponsors general info conduct & safety code for lib code lib march - • online register now! sponsor code lib! image attributions the conference for people who code for libraries. an annual gathering of technologists from around the world, who largely work for and with libraries, archives, and museums and have a commitment to open technologies. registration registration is open! fees & additional information starting on january th register! proposals posters now through february th submit a poster thanks to our sponsors! platinum welcome to code lib code lib is everything to me. in the community i feel like my work and knowledge is appreciated, so i feel very comfortable and motivated to volunteer, give talks, teach workshops, participate in conferences, host events. it's a great support network, i've never felt as comfortable as i do in this library group! kim pham university of denver the confluence of technology and libraries drew me to code lib when i was a young librarian straddling the areas of library metadata and technology. after eleven years in the community i am still amazed and humbled by the people i meet in the community and the work they do. there isn't another space that seamlessly combines libraries, technology, and the human aspect quite like code lib in the library world. becky yoose i came away from code lib wanting to invite most of the people i met into my office and ask all of the questions about what everyone is doing and how they’re doing it and how can i do those things and what would they change about their tools; what's better is many of them would gladly help. years on i keep coming back for more because over the years technical excellence isn't the only metric used in this community's continued growth. i have made friends in the code lib community. francis kayiwa princeton university libraries code lib offers the space to be self-aware, outwardly conscious, and vastly creative. the community seems to be expanding and learning without ego, and i feel lucky to have been welcomed into the group with open arms. the conference is a place where one can look holistically at technology alongside thoughtful practitioners, while building lasting friendships. alyssa loera cal poly pomona code lib has been transformative for me. when i first learned of code lib, i was considering leaving libraryland. attending the first code lib conference opened my eyes to the community i never knew i had. code lib continues to humble, to inspire, and to anchor; our collective work is grounded in the cultural heritage mission and in the value of working inclusively in the open for the collective good. here's to another twelve years, code lib---and then some! michael giarlo stanford university i attended code lib on a diversity scholarship and i will always be grateful for that opportunity. it was free of buzzwords, full of welcoming people, and the sessions were interesting and accessible even though i don't work closely with technology or coding. i'm more motivated to explore new areas of technology and librarianship, i've started volunteering with the web committee, and i'm looking forward to attending the conference again! laura chuang attending my first code lib allowed me to explore the potential of technology, while upholding the importance of empathy and community-building. the connections i made at code lib have continued to deepen over the last year, and it has been fantastic to see how we have implemented ideas that were shaped by conversations there. code lib has modeled accountability and care, including publicly standing up against harassment and organizing support for our community. nicky andrews north carolina state university code lib has been a great conference for me as a metadata person interested in gaining computer science skills and insights. presentations and topics are selected by the community. as such, i find a greater portion of presentations at this conference to be interesting, relevant, and educational than at other conferences in which presentations are jury selected. they also offer generous scholarships to underrepresented folks in the code lib community. yay!. sonoe nakasone north carolina state university libraries at code lib, you really get the sense that people are there to share with and learn from one another — to advance their work individually and collectively — and have fun while they’re at it. i left the conference reminded of the widespread passion for libraries as critical features of our society, the passion that draws interesting, creative people to library work, and found i had a renewed sense of purpose in my job. hannah frost stanford university home schedule speakers sponsors general info conduct & safety contact us except where otherwise noted, content on this site is licensed under a creative commons attribution . international license. dshr's blog: blockchain: what's not to like? dshr's blog i'm david rosenthal, and this is a place to discuss the work i'm doing in digital preservation. monday, december , blockchain: what's not to like? i gave a talk at the fall cni meeting entitled blockchain: what's not to like? the abstract was: we're in a period when blockchain or "distributed ledger technology" is the solution to everything™, so it is inevitable that it will be proposed as the solution to the problems of academic communication and digital preservation. these proposals typically assume, despite the evidence, that real-world blockchain implementations actually deliver the theoretical attributes of decentralization, immutability, anonymity, security, scalability, sustainability, lack of trust, etc. the proposers appear to believe that satoshi nakamoto revealed the infallible bitcoin protocol to the world on golden tablets; they typically don't appreciate or cite the nearly three decades of research and implementation that led up to it. this talk will discuss the mis-match between theory and practice in blockchain technology, and how it applies to various proposed applications of interest to the cni audience. below the fold, an edited text of the talk with links to the sources, and much additional material. the colored boxes contain quotations that were on the slides but weren't spoken. update: the video of my talk has now been posted on youtube and vimeo. it’s one of these things that if people say it often enough it starts to sound like something that could work, sadhbh mccarthy i'd like to start by thanking cliff lynch for inviting me back even though i'm retired, and for letting me debug the talk at berkeley's information access seminar. i plan to talk for minutes, leaving plenty of time for questions. a lot of information will be coming at you fast. afterwards, i encourage you to consult the whole text of the talk and much additional material on my blog. follow the links to the sources to get the details you may have missed. we're in a period when blockchain or "distributed ledger technology" is the solution to everything™ so it is inevitable that it will be proposed as the solution to the problems of academic communication and digital preservation. in the second of a three-part series ian mulvaney has a comprehensive review of the suggested applications of blockchain in academia in three broad classes: priority claims claims about authorship of a paper reviews of articles tracking article versions from preprint to publication claims about generation of data linking research artefacts together claims about facts and micro statements resources access to compute time access to lab time tracking of physical reagents rights rights transfers around copyright, articles or journals blockchain in stem - part ian mulvaney priority claims access to resources rights mulvaney discusses each in some detail and doesn't find a strong case for any of them. in a third part he looks at some of the implementation efforts currently underway and divides their motivations into two groups. i quote: the first comes from commercial interests where management of rights, ip and ownership is complex, hard to do, and has led to unusable systems that are driving researchers to sites like scihub, scaring the bejesus out of publishers in the process. the other trend is for a desire to move to a decentralised web and a decentralised system of validation and reward, in a way trying to move even further away from the control of publishers. it is absolutely fascinating to me that two diametrically opposite philosophical sides are converging on the same technology as the answer to their problems. could this technology perhaps be just holding up an unproven and untrustworthy mirror to our desires, rather than providing any real viable solutions? this talk answers mulvaney's question in the affirmative. i've been writing skeptically about cryptocurrencies and blockchain technology for more than five years. what are my qualifications for such a long history of pontification? this is not to diminish nakamoto's achievement but to point out that he stood on the shoulders of giants. indeed, by tracing the origins of the ideas in bitcoin, we can zero in on nakamoto's true leap of insight—the specific, complex way in which the underlying components are put together. bitcoin's academic pedigree, arvind narayanan and jeremy clark more than fifteen years ago, nearly five years before satoshi nakamoto published the bitcoin protocol, a cryptocurrency based on a decentralized consensus mechanism using proof-of-work, my co-authors and i won a "best paper" award at the prestigious sosp workshop for a decentralized consensus mechanism using proof-of-work. it is the protocol underlying the lockss system. the originality of our work didn't lie in decentralization, distributed consensus, or proof-of-work. all of these were part of the nearly three decades of research and implementation leading up to the bitcoin protocol, as described by arvind narayanan and jeremy clark in bitcoin's academic pedigree. our work was original only in its application of these techniques to statistical fault tolerance; nakamoto's only in its application of them to preventing double-spending in cryptocurrencies. we're going to walk through the design of a system to perform some function, say monetary transactions, storing files, recording reviewers' contributions to academic communication, verifying archival content, whatever. being of a naturally suspicious turn of mind, you don't want to trust any single central entity, but instead want a decentralized system. you place your trust in the consensus of a large number of entities, which will in effect vote on the state transitions of your system (the transactions, reviews, archival content, ...). you hope the good entities will out-vote the bad entities. in the jargon, the system is trustless (a misnomer). techniques using multiple voters to maintain the state of a system in the presence of unreliable and malign voters were first published in the byzantine generals problem by lamport et al in . alas, byzantine fault tolerance (bft) requires a central authority to authorize entities to take part. in the blockchain jargon, it is permissioned. you would rather let anyone interested take part, a permissionless system with no central control. in the case of blockchain protocols, the mathematical and economic reasoning behind the safety of the consensus often relies crucially on the uncoordinated choice model, or the assumption that the game consists of many small actors that make decisions independently. the meaning of decentralization, vitalik buterin, co-founder of ethereum the security of your permissionless system depends upon the assumption of uncoordinated choice, the idea that each voter acts independently upon its own view of the system's state. if anyone can take part, your system is vulnerable to sybil attacks, in which an attacker creates many apparently independent voters who are actually under his sole control. if creating and maintaining a voter is free, anyone can win any vote they choose simply by creating enough sybil voters. from a computer security perspective, the key thing to note ... is that the security of the blockchain is linear in the amount of expenditure on mining power, ... in contrast, in many other contexts investments in computer security yield convex returns (e.g., traditional uses of cryptography) ... analogously to how a lock on a door increases the security of a house by more than the cost of the lock. the economic limits of bitcoin and the blockchain, eric budish, booth school, university of chicago so creating and maintaining a voter has to be expensive. permissionless systems can defend against sybil attacks by requiring a vote to be accompanied by a proof of the expenditure of some resource. this is where proof-of-work comes in; a concept originated by cynthia dwork and moni naor in . to vote in a proof-of-work blockchain such as bitcoin's or ethereum's requires computing very many otherwise useless hashes. the idea is that the good voters will spend more, compute more useless hashes, than the bad voters. the blockchain trilemma much of the innovation in blockchain technology has been aimed at wresting power from centralised authorities or monopolies. unfortunately, the blockchain community’s utopian vision of a decentralised world is not without substantial costs. in recent research, we point out a ‘blockchain trilemma’ – it is impossible for any ledger to fully satisfy the three properties shown in [the diagram] simultaneously ... in particular, decentralisation has three main costs: waste of resources, scalability problems, and network externality inefficiencies. the economics of blockchains, markus k brunnermeier & joseph abadi, princeton brunnermeir and abadi's blockchain trilemma shows that a blockchain has to choose at most two of the following three attributes: correctness decentralization cost-efficiency obviously, your system needs the first two, so the third has to go. running a voter (mining in the jargon) in your system has to be expensive if the system is to be secure. no-one will do it unless they are rewarded. they can't be rewarded in "fiat currency", because that would need some central mechanism for paying them. so the reward has to come in the form of coins generated by the system itself, a cryptocurrency. to scale, permissionless systems need to be based on a cryptocurrency; the system's state transitions will need to include cryptocurrency transactions in addition to records of files, reviews, archival content, whatever. your system needs names for the parties to these transactions. there is no central authority handing out names, so the parties need to name themselves. as proposed by david chaum in they can do so by generating a public-private key pair, and using the public key as the name for the source or sink of each transaction. we created a small bitcoin wallet, placed it on images in our honeyfarm, and set up monitoring routines to check for theft. two months later our monitor program triggered when someone stole our coins. this was not because our bitcoin was stolen from a honeypot, rather the graduate student who created the wallet maintained a copy and his account was compromised. if security experts can't safely keep cryptocurrencies on an internet-connected computer, nobody can. if bitcoin is the "internet of money," what does it say that it cannot be safely stored on an internet connected computer? risks of cryptocurrencies, nicholas weaver, u.c. berkeley in practice this is implemented in wallet software, which stores one or more key pairs for use in transactions. the public half of the pair is a pseudonym. unmasking the person behind the pseudonym turns out to be fairly easy in practice. the security of the system depends upon the user and the software keeping the private key secret. this can be difficult, as nicholas weaver's computer security group at berkeley discovered when their wallet was compromised and their bitcoins were stolen. -year bitcoin "price" history the capital and operational costs of running a miner include buying hardware, power, network bandwidth, staff time, etc. bitcoin's volatile "price", high transaction fees, low transaction throughput, and large proportion of failed transactions mean that almost no legal merchants accept payment in bitcoin or other cryptocurrency. thus one essential part of your system is one or more exchanges, at which the miners can sell their cryptocurrency rewards for the "fiat currency" they need to pay their bills. who is on the other side of those trades? the answer has to be speculators, betting that the "price" of the cryptocurrency will increase. thus a second essential part of your system is a general belief in the inevitable rise in "price" of the coins by which the miners are rewarded. if miners believe that the "price" will go down, they will sell their rewards immediately, a self-fulfilling prophesy. permissionless blockchains require an inflow of speculative funds at an average rate greater than the current rate of mining rewards if the "price" is not to collapse. to maintain bitcoin's price at $ k requires an inflow of $ k/hour. ether miners / / can we really say that the uncoordinated choice model is realistic when % of the bitcoin network’s mining power is well-coordinated enough to show up together at the same conference? the meaning of decentralization, vitalik buterin in order to spend enough to be secure, say $ k/hour, you need a lot of miners. it turns out that a third essential part of your system is a small number of “mining pools”. bitcoin has the equivalent of around m antminer s s, and a block time of minutes. each s , costing maybe $ k, can expect a reward about once every years. it will be obsolete in about a year, so only in will ever earn anything. to smooth out their income, miners join pools, contributing their mining power and receiving the corresponding fraction of the rewards earned by the pool. these pools have strong economies of scale, so successful cryptocurrencies end up with a majority of their mining power in - pools. each of the big pools can expect a reward every hour or so. these blockchains aren’t decentralized, but centralized around a few large pools. at multiple times in one mining pool controlled more than % of the bitcoin mining power. at almost all times since - pools have controlled the majority of the bitcoin mining power. currently two of them are controlled by bitmain, the dominant supplier of mining asics. with the advent of mining-as-a-service, % attacks have become endemic among the smaller alt-coins. the security of a blockchain depends upon the assumption that these few pools are not conspiring together outside the blockchain; an assumption that is impossible to verify in the real world (and by murphy's law is therefore false). similar off-chain collusion among cryptocurrency traders allows for extremely profitable pump-and-dump schemes. since then there have been other catastrophic bugs in these smart contracts, the biggest one in the parity ethereum wallet software ... the first bug enabled the mass theft from "multisignature" wallets, which supposedly required multiple independent cryptographic signatures on transfers as a way to prevent theft. fortunately, that bug caused limited damage because a good thief stole most of the money and then returned it to the victims. yet, the good news was limited as a subsequent bug rendered all of the new multisignature wallets permanently inaccessible, effectively destroying some $ m in notional value. this buggy code was largely written by gavin wood, the creator of the solidity programming language and one of the founders of ethereum. again, we have a situation where even an expert's efforts fell short. risks of cryptocurrencies, nicholas weaver, u.c. berkeley in practice the security of a blockchain depends not merely on the security of the protocol itself, but on the security of the core software and the wallets and exchanges used to store and trade its cryptocurrency. this ancillary software has bugs, such as the recently revealed major vulnerability in bitcoin core, the parity wallet fiasco, and the routine heists using vulnerabilities in exchange software. recent game-theoretic analysis suggests that there are strong economic limits to the security of cryptocurrency-based blockchains. for safety, the total value of transactions in a block needs to be less than the value of the block reward. your system needs an append-only data structure to which records of the transactions, files, reviews, archival content, whatever are appended. it would be bad if the miners could vote to re-write history, undoing these records. in the jargon, the system needs to be immutable (another misnomer). merkle tree (source) the necessary data structure for this purpose was published by stuart haber and w. scott stornetta in . a company using their technique has been providing a centralized service of securely time-stamping documents for nearly a quarter of a century. it is a form of merkle or hash tree, published by ralph merkle in . for blockchains it is a linear chain to which fixed-size blocks are added at regular intervals. each block contains the hash of its predecessor; a chain of blocks. the blockchain is mutable, it is just rather hard to mutate it without being detected, because of the merkle tree’s hashes, and easy to recover, because there are lots of copies keeping stuff safe. but this is a double-edged sword. immutability makes systems incompatible with the gdpr, and immutable systems to which anyone can post information will be suppressed by governments. btc transaction fees cryptokitties’ popularity exploded in early december and had the ethereum network gasping for air. ... ethereum has historically made bold claims that it is able to handle unlimited decentralized applications ... the crypto-kittie app has shown itself to have the power to place all network processing into congestion. ... at its peak [cryptokitties] likely only had about , daily users. neopets, a game to which cryptokitties is often compared, once had as many as million users. how crypto-kitties disrupted the ethereum network, open trading network a user of your system wanting to perform a transaction, store a file, record a review, whatever, needs to persuade miners to include their transaction in a block. miners are coin-operated; you need to pay them to do so. how much do you need to pay them? that question reveals another economic problem, fixed supply and variable demand, which equals variable "price". each block is in effect a blind auction among the pending transactions. so lets talk about cryptokitties, a game that bought the ethereum blockchain to its knees despite the bold claims that it could handle unlimited decentralized applications. how many users did it take to cripple the network? it was far fewer than non-blockchain apps can handle with ease; cryptokitties peaked at about k users. neopets, a similar centralized game, peaked at about , times as many. cryptokitties average "price" per transaction spiked % between november and december as the game got popular, a major reason why it stopped being popular. the same phenomenon happened during bitcoin's price spike around the same time. cryptocurrency transactions are affordable only if no-one wants to transact; when everyone does they immediately become un-affordable. nakamoto's bitcoin blockchain was designed only to support recording transactions. it can be abused for other purposes, such as storing illegal content. but it is likely that you need additional functionality, which is where ethereum's "smart contracts" come in. these are fully functional programs, written in a javascript-like language, embedded in ethereum's blockchain. they are mainly used to implement ponzi schemes, but they can also be used to implement initial coin offerings, games such as cryptokitties, and gambling parlors. further, in on-chain vote buying and the rise of dark daos philip daian and co-authors show that "smart contracts" also provide for untraceable on-chain collusion in which the parties are mutually pseudonymous. ico returns the first big smart contract, the dao or decentralized autonomous organization, sought to create a democratic mutual fund where investors could invest their ethereum and then vote on possible investments. approximately % of all ethereum ended up in the dao before someone discovered a reentrancy bug that enabled the attacker to effectively steal all the ethereum. the only reason this bug and theft did not result in global losses is that ethereum developers released a new version of the system that effectively undid the theft by altering the supposedly immutable blockchain. risks of cryptocurrencies, nicholas weaver, u.c. berkeley "smart contracts" are programs, and programs have bugs. some of the bugs are exploitable vulnerabilities. research has shown that the rate at which vulnerabilities in programs are discovered increases with the age of the program. the problems caused by making vulnerable software immutable were revealed by the first major "smart contract". the decentralized autonomous organization (the dao) was released on th april , but on th may dino mark, vlad zamfir, and emin gün sirer posted a call for a temporary moratorium on the dao, pointing out some of its vulnerabilities; it was ignored. three weeks later, when the dao contained about % of all the ether in circulation, a combination of these vulnerabilities was used to steal its contents. the loot was restored by a "hard fork", the blockchain's version of mutability. since then it has become the norm for "smart contract" authors to make them "upgradeable", so that bugs can be fixed. "upgradeable" is another way of saying "immutable in name only". permissionless systems trust: the core developers of the blockchain software not to write bugs. the developers of your wallet software not to write bugs. the developers of the exchanges not to write bugs. the operators of the exchanges not to manipulate the markets or to commit fraud. the developers of your upgradeable "smart contracts" not to write bugs. the owners of the smart contracts to keep their secret key secret. the owners of the upgradeable smart contracts to avoid losing their secret key. the owners and operators of the dominant mining pools not to collude. the speculators to provide the funds needed to keep the “price” going up. users' ability to keep their secret key secret. users’ ability to avoid losing their secret key. other users not to transact when you want to. so, this is the list of people your permissionless system has to trust if it is going to work as advertised over the long term. you started out to build a trustless, decentralized system but you have ended up with: a trustless system that trusts a lot of people you have every reason not to trust. a decentralized system that is centralized around a few large mining pools that you have no way of knowing aren’t conspiring together. an immutable system that either has bugs you cannot fix, or is not immutable a system whose security depends on it being expensive to run, and which is thus dependent upon a continuing inflow of funds from speculators. a system whose coins are convertible into large amounts of "fiat currency" via irreversible pseudonymous transactions, which is thus an irresistible target for crime. if the “price” keeps going up, the temptation for your trust to be violated is considerable. if the "price" starts going down, the temptation to cheat to recover losses is even greater. maybe it is time for a re-think. suppose you give up on the idea that anyone can take part and accept that you have to trust a central authority to decide who can and who can’t vote. you will have a permissioned system. the first thing that happens is that it is no longer possible to mount a sybil attack, so there is no reason running a node need be expensive. you can use bft to establish consensus, as ibm’s hyperledger, the canonical permissioned blockchain system does. you need many fewer nodes in the network, and running a node just got way cheaper. overall, the aggregated cost of the system got orders of magnitude cheaper. now there is a central authority it can collect “fiat currency” for network services and use it to pay the nodes. no need for cryptocurrency, exchanges, pools, speculators, or wallets, so much less temptation for bad behavior. permissioned systems trust: the central authority. the software developers. the owners and operators of the nodes. the secrecy of a few private keys. this is now the list of entities you trust. trusting a central authority to determine the voter roll has eliminated the need to trust a whole lot of other entities. the permissioned system is more trustless and, since there is no need for pools, the network is more decentralized despite having fewer nodes. faults replicas a byzantine quorum system of size could achieve better decentralization than proof-of-work mining at a much lower resource cost. decentralization in bitcoin and ethereum networks, adem efe gencer soumya basu, ittay eyal, robbert van renesse and emin gün sirer how many nodes does your permissioned blockchain need? the rule for bft is that f + nodes can survive f simultaneous failures. that's an awful lot fewer than you need for a permissionless proof-of-work blockchain. what you get from bft is a system that, unless it encounters more than f simultaneous failures, remains available and operating normally. the problem with bft is that if it encounters more than f simultaneous failures, the state of the system is irrecoverable. if you want a system that can be relied upon for the long term you need a way to recover from disaster. successful permissionless blockchains have lots of copies keeping stuff safe, so recovering from a disaster that doesn't affect all of them is manageable. source so in addition to implementing bft you need to back up the state of the system each block time, ideally to write-once media so that the attacker can't change it. but if you're going to have an immutable backup of the system's state, and you don't need continuous uptime, you can rely on the backup to recover from failures. in that case you can get away with, say, replicas of the blockchain in conventional databases, saving even more money. i've shown that, whatever consensus mechanism they use, permissionless blockchains are not sustainable for very fundamental economic reasons. these include the need for speculative inflows and mining pools, security linear in cost, economies of scale, and fixed supply vs. variable demand. proof-of-work blockchains are also environmentally unsustainable. the top cryptocurrencies are estimated to use as much energy as the netherlands. this isn't to take away from nakamoto's ingenuity; proof-of-work is the only consensus system shown to work well for permissionless blockchains. the consensus mechanism works, but energy consumption and emergent behaviors at higher levels of the system make it unsustainable. additional material it can be very hard to find reliable sources about cryptocurrencies because almost all cryptocurrency journalism is bought and paid for. when cryptocurrency issuers want positive coverage for their virtual coins, they buy it. self-proclaimed social media personalities charge thousands of dollars for video reviews. research houses accept payments in the cryptocurrencies they are analyzing. rating “experts” will grade anything positively, for a price. all this is common, according to more than two dozen people in the cryptocurrency market and documents reviewed by reuters. ... “the main reason why so many inexperienced individuals invest in bad crypto projects is because they listen to advice from a so-called expert,” said larry cermak, head of analysis at cryptocurrency research and news website the block. cermak said he does not own any cryptocurrencies and has never promoted any. “they believe they can take this advice at face value even though it is often fraudulent, intentionally misleading or conflicted.” special report: little known to many investors, cryptocurrency reviews are for sale anna irrera & elizabeth dilts, reuters see also: crypto-shills, jemima kelly a recent example: the boxer floyd mayweather and the music producer dj khaled have been fined for unlawfully touting cryptocurrencies. the two have agreed to pay a combined $ , in fines and penalties, the securities and exchange commission (sec) said in a statement on thursday. they neither admitted nor denied the regulator’s charges. according to the sec, mayweather and khaled failed to disclose payments from three initial coin offerings (icos), in which new currencies are sold to investors. floyd mayweather and dj khaled fined over cryptocurrency promotion dominic rushe, the guardian some idea of the cryptocurrency milieu can be gained from laurie penny's four days trapped at sea with crypto’s nouveau riche. here's a taste: the women on this boat are polished and perfect; the men, by contrast, seem strangely cured—not like medicine, but like meat. they are almost all white, between the ages of and , and are trying very hard to have the good time they paid thousands for, while remaining professional in a scene where many thought leaders have murky pasts, a tendency to talk like youtube conspiracy preachers, and/or the habit of appearing in magazines naked and covered in strawberries. that last is -year-old john mcafee, who got rich with the anti-virus software mcafee security before jumping into cryptocurrencies. he is the man most of the acolytes here are keenest to get their picture taken with and is constantly surrounded by private security who do their best to aesthetically out-thug every armani-suited russian skinhead on deck. occasionally he commandeers the grand piano in the guest lounge, and the young live-streamers clamor for the best shot. john mcafee has never been convicted of rape and murder, but—crucially—not in the same way that you or i have never been convicted of rape or murder. on th december bitcoin's "price" was around $ , . bitcoin now at $ , . . those of you in the old school who believe this is a bubble simply have not understood the new mathematics of the blockchain, or you did not cared enough to try. bubbles are mathematically impossible in this new paradigm. so are corrections and all else tweet from john mcafee, th december similarly, most of what your read about blockchain technology is people hyping their vaporware. a "trio of monitoring, evaluation, research, and learning, (merl) practitioners in international development" started out enthusiastic about the potential of blockchain technology, so they did some research: we documented blockchain use-cases through internet searches, most of which were described with glowing claims like “operational costs… reduced up to %,” or with the assurance of “accurate and secure data capture and storage.” we found a proliferation of press releases, white papers, and persuasively written articles. however, we found no documentation or evidence of the results blockchain was purported to have achieved in these claims. we also did not find lessons learned or practical insights, as are available for other technologies in development. we fared no better when we reached out directly to several blockchain firms, via email, phone, and in person. not one was willing to share data on program results, merl processes, or adaptive management for potential scale-up. despite all the hype about how blockchain will bring unheralded transparency to processes and operations in low-trust environments, the industry is itself opaque. from this, we determined the lack of evidence supporting value claims of blockchain in the international development space is a critical gap for potential adopters. blockchain for international development: using a learning agenda to address knowledge gaps john burg, christine murphy, & jean paul pétraud i highly recommend david gerard's book attack of the -foot blockchain, and his blog. others to follow include arvind narayanan and his group at princeton, nicholas weaver at berkeley, emin gün sirer and the team at cornell who blog at hacking, distributed, and jemima kelly and the ft alphaville team. every time the word "price" appears here, it has quotes around it. the reason is that there is a great deal of evidence that the exchanges, operating an unregulated market, are massively manipulating the exchange rate between cryptocurrencies and the us dollar. the primary mechanism is the issuance of billions of dollars of tether, a cryptocurrency that is claimed to be backed one-for-one by actual us dollars in a bank account, and thus whose value should be stable. there has never been an audit to confirm this claim, and the trading patterns in tether are highly suspicious. tether, and its parent exchange bitfinex, are the subject of investigations by the cftc and federal prosecutors: as bitcoin plunges, the u.s. justice department is investigating whether last year’s epic rally was fueled in part by manipulation, with traders driving it up with tether -- a popular but controversial digital token. while federal prosecutors opened a broad criminal probe into cryptocurrencies months ago, they’ve recently homed in on suspicions that a tangled web involving bitcoin, tether and crypto exchange bitfinex might have been used to illegally move prices, said three people familiar with the matter. bitcoin-rigging criminal probe focused on tie to tether matt robinson and tom schoenberg, bloomberg social capital has a series explaining tether and the "stablecoin" scam: pumps, spoofs and boiler rooms tether, part one: the stablecoin dream tether, part two: pokedex tether, part three: crypto island tether's problems are in addition to the problems caused by exchanges' habit of losing their customers' coins (already in it was estimated that . % of all bitcoin in circulation had been stolen), front-running their trades, money laundering, "painting the tape", preventing customers withdrawing their funds, faking trading volume, and so on. john lewis is an economist at the bank of england. his the seven deadly paradoxes of cryptocurrency provides a skeptical view of the economics of cryptocurrencies that nicely complements my more technology-centric view. my comments on his post are here. remember that a permissionless blockchain requires a cryptocurrency; if the economics don't work neither does the blockchain. you can find my writings about blockchain over the past five years here. in particular: blockchain for verifying integrity of preserved content. "immutability" blockchain for storage mining asics peer review problem of immutable, publicly writable data why decentralize? economies of scale more detail on the bugs in the dao: the dao was designed as a series of contracts that would raise funds for ethereum-based projects and disperse them based on the votes of members. an initial token offering was conducted, exchanging ethers for "dao tokens" that would allow stakeholders to vote on proposals, including ones to grant funding to a particular project. that token offering raised more than $ m worth of ether at then-current prices, distributing over bn dao tokens. [in may ], however, news broke that a flaw in the dao's smart contract had been exploited, allowing the removal of more than m ethers. subsequent exploitations allowed for more funds to be removed, which ultimately triggered a 'white hat' effort by token-holders to secure the remaining funds. that, in turn, triggered reprisals from others seeking to exploit the same flaw. an effort to blacklist certain addresses tied to the dao attackers was also stymied mid-rollout after researchers identified a security vulnerability, thus forcing the hard fork option. the hard fork: what's about to happen to ethereum and the dao, michael del castillo, coindesk the dao heist isn't an anomaly; here's a recent example (click through to the medium post): ico token oyster prl was exit-scammed by its founder, “bruno blocks” — who nobody has ever met — who took million tokens via a deliberately-maintained back door in the smart contract code. how does this keep happening? fortunately, the developers are on the case … by printing million new tokens for themselves. david gerard exit scams are rife in the ico world. here is a recent example: blockchain company pure bit has seemingly walked off with $ . million worth of investors’ money after raising , ethereum in an ico. transaction history shows that hours after moving all raised funds out of its wallet, the company proceeded to take down its website. it now returns a blank page. ... this is the latest in a string of exit scams that took place in the blockchain space in . indeed, reports suggested exit scammers have thieved more than $ million worth of cryptocurrency over the last two years alone. subsequent investigations hint the actual sum of stolen cryptocurrency could be even higher. south korean cryptocurrency startup reportedly pulls a $ . m exit scam the next web more detail on the lack of decentralization in practice: in bitcoin, the weekly mining power of a single entity has never exceeded % of the overall power. in contrast, the top ethereum miner has never had less than % of the mining power. moreover, the top four bitcoin miners have more than % of the average mining power. on average, % of the weekly power was shared by only three ethereum miners. these observations suggest a slightly more centralized mining process in ethereum. although miners do change ranks over the observation period, each spot is only contested by a few miners. in particular, only two bitcoin and three ethereum miners ever held the top rank. the same mining pool has been at the top rank for % of the time in bitcoin and % of the time in ethereum. over % of the mining power has exclusively been shared by eight miners in bitcoin and five miners in ethereum throughout the observed period. even % of the mining power seems to be controlled by only miners in bitcoin and only miners in ethereum. decentralization in bitcoin and ethereum networks, adem efe gencer, soumya basu, ittay eyal, robbert van renesse and emin gün sirer more on the lack of decentralization highlighted by balaji s. srinivasan and leland lee in quantifying decentralization, with their use of the "nakamoto coefficient": "ethereum’s smart contract ecosystem has a considerable lack of diversity. most contracts reuse code extensively, and there are few creators compared to the number of overall contracts. ... the high levels of code reuse represent a potential threat to the security and reliability. ethereum has been subject to high-profile bugs that have led to hard forks in the blockchain (also here) or resulted in over $ million worth of ether being frozen; like with dns’s use of multiple implementations, having multiple implementations of core contract functionality would introduce greater defense-in-depth to ethereum." analyzing etheruem's contract topology, lucianna kiffer, dave levin and alan mislove more detail on pump-and-dump (p&d) schemes: p&ds have dramatic short-term impacts on the prices and volumes of most of the pumped tokens. in the first seconds after the start of a p&d, the price increases by % on average, trading volume increases times, and the average -second absolute return reaches %. a quick reversal begins seconds after the start of the p&d. after an hour, most of the initial effects disappear. ... prices of pumped tokens begin rising five minutes before a p&d starts. the price run-up is around %, together with an abnormally high volume. these results are not surprising, as pump group organizers can buy the pumped tokens in advance. when we read related messages posted on social media, we find that some pump group organizers offer premium memberships to allow some investors to receive pump signals before others do. the investors who buy in advance realize great returns. calculations suggest that an average return can be as high as %, even after considering the time it may take to unwind positions. for an average p&d, investors make one bitcoin (about $ , ) in profit, approximately one-third of a token’s daily trading volume. the trading volume during the minutes before the pump is % of the total volume during the minutes after the pump. this implies that an average trade in the first minutes after a pump has a % chance of trading against these insiders and on average they lose more than % ( %* %). cryptocurrency pump-and-dump schemes tao li, donghwa shin and baolian wang a summary of the bad news about vote-buying in blockchains: the existence of trust-minimizing vote buying and dark dao primitives imply that users of all on-chain votes are vulnerable to shackling, manipulation, and control by plutocrats and coercive forces. this directly implies that all on-chain voting schemes where users can generate their own keys outside of a trusted environment inherently degrade to plutocracy, ... our schemes can also be repurposed to attack proof of stake or proof of work blockchains profitably, posing severe security implications for all blockchains. on-chain vote buying and the rise of dark daos philip daian, tyler kell, ian miers, and ari juels source here is a typical day on the bitcoin blockchain. it is averaging transactions/sec, and has a queue of an hour's worth of them waiting to be confirmed. back in testing showed: visanet handles an average of million transactions every day and is capable of handling more than , transactions per second. eight years ago that was about , times as many transactions/sec on average, using much less electricity than austria to do it. source s&p companies are slowly figuring out that there is no there there in blockchains and cryptocurrencies, and they're not the only ones: still new to nyc, but i met this really cool girl. energy sector analyst or some such. four dates in, she uncovers my love for bitcoin. completely ghosted. zack voell posted by david. at : pm labels: bitcoin, digital preservation, scholarly communication comments: david. said... cullen roche clearly debunks this founding narrative of the cryptocurrency cult: “governments are big bad terrible entities that will print money and ruin society and we can create a better form of decentralized money that won’t inflate our living standards away.” december , at : am david. said... just to illustrate how far from reality the hype around blockchain and cryptocurrency is, three successive stories at /. this morning link to: ) mike orcutt's ethereum thinks it can change the world. it’s running out of time to prove it: "the reason devcon feels so upbeat despite these storm clouds is that the people building ethereum have something bigger in mind—something world-changing, in fact. yet to achieve its goal, this ragtag community needs to crack a problem as complicated as any of the toe-curling technical challenges it faces: how to govern itself. it must find a way to organize a scattered global network of contributors and stakeholders without sacrificing “decentralization”—the principle, which any cryptocurrency community strives for, that no one entity or group should be in control." good luck with that! at least orcutt shows some level of skepticism. ) paul sawer's linkedin: ‘blockchain developer’ is the fastest-growing u.s. job: "using data gleaned from the linkedin economic graph, which serves as a “digital representation of the global economy” by analyzing the skills and job openings from across million members and million companies, linkedin found that “blockchain developers” has grown -fold in the past four years. in this case, “emerging jobs” refers to the growth of specific job titles on linkedin profiles in the period between and ." ) olga kharif's ranks of crypto users swelled in even as bitcoin tumbled: "the number of verified users of cryptocurrencies almost doubled in the first three quarters of the year even as the market bellwether bitcoin tumbled almost percent, according to a study from the cambridge centre for alternative finance. users climbed from million to million this year." december , at : am david. said... the insanity continues: "as president donald trump threatened to allow a government shutdown if congress did not provide funding for his proposed wall along the mexican border, a republican congressman from ohio offered up alternative routes to getting the wall built: through internet crowdfunding or through an initial coin offering. ... rep. davidson told npr's steve inskeep that the donations could come from anyone and be gathered in a number of ways."you could do it with this sort of, like, crowdfunding site," davidson explained. "or you could do it with blockchain—you could have wall coins." december , at : pm david. said... barry ritholtz takes prophets to the woodshed: "fundstrat’s tom lee’s forecast for $ , bitcoin was reduced last month to $ , by year-end. (the cryptocurrency recently traded at about $ , .) as foolish as that sounds, it was modest compared to the rest of the asylum. michael novogratz forecast that “$ , was possible by the end of .” kay van-petersen of saxo bank predicted bitcoin would rise to $ , to $ , by the end of this year. john mcafee, the eccentric tech entrepreneur, has called for $ million bitcoin by . analogizing crypto to the internet, tim draper doubles mcafee, coming in at $ million. all of these are notable not just for being wrong, but for their sheer recklessness." december , at : am david. said... jemima kelly reports that one of the most-touted "stablecoin" companies ran headlong into the regulations: "remember basis, the “stablecoin” backed by . . . stability? back in june we broke the news that stanford economist, and author of the “taylor rule”, john taylor had joined the project. well, it's shutting up shop. basis had a big idea — it wanted to marry traditional monetary theory with cryptonomics, to deliver “a stable cryptocurrency with an algorithmic central bank”. ... the ambitious proposition didn't really convince us, but it did convince many. it had some big financial backers: silicon valley vc heavyweights andreessen horowitz and bain capital ventures invested, alongside billionaire hedge fund manager stanley druckenmiller, and kevin warsh, a former governor of the federal reserve. a total of $ m was raised (in late , mid-cryptomania, we should point out). that money is now being given back to investors, however, and basis is closing down. the reason? regulators appear to have told them that “bond tokens” and “share tokens” are a bit like “bonds” and “shares” and need to be treated as such." december , at : pm david. said... businessweek's cover is rhymes with bitcoin. after a whole year of "price" collapse, piling on is fine, although it would have been better before the massive pump-and-dump. but it is a shame that, for example, lionel laurent's the messy political story of bitcoin gets so much wrong, including the credulous belief that bitcoin actually delivers the promised anonymity and decentralization. nouriel roubini's the big blockchain lie is much better: "yet far from ushering in a utopia, blockchain has given rise to a familiar form of economic hell. a few self-serving white men (there are hardly any women or minorities in the blockchain universe) pretending to be messiahs for the world’s impoverished, marginalized, and unbanked masses claim to have created billions of dollars of wealth out of nothing. but one need only consider the massive centralization of power among cryptocurrency “miners,” exchanges, developers, and wealth holders to see that blockchain is not about decentralization and democracy; it is about greed." december , at : am david. said... in march tim swanson wrote what is the difference between hyperledger and hyperledger?, a fascinating look at the backstory of hyperledger. december , at : am david. said... jeff wise's ‘my power to demolish is ten times greater than my power to promote’ how john mcafee became the spokesman for the crypto bubble. should be required reading for anyone still trapped in one of the cryptocurrency cults: "though mcafee had not been an early adopter of crypto, by chance he had happened to craft the perfect persona for touting it. he had both a credible claim to technical expertise and enough of a moral taint to come across as savvy. just as candidate trump claimed that he alone could clean up the swamp because he had decades’ experience buying off politicians, mcafee’s self-advertised misbehavior made him plausible as a guide to a marketplace awash with con artists." december , at : pm david. said... combining blockchain and emulation, its blockchain for the apple ][! december , at : pm david. said... more evidence that cryptocurrency exchanges are manipulating the market. from the blockchain transparency institute's december report: "we have calculated the true volume of the cmc top btc trading pairs. most of these pairs actual volume is under % of their reported volume on cmc. we noted only out of the top pairs not to be grossly wash trading their volume, binance and bitfinex." december , at : am david. said... the economics of cryptocurrency pump and dump schemes by jt hamrick et al is anoterh look at the epidemic of cryptocurrency pump-and-dump schemes: "we identified , different pump signals advertised on telegram and another , different pump signals advertised on discord during a six-month period in . the schemes promoted more than cryptocurrencies. this comprehensive data provides the first measure of the scope of pump and dump schemes across cryptocurrencies and suggest that this phenomenon is widespread and often quite profitable. this should raise concerns among regulators. we then examine which factors that affect the “success” of the pump, as measured by the percentage increase in price near the pump signal. we find that the coin’s rank (market capitalization/volume) is the most important factor in determining the profitability of the pump: pumping obscure coins (with low volume) is much more profitable than pumping the dominant coins in the ecosystem." but note, as david gerard writes: "how hard are the bitcoin price pumpers pushing the price? december was bitcoin’s single highest trading volume day in its entire history." december , at : pm david. said... nicholas weaver makes the same fundamental point as i did: "the whole point of proof-of-*whatever* is really not about "security" but preventing sybils: someone from spinning up a gazillion validators and voting themselves all the money. pow, pos, they all fail to do this well." cheap blockchain = insecure blockchain. december , at : am david. said... nancy nakamoto's the princess bride and the mystery of the tether business model is an elegant explanation of why tether should be treated skeptically. january , at : pm david. said... last thursday was bitcoin's tenth anniversary, celebrated appropriately by jemima kelly in happy birthday bitcoin. your gift: a log chart in the times with a study in the different impact of log-linear vs. linear charts. january , at : am david. said... blockchains are immutable, right? not so much. dan goodin reports on the latest double-spending" attack: "attackers have stolen almost $ , worth of the ethereum classic digital currency by carrying out a compute-intensive hack that rewrote its blockchain, officials with coinbase, one of the leading crypto currency exchanges, said on monday." according to coinmarketcap.com, etc is the th biggest cryptocurrency, with a "market cap" of $ , , . but that's not enough to keep it secure. only a few of the biggest altcoins have enough mining power relative to their "price" to deter % attacks. january , at : pm david. said... in "proof-of-work" proves not to work from , ben laurie and richard clayton analyzed cynthia dwork and moni naor's proposal to use proof-of-work to mitigate spam. from their abstract: "we attempt to determine how difficult that puzzle should be so as to be effective in preventing spam. we analyse this both from an economic perspective, "how can we stop it being cost-effective to send spam", and from a security perspective, "spammers can access insecure end-user machines and will steal processing cycles to solve puzzles". both analyses lead to similar values of puzzle difficulty. unfortunately, real-world data from a large isp shows that these difficulty levels would mean that significant numbers of senders of legitimate email would be unable to continue their current levels of activity. we conclude that proof-of-work will not be a solution to the problem of spam." hat tip to commentor clive robinson at schneier on security. january , at : am david. said... tracing stolen bitcoin by ross anderson sparked an interesting discussion of the many legal frameworks about the proceeds of crime. january , at : am david. said... alex tarrabok's bitcoin is less secure than most people think is a detailed discussion of eric budish's the economic limits of bitcoin and the blockchain. it is worth reading. january , at : am david. said... the % attacks on ethereum classic continue, at a cost of $ /hr. catalin cimpanu reports: "coinbase also updated its original report with details on another double-spend attacks, bringing the total of stolen funds to , etc (~$ . million)." january , at : am david. said... in why the ethereum classic hack is a bad omen for the blockchain, russell brandom explains the rash of % attacks and quotes nicholas weaver: "as weaver puts it, it’s “a nice illustration of how proof-of-waste schemes cannot be both efficient and secure.” the more it costs to mine a block, the more expensive it is to outspend the honest miners for long to reverse a transaction. electricity prices vary from miner to miner, but weaver estimates that the bitcoin network currently runs through about $ , in electricity each hour, while the smaller ethereum network runs at roughly $ , per hour. for weaver, any coin much smaller than that is at risk of a percent attack. ethereum classic clocks in at roughly $ , per hour." january , at : am david. said... izabella kaminska is back, contrasting mckinsey's pre- and post-price-crash reports on blockchain. she bolds these quotes from the latest: "the bottom line is that despite billions of dollars of investment, and nearly as many headlines, evidence for a practical scalable use for blockchain is thin on the ground." and: "the fact was that billions of dollars had been sunk but hardly any use cases made technological, commercial, and strategic sense or could be delivered at scale." january , at : am david. said... joseph bonneau's hostile blockchain takeovers complements eric budish's work with a detailed analysis of "goldfinger attacks", those aimed at discrediting a currency that the attacker is shorting. january , at : am david. said... more on the mckinsey report in blockchain’s groundbreaking, world-shaking, life-changing technology revolution has been cancelled by terence corcoran. january , at : pm david. said... matt levine writes about how exchanges are the roach motels of cryptocurrencies: "i feel like i am constantly reading about transaction limits for withdrawals of money from the crypto ecosystem, while i much less frequently read about transaction limits for putting money in. similarly anti-money-laundering and know-your-customer procedures seem to constrain people from taking money out of cryptocurrencies more than they constrain people trying to put money in. it is an odd one-way ratchet." january , at : pm david. said... yet another theoretical attribute of blockchains, censorship resistance, turns out to be problematic in practice, as mit's the download reports in china will now officially try to extend its great firewall to blockchains: "the cyberspace administration of china (cac) will require any “entities or nodes” that provide “blockchain information services” to collect users’ real names and national id or telephone numbers, and allow government officials to access that data. it will ban companies from using blockchain technology to “produce, duplicate, publish, or disseminate” any content that chinese law prohibits." january , at : am david. said... louis denicola reports: "collectively, u.s. investors who have sold their bitcoin incurred realized losses of approximately $ . billion. as for those who haven’t sold yet, their unrealized losses total an approximate $ . billion." january , at : pm david. said... bitmex has a report on tracking us$ billion of tokens ico makers allocated to themselves: "this report is based on tokens where the team controlled holding’s were worth an astonishing us$ . billion on issuance (in reality liquidity was too low for this value to be realized). today this figure has fallen to around us$ billion, with the difference primarily being caused by a fall in the market value of the tokens, alongside us$ . billion of transfers away from team address clusters (possibly disposals)." january , at : pm david. said... amy castor's the curious case of tether: a complete timeline of events is an impressive compilation . january , at : pm david. said... larry cermak maintains a spreadsheet of losses from exchanges being hacked or "hacked". the running total is now $ . billion (based on "prices" at the time the loss was discovered). january , at : pm david. said... it isn't just your cryptocurrency that gets stolen from exchanges. cryptoline reports that: "according to reports coming from ccn, a hacker is selling hacked kyc data on the dark web, data which the hacker claims to have collected from some of the cryptocurrency exchanges such as poloniex, binance, bittrex and bitfinex. ... the ad put up by the hacker is said to have been up and running since july and the hacked kyc data discloses personal information ranging from drivers licence, id cards to passport data." kyc is "know your customer", part of the anti-money-laundering regulations. january , at : pm david. said... now we know why trump wants to invade venezuela. it is currently the base for the mcafee presidential campaign. ben munster's a presidential campaign in exile has interviews with some of the campaign staffers. tip of the hat to david gerard. january , at : pm david. said... munawar gul is a bitcoin enthusiast. his the fall of the blockchain hype men is a diatribe against the explosion of patents littering the blockchain space, and the way big corporations are using them to defend their permissioned blockchain implementations: "recently over at devcon , the annual developers conference for ethereum. vitalik buterin told quartz why he’s distressed about corporations patenting everything about blockchain. behind the scenes, companies like ibm, bank of america, barclays, mastercard and alibaba are in a race with one another to file patents on blockchain-based systems. even more surprising is the fact that china being so tough on cryptocurrencies is leading the race in the number of blockchain related patent applications." so another reason not to like blockchain is that it will be infested with patent trolls. january , at : pm david. said... mit's technology review retorts that just two hacker groups may have stolen $ billion in cryptocurrency: "blockchain analytics firm chainalysis spent around three months tracking funds that had been stolen in known hacks. it was able to link much of that money to two groups, which it dubbed alpha and beta. if the group’s analysis is correct, then the two groups would account for % of all publicly reported crypto-heists." even the crime is centralized! january , at : am david. said... david gerard takes cryptocurrencies and blockchains to the woodshed in the buttcoin standard: the problem with bitcoin. go read it. january , at : am david. said... dan goodin's digital exchange loses $ million as founder takes passwords to the grave describes yet another reason not to trust cryptocurrency exchanges: "a cryptocurrency exchange in canada has lost control of at least $ million of its customers’ assets following the sudden death of its founder, who was the only person known to have access the the offline wallet that stored the digital coins. british columbia-based quadrigacx is unable to access most or all of another $ million because it’s tied up in disputes with third parties." february , at : pm david. said... longhash has an interesting interview with emin gün sirer: "the protocols that we have are not very good at scaling to large numbers of participants, they have built in forces toward centralization. in proof of work currencies, economies of scale, ability to acquire cheap electricity and access to supply chains mean that there will always be a few hardware manufacturers that dominate the mining industry. we’ve seen that bitcoin mining tends toward centralization, and certain groups become more and more prominent. the only force aiding us is that these mining concerns operate in a competitive industry, and there’s high turnover. but right now, just a few players can easily launch % attacks and can censor transactions if compelled." he's a lot less skeptical than i am about proof of stake. february , at : pm david. said... i was remiss in not pointing out that at the same time in as we published the lockss protocol using decentralized consensus and proof of work, vivek vishnumurthy, sangeeth chandrakumar and emin gün sirer published karma, an actual cryptocurrency using decentralized consensus and proof of work. both teams started work about the same time in , so the ideas were in the air then. february , at : am david. said... edward robinson's crypto is over: paris fintech summit returns to disrupting banks reports: "with the top crypto assets down percent in the last months and skepticism mounting, many fintech pros concluded that the technology may not be ready for prime time, especially in an industry this heavily regulated. ... perhaps nothing drove that point home more than the face-off between gottfried leibbrandt, the chief executive officer of swift, and brad garlinghouse, the ceo of san francisco’s ripple labs inc. swift is a -year-old cooperative that directs trillions of dollars in cross-border payments between thousands of banks. garlinghouse has repeatedly vowed to leapfrog swift’s s-conceived system with a faster, cheaper blockchain-like one." so what about ripple? "nobody uses ripple labs’ tech — ripple’s “ + institutional clients” claim is a scam. “not a single one of ripple’s clients appears to be real, or an actual client.” ... remember how ripple labs claimed a “partnership” with santander — and told the press things like “we are covering % of all the fx payments the santander group does annually,” then tweeted them from the official ripple account? it turns out they’re talking about an iphone app, “santander one pay fx,” which uses ripple’s xcurrent. the app has ratings and one review — almost nobody uses it. it turns out that “covering” means the app is available in spain, uk, brazil and poland — which together contain % of santander’s customers — and does not in any way mean that those customers use it." february , at : pm david. said... josh reviews everything has an interest rates primer for cryptocurrency folks: "the market for personal loans in cryptocurrency isn’t exactly huge, so the main source of demand to borrow bitcoin comes from people who want to short it. as the demand to short bitcoin through futures grows, the futures price will trade at a wider discount to spot, and the implied bitcoin interest rate will move higher. and this is what’s been happening since mid-november, when bitcoin prices collapsed below $ and never looked back. the demand to short bitcoin in the futures market increased; so the futures started trading at a bigger discount to spot; and that increased the implied bitcoin interest rate." so he computes: "the implied interest rate for btc is about . % annualised." tip of the hat to barry ritholtz. february , at : am david. said... hyping stablecoins to compete with tether is hard: "five unaffiliated otc desks all told coindesk there was no demand for gusd among their trading networks. otc traders said it appears a high percentage of perceived gusd trading activity is concentrated on exchanges that they do not use. “the trading volume is dictated by [gemini’s] actions and has nothing to do with the market, per se,” said one anonymous otc trader who has worked with gusd. according to coinmarketcap, the exchanges with the highest gusd trading volume include oex, hotbit, bitrue and fatbtc. gemini has no direct association with these exchanges. in particular, oex and fatbtc are both ranked on the blockchain transparency institutes’ advisory list. the nonprofit estimated that over percent of those exchanges’ activity comes from automated trades, which typically involves bots rather than real users." february , at : am david. said... bruce schneier joins the chorus pointing out that there's no there there in blockchain technology: "do you need a public blockchain? the answer is almost certainly no. a blockchain probably doesn’t solve the security problems you think it solves. the security problems it solves are probably not the ones you have. (manipulating audit data is probably not your major security risk.) a false trust in blockchain can itself be a security risk. the inefficiencies, especially in scaling, are probably not worth it. i have looked at many blockchain applications, and all of them could achieve the same security properties without using a blockchain—of course, then they wouldn’t have the cool name." february , at : am david. said... i've observed before that mining a public blockchain these days necessarily involves hosting content that allows the authorities to jail you at a moment's notice in almost all jurisdictions, such as child porn. now bitcoin sv, one of the increasingly many forks of staoshi's original, has raised its transaction size limit to kb, allowing much bigger and better content to be stored immutably in its blockchain. and, naturally, as david canellis reports in bitcoinsv ‘feature’ exploited to store child abuse imagery on the blockchain, among the first content to take advantage of this feature is child porn: "a change to the bitcoin satoshi’s vision (bsv) protocol has inadvertently led to child exploitation material being posted to its blockchain, forcing apps and block explorers into actively monitoring the network for illegal content." filtering blockchains for illegal content is problematic because, as nicholas weaver points out in his talk for enigma, it opens the blockchain up to a very simple and effective attack: "in late the bitcoin network hit a capacity limit which caused fees to enter a death spiral, so if you wanted your transaction to go through you had to outbid everyone else, this is exploitable. someone could spam the network whenever its below the death spiral point, shutting it down at will. and when it is above, do nothing but laugh. keep this up until the network installs spam filters, and then the attacker starts a more interesting game: tuning spam not to get through the filters but to have the filters trigger false positives. how well would a currency work if - % of transactions are randomly blocked by spam filters? ethereum seems a particularly ripe target, with a full blockchain of over tb and working sets measured in the s of gb. what happens if an attacker adds a or two to those numbers?" february , at : am david. said... paul krugman's thread on niall ferguson joining the recapitulation of the failed basis stablecoin features his opinion: "i'm on record as saying that crypto is a mishmash of technobabble and libertarian derp. but i guess that i should add that it's also a giant draw for sufferers from dunning-kruger syndrome" february , at : pm david. said... david gerard provides another example of the reason crypto "price" is in quotes: "an upset mt. gox creditor analyses the data from the bankruptcy trustee’s sale of bitcoins. he thinks he’s demonstrated incompetent dumping by the trustee — but actually shows that a “market cap” made of million btc can be crashed by selling , btc, over months, at market prices, which suggests there is no market." february , at : am david. said... the ft's alphaville has two fun reads. camilla hodgson's anatomy of a cryptocurrency scam starts: "in november this guesting alphavillian wrote an article for the financial times about the falling price of bitcoin (what's new?). half an hour later we received a direct twitter message from an account saying it was a good time to invest in the cryptocurrency, which could become a “second source of income.” they said with an investment of $ in crypto, on trading platform crypto , we could earn $ , in five days." and elon musk continues to provide izabella kaminska with material for the billions of blistering barnacles series such as: "paper money is going away, and crypto is a far better way to transfer value than pieces of paper, that’s for sure." february , at : am david. said... mike orcutt's once hailed as unhackable, blockchains are now getting hacked is an overview of vulnerabilities including % attacks, "toward the middle of , attackers began springing % attacks on a series of relatively small, lightly traded coins including verge, monacoin, and bitcoin gold, stealing an estimated $ million in total. in the fall, hackers stole around $ , using a series of attacks on a currency called vertcoin. the hit against ethereum classic, which netted more than $ million, was the first against a top- currency." and bugs in "smart contracts": "last month, tsankov’s team at chainsecurity saved ethereum from a possible repeat of the dao catastrophe. just a day before a major planned software upgrade, the company told ethereum’s lead developers that it would have the unintended consequence of leaving some contracts on the blockchain newly vulnerable to the same kind of bug that led to the dao hack. the developers promptly postponed the upgrade and will give it another go later this month." february , at : am david. said... eric hellman's "blockchain for libraries" is snake oil is not just right about blockchain, but about snake oil too! the video of eric's talk is here. although i have a couple of quibbles: - slide says "participants can be anonymous" but means pseudonymous. and "consensus mechanism prevents double-spending.–secure- except for bugs" but ignores % attacks. it should say "consensus mechanism makes double-spending costly" but in many cases now it isn't costly enough. - slide says lockss uses byzantine fault tolerance, which is wrong. it uses an original consensus mechanism that is related to bft but is statistical in nature. february , at : pm david. said... roflmao at catalin cimpanu's cryptocurrency wallet caught sending user passwords to google's spellchecker. the sub-head says it all: "coinomi wallet bug sends users' secret passphrases to google's spellcheck api via http, in plaintext." sadly, this appears typical of the competence level of programmers in the crypto wallet space. february , at : pm david. said... it turns out that cryptocurrency exchanges are the roach motel of personal finance. cory doctorow writes: "the latest group of hacking team war criminals to find themselves reaccepted into polite society is the staff of neutrino, a startup acquired by the cryptocurrency company coinbase, to do forensic tracking of blockchain transactions. many coinbase users have concluded that they do not want to entrust their finances to a company that includes these unsavory characters, and so was born the #deletecoinbase movement to coordinate divestiture from the company. however, coinbase will only allow you to delete your account if it has a zero balance, free of "dust" (infinitesimal residues left behind from fractional cryptocurrency transactions) and users are finding it impossible to rid themselves of their dust, which coinbase insists is merely an accident and nothing to do with not wanting disgruntled users to leave." march , at : pm david. said... david gerard notes that: "coinbase buys neutrino, a startup founded by hacking team, a hacker group with an extensive track record of selling surveillance technology to oppressive governments, whose ceo regularly signed his emails with old fascist slogans. why did coinbase buy neutrino? because their previous data analysis provider was selling coinbase customer data to third parties!" march , at : pm david. said... anyone who thinks that using cryptocurrencies is a way of avoiding the attention of law enforcement should read today's news post from david gerard. he notes, in order: ) morgan rockoons pled guilty to wire fraud for the "bitcointopia" scam. ) the us has charged the founders of onecoin with wire fraud - there wasn't even one coin. ) pool/ broker has to pay a $ , fine and refund their customers bitcoin even though they were based outside the us. ) two australian exchanges got suspended for links to organized crime and drugs. ) the canada revenue agency is auditing bitcoin users. here's the -page questionnaire to give some idea of the answers users need to be prepared with. ) remember that wash trading is illegal: "crypto integrity found that up to % of reported volumes in february were artificial at some of the highest reported volume exchanges. on some less liquid trading pairs, the group estimates up to % of the volume is fake." gerard also links to kyle gibson's informative crypto thefts: a timeline of hacks, glitches, exit scams, and other lost cryptocurrency incidents. march , at : pm david. said... tim copeland's the complete story of the quadrigacx $ million scandal is unlikely to be the "complete story" but it is very informative. march , at : am david. said... david gerard comments on the big news in cryptocurrrencies, that tether has finally admitted that usdt is not actually backed one-for-one by actual usd in a bank account. bitfinex'ed takes a bow! march , at : pm david. said... bitwise asset management's detailed comments to the sec about btc/usdt trading on unregulated exchanges: "demonstrate in multiple different ways that approximately % of this volume is fake and/or non-economic in nature, and that the real market for bitcoin is significantly smaller, more orderly, and more regulated than commonly understood." hat tip to msmash at /.. march , at : am david. said... hugh son reports bank of america tech chief is skeptical on blockchain even though bofa has the most patents for it. march , at : am david. said... in the bank of hodlers [sic] (sigh), jemima kelly uses the example of the enron fiasco to explain that decentralizing utility functions like electricity and banking isn't a good idea: "the problem is, just like with the california example, every time deregulation, decentralisation and automation has taken foot in the financial world, bad things have tended to follow. most of the time what is revealed is that middlemen or rent-seekers weren't actually eliminated or disempowered, but rather regenerated into new forms. meanwhile, where algos took on the responsibility for human judgments, they were easily gamed, and introduced all sorts of new risks into the system. those bad things then justified the return of regulation and the effective reformulation of human-overseen processes that “recentralise” the industry." march , at : am david. said... flaws in bitcoin make a lasting revival unlikely from the economist is a good summary: "the cryptocurrency fiasco has exposed three deep and related problems: the extent of genuine activity is hugely exaggerated; the technology does not scale well; and fraud may be endemic." march , at : am david. said... in inside the rise and fall (and rise?) of crypto mining giant bitmain, the south china morning post's zheping huang writes a fascinating account of the rise of bitmain to dominate the bitcoin mining chip business, and its subsequent troubles, leading to a failed ipo and irresolvable differences between the co-ceos: "beijing-based bitmain technologies on tuesday called off its plan to go public in hong kong after its application lapsed after six months. the failure of what was billed as potentially the world’s largest crypto-related ipo adds to news of retrenchments. a new ceo has been appointed to replace the two main founders, who had previously shared the role." tip of the hat to david gerard. march , at : pm david. said... my talk referred to tether, an allegedly asset-backed "stablecoin". but it didn't cover the alternative form of "stablecoin", the algorithmic variety. fortunately, ben dyson at the bank of england repairs the omission. using the example of "basis", a proposed stablecoin that never actually materialized, he concludes: "whilst algorithmic stablecoins like basis manage to eliminate the need for trust in a third party, they instead end up being heavily dependent on investor belief and confidence. as long as all users believe that the coin will be stable, their behaviour ensures that it will be stable, but if some users start to lose confidence and sell, the coin risks falling into a downward spiral. what does this mean for issuers and users of algorithmic stablecoins? if these stablecoins are destined to lose their stability, then over a long-enough time frame, buyers of algorithmic stablecoins will make significant losses. but the initial sellers of these coins – the founders and share-holders – will make significant gains, since they sold something created at no cost (the coins) in exchange for fiat currencies. if the peg fails, then the end result will have been a significant transfer of wealth from the buyers to the issuers." the talk pointed out that regular cryptocurrencies such as bitcoin depend on a belief that the "price" will rise, and thus transfer wealth from later to earlier adopters. so too "stablecoins" depend on a belief that the "price" will be stable. since, as dyson shows, this is very unlikely to true, they will transfer wealth from later to earlier adopters. march , at : pm david. said... jemima kelly dives into the depths of "solution to everything™" self-parody with blockchain for brexit: a wonderfully terrible idea: "the thing is though, the more this idea gets bandied about, the more the blockchainers expose the ridiculousness of their other ideas. where an open-minded and uninformed reader might have (wrongly) bought into the idea that a blockchain can help combat the illegal fishing of tuna or ensure clinical trial integrity, they are likely to draw a line at the idea that blockchain can in any way solve a problem as intractable as brexit. or for that matter, solve the utter lack of any form of consensus in parliament." april , at : am david. said... in a 'blockchain bandit' is guessing private keys and scoring millions reports what happened after a researcher tried " " as the private key for an ethereum address: "the researchers not only found that cryptocurrency users have in the last few years stored their crypto treasure with hundreds of easily guessable private keys, but also uncovered what they call a "blockchain bandit." a single ethereum account seems to have siphoned off a fortune of , ether—worth at one point more than $ million—using those same key-guessing tricks." april , at : am david. said... the new york attorney general has sued bitfinex and has apparently discovered an $ m hole in tether's reserves. april , at : pm david. said... izabella kaminska's we all become mf global eventually, tether edition has a detailed run-down on the history behind the new york ag's suit against bitfinex. april , at : pm david. said... tether's general counsel admits that tether isn't % backed by cash after all: "tether has cash and cash equivalents (short term securities) on hand totaling approximately $ . billion, representing approximately percent of the current outstanding tethers." april , at : pm david. said... tether's missing $ m apparently vanished into crypto capital. david gerard reports that: "crypto capital were the money transmitters for troubled crypto exchange bitfinex and dubious dollar-substitute coin tether. they operated in the us through a company called global trading solutions. the founder of gts, reggie fowler, was indicted on tuesday. the us government has filed a motion to detain fowler as a flight risk. and it’s amazing. reggie fowler is an american football player turned businessman. his most recent brush with fame was when the alliance of american football, an attempt to form a new football league, collapsed after fowler withdrew funding. fowler had to withdraw funding for the aaf because the department of justice had seized his bank accounts in october — just after bitfinex had started sending money through global trading solutions." fowler was caught with $ , in counterfeit $ bills, among other details casting doubt on his financial operations. note that future comments on this topic will appear on this more recent version of the talk. may , at : pm post a comment newer post older post home subscribe to: post comments (atom) blog rules posts and comments are copyright of their respective authors who, by posting or commenting, license their work under a creative commons attribution-share alike . united states license. off-topic or unsuitable comments will be deleted. dshr dshr in anwr recent comments full comments blog archive ► ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ▼ ( ) ▼ december ( ) securing the hardware supply chain meta: impending blog slowdown securing the software supply chain software preservation network blockchain: what's not to like? irina bolychevsky on solid selective amnesia ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► july ( ) ► june ( ) ► february ( ) ► ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) lockss system has permission to collect, preserve, and serve this archival unit. simple theme. powered by blogger. none amazon elasticache for redis click here to return to amazon web services homepage contact sales support english my account create an aws account products solutions pricing documentation learn partner network aws marketplace customer enablement events explore more عربي bahasa indonesia deutsch english español français italiano português tiếng việt türkçe Ρусский ไทย 日本語 한국어 中文 (简体) 中文 (繁體) aws management console account settings billing & cost management security credentials aws personal health dashboard support center knowledge center aws support overview click here to return to amazon web services homepage products solutions pricing introduction to aws getting started documentation training and certification developer center customer success partner network aws marketplace support log into console download the mobile app amazon elasticache for redis overview elasticache engines features pricing resources customers elasticache for memcached elasticache for redis global datastore products database amazon elasticache amazon elasticache for redis amazon elasticache for redis redis compatible in-memory data store built for the cloud. power real-time applications with sub-millisecond latency. start with amazon elasticache for redis for free amazon elasticache for redis is a blazing fast in-memory data store that provides sub-millisecond latency to power internet-scale real-time applications. built on open-source redis and compatible with the redis apis, elasticache for redis works with your redis clients and uses the open redis data format to store your data. your self-managed redis applications can work seamlessly with elasticache for redis without any code changes. elasticache for redis combines the speed, simplicity, and versatility of open-source redis with manageability, security, and scalability from amazon to power the most demanding real-time applications in gaming, ad-tech, e-commerce, healthcare, financial services, and iot. introduction to amazon elasticache for redis ( : ) benefits extreme performance amazon elasticache for redis works as an in-memory data store to support the most demanding applications requiring sub-millisecond response times. in comparison to disk-based databases where most operations need a roundtrip to disk, in-memory data stores manage data in-memory which is an order of magnitude faster than disks. the result is blazing fast performance with average read or write operations taking less than a millisecond and support for hundreds of millions of operations per second within a cluster. elasticache gives you an optimized end-to-end hardware and software stack for blazing fast performance. secure starting with amazon elasticache for redis , elasticache now provides you with the ability to create and manage users and user groups that can be used to set up role-based access control (rbac) for redis commands. you can now simplify your architecture while maintaining security boundaries and also take advantage of granular access control to manage groups. amazon elasticache for redis supports amazon vpc, so you can isolate your cluster to the ip ranges you choose for your nodes. the elasticache team continuously monitors for known security vulnerabilities in open-source redis, operating system, and firmware to keep your redis environment secure. it is pci compliant, hipaa eligible, fedramp authorized, and offers encryption in transit, at rest (including customer managed cmk stored in aws kms), and redis auth for secure internode communications to help keep sensitive data such as personally identifiable information (pii) safe. learn more about role-based access control (rbac) » learn more about vpc for elasticache » fully managed and hardened amazon elasticache for redis is a fully managed service. you no longer need to perform management tasks such as hardware provisioning, software patching, setup, configuration, monitoring, failure recovery, and backups. elasticache continuously monitors your clusters to keep your redis up and running so that you can focus on higher value application development. it provides detailed monitoring metrics of your redis usage, enabling you to track application trends and adjust cluster configuration, as needed. elasticache adds automatic write throttling, intelligent swap memory management, and failover enhancements to improve upon the availability and manageability of open source redis. learn more about amazon elasticache for redis » redis-compatible redis is a widely adopted in-memory data store for use as a database, cache, message broker, queue, session store, and leaderboard. amazon elasticache for redis maintains compatibility with the open source redis data formats, redis apis, and works with redis clients. you can migrate your self-managed redis workloads to elasticache for redis without any code change. learn more about open-source redis » new: redis compatibility for amazon elasticache » highly available and reliable amazon elasticache for redis supports both redis cluster and non-cluster modes and provides high availability via support for automatic failover by detecting primary node failures and promoting a replica to be primary with minimal impact. it allows for read availability for your application by supporting read replicas (across availability zones), to enable the reads to be served when the primary is busy with the increased workload. elasticache for redis supports enhanced failover logic to allow for automatic failover in cases when majority of the primary nodes for redis cluster mode are unavailable. on redis . . onwards, auto-failover enabled clusters provide online configuration changes for all planned operations. learn more about automatic failover for elasticache » easily scalable with amazon elasticache for redis, you can start small and easily scale your redis data as your application grows - all the way up to a cluster with tb of in-memory data. it allows you to scale your redis cluster environment up to nodes and shards. it supports online cluster resizing to scale-out and scale-in your redis clusters without downtime and adapts to changing demand. to scale read capacity, elasticache allows you to add up to five read replicas across multiple availability zones. to scale write capacity, elasticache supports redis cluster mode which enables you to partition your write traffic across multiple primaries. learn more about scaling elasticache » how it works use cases amazon elasticache for redis is a great choice for real-time transactional and analytical processing use cases such as caching, chat/messaging, gaming leaderboards, geospatial, machine learning, media streaming, queues, real-time analytics, and session store. caching amazon elasticache for redis is a great choice for implementing a highly available, distributed, and secure in-memory cache to decrease access latency, increase throughput, and ease the load off your relational or nosql databases and applications. elasticache can serve frequently requested items at sub-millisecond response times, and enables you to easily scale for higher loads without growing the costlier backend databases. database query results caching, persistent session caching, and full-page caching are all popular examples of caching with elasticache for redis. learn how to build a caching application with elasticache for redis. chat and messaging amazon elasticache for redis supports the pub/sub standard with pattern matching. this allows elasticache for redis to support high performance chat rooms, real-time comment streams, and server intercommunication. you can also use pub/sub to trigger actions based on published events. learn how to build a chat application with elasticache for redis. gaming leaderboards real-time gaming leaderboards are easy to create with amazon elasticache for redis. just use the redis sorted set data structure, which provides uniqueness of elements while maintaining the list sorted by their scores. creating a real-time ranked list is as simple as updating a user's score each time it changes. you can also use sorted sets to handle time series data by using timestamps as the score. geospatial amazon elasticache for redis offers purpose-built in-memory data structures and operators to manage real-time geospatial data at scale and speed. you can use elasticache for redis to add location-based features such as drive time, drive distance, and points of interests to your applications. learn how to build a geospatial application with elasticache for redis. machine learning amazon elasticache for redis gives you a fast in-memory data store to build and deploy machine learning models quickly. use elasticache for redis for use cases such as fraud detection in gaming and financial services, real-time bidding in ad-tech, and matchmaking in dating and ride sharing to process live data and make decisions within tens of milliseconds. learn how coffee meets bagel uses elasticache for real-time machine learning-based dating recommendations. media streaming amazon elasticache for redis offers a fast, in-memory data store to power live streaming use cases. elasticache for redis can be used to store metadata for user profile and viewing history, authentication information/tokens for millions of users, and manifest files to enable cdns to stream videos to millions of mobile and desktop users at a time. queues amazon elasticache for redis offers list data structure making it easy to implement a lightweight, persistent queue. lists offer atomic operations as well as blocking capabilities, making them suitable for a variety of applications that require a reliable message broker or a circular list. real-time analytics use amazon elasticache for redis with streaming solutions such as apache kafka and amazon kinesis as an in-memory data store to ingest, process, and analyze real-time data with sub-millisecond latency. elasticache is an ideal choice for real-time analytics use cases such as social media, ad targeting, personalization, and iot and time-series data analytics. session store amazon elasticache for redis is highly suited as a session store to manage session information such as user authentication tokens, session state, and more. simply use elasticache for redis as a fast key-value store with appropriate ttl on session keys to manage your session information. session management is commonly required for online applications, including games, e-commerce websites, and social media platforms. learn how to use elasticache for redis as a session store. customers learn how adobe uses elasticache for its api platform. recommendation models using amazon elasticache for redis at coffee meets bagel learn how capitalone uses elasticache for real-time banking apps. learn how dream scaled-in and out their platform to meet . million requests per second. more customer stories >> learn how grab uses elasticache for its ride-hailing app. learn how zynga uses elasticache for multiplayer gaming experience get started with amazon elasticache for redis sign up get access to the elasticache free tier. learn with simple tutorials explore how to create a redis cluster. start building begin building with help from the user guide. more elasticache resources amazon elasticache for redis faqs self-managed vs. fully managed redis getting started with amazon elasticache more amazon elasticache for redis customers amazon elasticache blog take the free introduction to amazon elasticache course ready to get started? sign up have more questions? contact us sign in to the console learn about aws what is aws? what is cloud computing? what is devops? what is a container? what is a data lake? aws cloud security what's new blogs press releases resources for aws getting started training and certification aws solutions portfolio architecture center product and technical faqs analyst reports aws partner network developers on aws developer center sdks & tools .net on aws python on aws java on aws php on aws javascript on aws help contact us aws careers file a support ticket knowledge center aws support overview legal create an aws account amazon is an equal opportunity employer: minority / women / disability / veteran / gender identity / sexual orientation / age. language عربي bahasa indonesia deutsch english español français italiano português tiếng việt türkçe Ρусский ไทย 日本語 한국어 中文 (简体) 中文 (繁體) privacy | site terms | cookie preferences | © , amazon web services, inc. or its affiliates. all rights reserved. dshr's blog dshr's blog i'm david rosenthal, and this is a place to discuss the work i'm doing in digital preservation. talk at berkeley's information access seminar chromebook linux update effort balancing and rate limits isp monopolies the bitcoin "price" two million page views! the new oldweb.today michael nelson's group on archiving twitter stablecoins risc vs. cisc max ungrounding i rest my case storage media update even more on the ad bubble the order flow the long now unbanking the banked a note on blockchains liability in the software supply chain moxie marlinspike on decentralization don't say we didn't warn you amazon is profitable? open source saturation shout-out to gutenberg project lack of anti-trust enforcement widely used open source software contained bitcoin-stealing backdoor | ars technica skip to main content biz & it tech science policy cars gaming & culture store forums subscribe close navigate store subscribe videos features reviews rss feeds mobile site about ars staff directory contact us advertise with ars reprints filter by topic biz & it tech science policy cars gaming & culture store forums settings front page layout grid list site theme black on white white on black sign in comment activity sign up or login to join the discussions! stay logged in | having trouble? sign up to comment and more sign up poisoning the well — widely used open source software contained bitcoin-stealing backdoor malicious code that crept into event-stream javascript library went undetected for weeks. dan goodin - nov , : pm utc jeremy brooks / flickr reader comments with posters participating share this story share on facebook share on twitter share on reddit a hacker or hackers sneaked a backdoor into a widely used open source code library with the aim of surreptitiously stealing funds stored in bitcoin wallets, software developers said monday. the malicious code was inserted in two stages into event-stream, a code library with million downloads that’s used by fortune companies and small startups alike. in stage one, version . . , published on september , included a benign module known as flatmap-stream. stage two was implemented on october when flatmap-stream was updated to include malicious code that attempted to steal bitcoin wallets and transfer their balances to a server located in kuala lumpur. the backdoor came to light last tuesday with this report from github user ayrton sparling. officials with the npm, the open source project manager that hosted event-stream, didn’t issue an advisory until monday, six days later. npm officials said the malicious code was designed to target people using a bitcoin wallet developed by copay, a company that incorporated event-stream into its app. this release from earlier this month shows copay updating its code to refer to flatmap-stream, but a copay official said in a github discussion that the malicious code was never deployed in any platforms. after this post went live, copay officials updated their comment to say they did, in fact, release platforms that contained the backdoor. in a blog post published after this post went live, copay officials said versions . . through . . were affected by the backdoor and that users with these versions installed should avoid running the app until after installing version . . . the post stated: users should assume that private keys on affected wallets may have been compromised, so they should move funds to new wallets (v . . ) immediately. users should not attempt to move funds to new wallets by importing affected wallets' twelve word backup phrases (which correspond to potentially compromised private keys). users should first update their affected wallets ( . . - . . ) and then send all funds from affected wallets to a brand new wallet on version . . , using the send max feature to initiate transactions of all funds. the company continues to investigate the attack. it is also contacting copay-dash, another developer that uses the same open source code in its wallet app. “this compromise was not targeting module developers in general or really even developers,” an npm official told ars in an email. “it targeted a select few developers at a company, copay, that had a very specific development environment set up. even then, the payload itself didn’t run on those developers’ computers; rather, it would be packaged into a consumer-facing app when the developers built a release. the goal was to steal bitcoin from this application’s end users.” advertisement supply chain attacks abound according to the github discussion that exposed the backdoor, the longtime event-stream developer no longer had time to provide updates. so several months ago, he accepted the help of an unknown developer. the new developer took care to keep the backdoor from being discovered. besides being gradually implemented in stages, it also narrowly targeted only the copay wallet app. the malicious code was also hard to spot because the flatmap-stream module was encrypted. further reading two new supply-chain attacks come to light in less than a week the attack is the latest to exploit weaknesses in a widely used supply chain to target downstream end users. last month, two supply-side attacks came to light in a single week. one targeted vestacp, a control-panel interface that system administrators use to manage servers. the attackers then modified an installer that was available on vestacp’s website. the second supply-chain attack slipped a malicious package into pypi, the official repository for the widely used python programming language. the pypi event came two years after a college student’s bachelor thesis used a similar technique to get an unauthorized python module executed more than , times on more than , separate domains. some belonged to us governmental and military organizations. the supply-chain attacks show one of the weaknesses of open source code. because of its openness and the lack of funds of many of its hobbyist developers and users, open source code can be subject to malicious modifications that often escape notice. npm uses a feature called lockfile that requests only specific versions of code. that makes it possible for people to use only known good versions of a package when there are buggy or malicious versions that they depend on. last year, npm also acquired lift security, a company that maintained a database of known javascript vulnerabilities. npm has since built the database directly into its command-line tool. the ability for malicious code to make its way into a code library used by so many applications and then escape notice for weeks shows that these npm measures, while useful, are by no means sufficient. the time has come for maintainers and users of open source software to devise new measures to better police the millions of packages being used all around us. this post was updated to add copay comments that some platforms deployed the backdoor after all and, later, to add comments from a blog post. reader comments with posters participating share this story share on facebook share on twitter share on reddit dan goodin dan is the security editor at ars technica, which he joined in after working for the register, the associated press, bloomberg news, and other publications. email dan.goodin@arstechnica.com // twitter @dangoodin advertisement you must login or create an account to comment. channel ars technica ← previous story next story → related stories sponsored stories powered by today on ars store subscribe about us rss feeds view mobile site contact us staff advertise with us reprints newsletter signup join the ars orbital transmission mailing list to get weekly updates delivered to your inbox. sign me up → cnmn collection wired media group © condé nast. all rights reserved. use of and/or registration on any portion of this site constitutes acceptance of our user agreement (updated / / ) and privacy policy and cookie statement (updated / / ) and ars technica addendum (effective / / ). ars may earn compensation on sales from links on this site. read our affiliate link policy. your california privacy rights | do not sell my personal information the material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of condé nast. ad choices none bibliographic wilderness bibliographic wilderness product management in my career working in the academic sector, i have realized that one thing that is often missing from in-house software development is &# ;product management.&# ; but what does that mean exactly? you don&# ;t know it&# ;s missing if you don&# ;t even realize it&# ;s a thing and people can use different terms to mean different roles/responsibilities. basically, &# ; continue reading product management &# ; rails auto-scaling on heroku we are investigating moving our medium-small-ish rails app to heroku. we looked at both the rails autoscale add-on available on heroku marketplace, and the hirefire.io service which is not listed on heroku marketplace and i almost didn&# ;t realize it existed. i guess hirefire.io doesn&# ;t have any kind of a partnership with heroku, but still uses &# ; continue reading rails auto-scaling on&# ;heroku &# ; managed solr saas options i was recently looking for managed solr &# ;software-as-a-service&# ; (saas) options, and had trouble figuring out what was out there. so i figured i&# ;d share what i learned. even though my knowledge here is far from exhaustive, and i have only looked seriously at one of the ones i found. the only managed solr options i &# ; continue reading managed solr saas&# ;options &# ; gem authors, check your release sizes most gems should probably be a couple hundred kb at most. i&# ;m talking about the package actually stored in and downloaded from rubygems by an app using the gem. after all, source code is just text, and it doesn&# ;t take up much space. ok, maybe some gems have a couple images in there. but if &# ; continue reading gem authors, check your release&# ;sizes &# ; every time you decide to solve a problem with code… every time you decide to solve a problem with code, you are committing part of your future capacity to maintaining and operating that code. software is never done. software is drowning the world by james abley updating solrcloud configuration in ruby we have an app that uses solr. we currently run a solr in legacy &# ;not cloud&# ; mode. our solr configuration directory is on disk on the solr server, and it&# ;s up to our processes to get our desired solr configuration there, and to update it when it changes. we are in the process of moving &# ; continue reading updating solrcloud configuration in&# ;ruby &# ; are you talking to heroku redis in cleartext or ssl? in &# ;typical&# ; redis installation, you might be talking to redis on localhost or on a private network, and clients typically talk to redis in cleartext. redis doesn&# ;t even natively support communications over ssl. (or maybe it does now with redis ?) however, the heroku redis add-on (the one from heroku itself) supports ssl connections via &# ;stunnel&# ;, &# ; continue reading are you talking to heroku redis in cleartext or&# ;ssl? &# ; comparing performance of a rails app on different heroku formations i develop a &# ;digital collections&# ; or &# ;asset management&# ; app, which manages and makes digitized historical objects and their descriptions available to the public, from the collections here at the science history institute. the app receives relatively low level of traffic (according to google analytics, around k pageviews a month), although we want it to be &# ; continue reading comparing performance of a rails app on different heroku&# ;formations &# ; deep dive: moving ruby projects from travis to github actions for ci so this is one of my super wordy posts, if that&# ;s not your thing abort now, but some people like them. we&# ;ll start with a bit of context, then get to some detailed looks at github actions features i used to replace my travis builds, with example config files and examination of options available. for &# ; continue reading deep dive: moving ruby projects from travis to github actions for&# ;ci &# ; unexpected performance characteristics when exploring migrating a rails app to heroku i work at a small non-profit research institute. i work on a rails app that is a &# ;digital collections&# ; or &# ;digital asset management&# ; app. basically it manages and provides access (public as well as internal) to lots of files and description about those files, mostly images. it&# ;s currently deployed on some self-managed amazon ec instances &# ; continue reading unexpected performance characteristics when exploring migrating a rails app to&# ;heroku &# ; faster_s _url: optimized s url generation in ruby subsequent to my previous investigation about s url generation performance, i ended up writing a gem with optimized implementations of s url generation. github: faster_s _url it has no dependencies (not even aws-sdk). it can speed up both public and presigned url generation by around an order of magnitude. in benchmarks on my macbook compared &# ; continue reading faster_s _url: optimized s url generation in&# ;ruby &# ; delete all s key versions with ruby aws sdk v if your s bucket is versioned, then deleting an object from s will leave a previous version there, as a sort of undo history. you may have a &# ;noncurrent expiration lifecycle policy&# ; set which will delete the old versions after so many days, but within that window, they are there. what if you were deleting &# ; continue reading delete all s key versions with ruby aws sdk&# ;v &# ; github actions tutorial for ruby ci on drifting ruby i&# ;ve been using travis for free automated testing (&# ;continuous integration&# ;, ci) on my open source projects for a long time. it works pretty well. but it&# ;s got some little annoyances here and there, including with github integration, that i don&# ;t really expect to get fixed after its acquisition by private equity. they also seem to &# ; continue reading github actions tutorial for ruby ci on drifting&# ;ruby &# ; more benchmarking optimized s presigned_url generation in a recent post, i explored profiling and optimizing s presigned_url generation in ruby to be much faster. in that post, i got down to using a aws::sigv ::signer instance from the aws sdk, but wondered if there was a bunch more optimization to be done within that black box. julik posted a comment on that &# ; continue reading more benchmarking optimized s presigned_url&# ;generation &# ; delivery patterns for non-public resources hosted on s i work at the science history institute on our digital collections app (written in rails), which is kind of a &# ;digital asset management&# ; app combined with a public catalog of our collection. we store many high-resolution tiff images that can be mb+ each, as well as, currently, a handful of pdfs and audio files. we &# ; continue reading delivery patterns for non-public resources hosted on&# ;s &# ; speeding up s url generation in ruby it looks like the aws sdk is very slow at generating s urls, both public and presigned, and that you can generate around an order of magnitude faster in both cases. this can matter if you are generating hundreds of s urls at once. my app the app i work is a &# ;digital collections&# ; or &# ; continue reading speeding up s url generation in&# ;ruby &# ; a custom local ohms front-end here at the science history institute, we’ve written a custom ohms viewer front-end, to integrate seamlessly with our local custom &# ;content management system&# ; (a rails-based digital repository app with source available), and provide some local functionality like the ability to download certain artifacts related to the oral history. we spent quite a bit of energy &# ; continue reading a custom local ohms&# ;front-end &# ; encrypting patron data (in rails): why and how special guest post by eddie rubeiz i&# ;m eddie rubeiz. along with the owner of this blog, jonathan rochkind, and our system administrator, dan, i work on the science history institute&# ;s digital collections website, where you will find, among other marvels, this picture of the inventor of styrofoam posing with a santa &# ;sculpture&# ;, which predates the &# ; continue reading encrypting patron data (in rails): why and&# ;how &# ; intentionally considering fixity checking in our digital collections app rewrite at science history institute, we took a moment to step back and be intentional about how we approach &# ;fixity checking&# ; features and ui, to make sure it&# ;s well-supporting the needs it&# ;s meant to. i think we do a good job of providing ui to let repository managers and technical &# ; continue reading intentionally considering fixity&# ;checking &# ; sprockets and your rails app sprockets . was released on october th , after several years of beta, congratulations and hooray. there are a couple confusing things that may give you trouble trying to upgrade to sprockets that aren&# ;t covered very well in the changelog or upgrade notes, although now that i&# ;ve taken some time to understand it, i &# ; continue reading sprockets and your rails&# ;app &# ; dshr's blog: travels with a chromebook dshr's blog i'm david rosenthal, and this is a place to discuss the work i'm doing in digital preservation. tuesday, january , travels with a chromebook two years ago i wrote a note of thanks as i switched my disposable travel laptop from an asus seashell to an acer c chromebook running linux. two years later i'm still traveling with a c . below the fold, an update on my experiences. as i wrote two years ago, i started by replacing chromeos with hugegreenbug's port of ubuntu . . this gave me almost exactly the ui i was then running on my desktop, so i was happy with one exception: i'm still not used to the much bigger touch pad, and keep accidentally moving the cursor by touching the touch pad with the balls of my thumbs. this turned out to be a permanent irritation (but see below for what i should have done to fix it). although the combination of the c and ubuntu . gave sterling service for a long time, i started noticing an occasional crash. one of them was a very spectacular snow crash, which caused me to suspect a hardware problem. it was a day and a half before a trip, so i resorted again to amazon's same-day delivery for a second c . by the time it arrived all seemed well again with the first one, so i left with it and had no problem on the trip. now i had two c s, so i could experiment with alternate linux ports. i installed galliumos on the second one, and worked with it for a while. it was totally crash-free, being based on xubuntu . was significantly more up-to-date, and it was certainly lean-and-mean. derrick diener's galliumos: the linux distro specially designed for chromebook is a good review of galliumos. but i just couldn't come to terms with the xfce desktop and the way the touchpad had been configured to mimic the chromeos touchpad. i'd traded one irritation for two. then i found mark solters' ubuntu . on the google pixel , which reported: i hit upon trying linux mint “sarah”, currently beta. this version of mint is . based, and the installer boots on the pixel ! so i downloaded linux mint . "serena" - cinnamon ( -bit), wrote the .iso to a usb drive, popped it into the first c and powered up. the mint live system came right up! i played with it for a while. everything seemed to work, and the cinnamon ui was a lot more to my taste than xfce. worth a try. after backing up my files from . , i double-clicked on "install linux mint". after you answer the questions about keyboard and time zone and so on, the install starts with a worrisome long pause with the spinning cursor, but then things proceed normally with a progress bar. after it finished and i rebooted, i had a default linux mint system with gb of free space. my next step was to connect to my wifi network and sudo apt-get update sudo apt-get upgrade which updated packages. i have no travel for the next month and a working if slightly irritating environment on c # , so i can take the opportunity to clean things up and customize a nice fresh environment on c # . for the programming environment i needed: git - the lockss program is converging on gitlab.com and gitlab ce. emacs - which i still use for programming although i also use vi. the jdk, ant, libxml -utils and junit - needed to build the lockss software. docker for more general needs i added: tex live- still my favorite environment for writing papers. keepassx - everyone should use a password safe. gnucash zoom.us - a program committee i'm on uses this for distributed meetings, it works really well. so far, everything else i need is in the default install. at this point i have gb free. not a lot, but as in the . days i will travel with an external ssd that holds my development environment. then i closed the lid, had a cup of tea, and opened the lid again. things seemed to be working but, as i quickly discovered there was a problem. it turns out that the mint kernel shows symptoms that appeared on chromebook linux wikis back in : suspend/resume works the first time, but the next time the screen locks but the processor keeps running indefinitely. shutdown behaves similarly. and after a resume /var/log/kern.log is continually spammed with messages from ehci-pci saying resume error - . not good. i tried various fixes found by google-ing but none worked. then i had an inspiration. i had two machines: c # with a working kernel and a userland i didn't like. c # with a broken kernel and a userland i liked. all i needed to do was to get c # to boot the galliumos kernel not the mint one. i re-installed vanilla mint . on c # then here is what i did to it. stage was to get the galliumos fixes relevant to the suspend/resume issue into the mint userland: check out the calliumos config files for haswell by: git clone https://github.com/galliumos/galliumos-haswell copy each of the files into its appropriate place. stage was to get the galliumos kernel installed on c # : create a file /etc/apt/sources.list.d/additional-repositories.list containing the line: deb http://apt.galliumos.org/ xenon main copy the file /etc/apt/trusted.gpg.d/galliumos.gpg from c # to c # . run synaptic, reload, search for linux-image-galliumos, mark it for installation, apply. set the grub bootloader to display its menu by default so that if something goes wrong you can boot the mint kernel using the "advanced options" menu item. edit /etc/default/grub to read: #grub_hidden_timeout= grub_hidden_timeout_quiet=false grub_timeout= run update-grub to make the changes take effect. reboot, and the system comes up: $ uname -r . . -galliumos $ now suspend/resume works just the way it does on galliumos! three years ago redditor successincircuit posted list of fixes for xubuntu . on the acer c . some of these fixes still seem to be relevant: galliumos follows successincircuit's advice to optimize for the c s ssd. so i edited the root file system entry in /etc/fstab to be: /dev/mapper/mint--vg-root / ext discard,relatime,errors=remount-ro i find it necessary to disable the touchpad while typing. successincircuit says to do this with syndaemon, but there is a better way for ubuntu . , installing a touchpad control applet in the tray: $ sudo add-apt-repository ppa:atareao/atareao $ sudo apt-get update $ sudo apt-get install touchpad-indicator brave souls have upgraded the m. ssd, reinstalled chromeos, then installed linux, but i'm not yet up for that despite the lack of space. i'm now working on building the nice fresh environment on c # before my next trip. if i encounter anything else interesting i'll comment on this post. update th may : after another strange crash on c # which corrupted its disk, i found c # refurbished for $ . on amazon. this time i also got a gb m. ssd to replace the stock one. update th may i installed the gb ssd, recovered chrome os, installed mint cinnamon . and now have over gb free space on disk. to install the ssd i: turned the c over and removed the screws holding the back cover on. cracked the case open using a small screwdriver to get a front corner open, then slid a guitar plectrum along to free up the the rest of the clips. this step is a bit scary! removed the screw holding down the original ssd, replaced it with the new one, and replaced the screw. put the back cover back in place, pressed down on it to get most of the clips seated. replaced the screws, tightening them gently to seat the remaining clips. so now i have functional c s, albeit one with a habit of occasional mysterious crashes, each with mint userland and galliumos kernel. gratuitous copy and paste i notice that the links to the instructions for installing linux from a note of thanks are now broken, so i'm cutting and pasting them from the archlinux wiki so i can find them easily: enabling developer mode developer mode is necessary in order to access the superuser shell inside chrome os, which is required for making changes to the system like allow booting through seabios. warning: enabling developer mode will wipe all of your data. to enable developer mode: turn on the chrome os device. press and hold the esc + f (refresh) keys, then press the power button. this enters recovery mode. press ctrl + d (no prompt). it will ask you to confirm, then the system will revert its state and enable developer mode. note: press ctrl + d (or wait seconds for the beep and boot) at the white boot splash screen to enter chrome os. accessing the superuser shell after you have enabled the developer mode you will need to access the superuser shell. how you do this depends on whether you have configured chrome os or not. accessing the superuser shell without chrome os configuration if you have not configured chrome os, just press ctrl + alt + f (f is the "forward" arrow on the top row, →), you will see a login prompt. use chronos as the username, it should not prompt you for a password. become superuser with sudo, use the command sudo su -. accessing the superuser shell with chrome os configuration if you have configured chrome os already: open a crosh window with ctrl + alt + t. open a bash shell with the shell command. become superuser with sudo, use the command sudo su - to accomplish that. enabling seabios if your chrome os device did not ship with seabios or you prefer to install a custom firmware, then continue to flashing a custom firmware. this method will allow you to access the pre-installed version of seabios through the developer mode screen in coreboot. inside your superuser shell enter: # crossystem dev_boot_usb= dev_boot_legacy= reboot the machine. booting the installation media plug the usb drive to the chrome os device and start seabios with ctrl + l at the white boot splash screen (if seabios is not set as default). press esc to get a boot menu and select the number corresponding to your usb drive. i'll update this post again after i've tried replacing the ssd. posted by david. at : am labels: amazon, linux comments: david. said... after several trips with the galliumos kernel and the mint user-land everything is mostly fine. suspend has worked almost always· but it is important to note that suspending takes a very variable amount of time, sometimes several minutes. and interrupting the process while it is underway, for example by plugging or unplugging the power, seems to trigger the catatonic state. this may be because the machine decides to hibernate rather than suspend; i still need to figure out how to completely disable hibernate mode. april , at : am david. said... one more anomaly. the power brick is rated . a on the ac side. with the system charging the battery and in use playing a youtube video, it actually draws . a w from a v supply (measured with a kill-a-watt meter). but if you plug it into some sockets, such as the sockets on hotel lamps (claimed to supply a), and some of the seat power sockets on united, it appears to trip the current limiter and it doesn't get power. plug a usb charger into the same socket and it gets power. i don't understand why supplying a bit over half an amp causes a problem. april , at : pm david. said... at ars technica, j. m. porup has a review of two ways to run linux on a chromebook, crouton and the one i use, galliumos. june , at : am david. said... while i was on the road in new zealand, the power brick failed. when i got home, i replaced it with a much better one, smaller, lighter, with an indicator light, only two pins not three. february , at : pm david. said... recently i've been plagued by a failure of the cinnamon screen lock program to allow me to type my password after the machine wakes up. the only way out seems to be power-cycle, which is annoying. so i've replaced cinnamon with mate, which uses a different screen lock, on one of my machines and will use it for a while to see if the problem recurs. june , at : pm david. said... i should have known better. the ztc gb ssd, model ztc-sm - g that i bought from hot deals less(!) on amazon lasted . weeks of light usage in c # before failing with a massive outbreak of bad blocks. i sent a message to the vendor via amazon, but after two business days have yet to hear back from them. a few more days of no response and they'll get a -star review for crappy product and lousy customer support. july , at : pm david. said... the good news is that, so far, mate's screen lock has performed flawlessly. july , at : pm david. said... the saga continues. i replaced the ztc gb ssd with an adata gb ssd. it, and c # have been working fine except that # 's touchpad suffers an annoying jitter, so it was only comfortably usable with a mouse. so, having overcome my qualms about opening c s' cases, i upgraded # with a transcend gb ssd. the one remaining issue is that, very occasionally, when they are asleep each of the c s appear to have a nightmare and wake up. this drains the battery, and if they're in the bag at the time, can get them quite hot. october , at : pm david. said... google has announced official linux support for chromebooks: "google today announced chrome os is getting linux support. as a result, chromebooks will soon be able to run linux apps and execute linux commands. a preview of linux on the pixelbook will be released first, with support for more devices coming soon." linux runs in a vm under chromeos. may , at : pm david. said... this comment has been removed by the author. may , at : pm david. said... this comment has been removed by the author. may , at : pm david. said... i deleted the two previous comments because further experimentation revealed they were wrong. it appears that up-to-date mint (mate) plus the galliumos userland fixes has two problems on the c : ) with the galliumos . . kernel it suffers a gpu hang on resume from sleep that logs out the user. ) with the mint generic kernel . . - it suffers the same long-standing resume problem with ehci. mint . with the galliumos . . kernel works fine. may , at : am david. said... also, installing mint with an encrypted disk works, but the second time the system starts from cold entering the disk passphrase you set does not successfully decrypt the disk. may , at : pm david. said... after some experimentation, i now have mint . running on a c with the vanilla . . - -generic kernel not the galliumos one. instructions: * git clone https://github.com/galliumos/galliumos-haswell * copy files into place * edit /etc/default/grub as above * run blkid to get the uuid of the swap partition * create a file /etc/initramfs-tools/conf.d/resume contents: resume=uuid= * run update-grub * run update-initramfs -u -k all this kernel does not have the gpu hang. august , at : pm david. said... mate . works fine on the c but it can't use full-disk encryption. selecting full-disk encryption during the install process works, but the first or sometimes the second restart corrupts the encrypted volume so that the passphrase set during installation does not decrypt it. home directory encryption works fine on an unencrypted disk. this problem does not happen on a pc; i have verified this by installing mate . on an intel nuc. august , at : am david. said... tim anderson points out another reason to run linux on your chromebook in buying a chromebook? don't forget to check that best-before date: "it is unlikely to be printed on the box, but every chromebook has an "auto update expiration (aue) date" after which the operating system is unsupported by google. the authoritative document on the subject is here, where google explains that it "provides each new hardware platform with . years of auto update support". while . years sounds reasonable, google starts the clock ticking "when the first device on the platform is released". ... what happens when the dreaded aue date passes? this means there will be no more automatic software updates from google, no technical support from google, and "business and education customers... should not expect that they can manage their devices as expected using the google admin console"." it is (deliberately?) hard to figure out what the aue date is for your chromebook, but it looks as though my acer c s aue was last june. but they still work fine and get software updates because they run linux! august , at : am david. said... another good source of c information is here. december , at : am david. said... the c 's touchpad is one of its downsides, so you need to travel with a mouse. the tracpoint is way the best travel mouse i've ever used, and one of my kickstarter successes. it is tiny, yet the ergonomics are great - the grip looks strange but it feels like a pen. it is so small it is easily usable on the c 's space alongside the touchpad, which measures a scant . " wide by . " high. march , at : am david. said... the battery in the first of my c s expired. i ordered a dentsing ap j k battery from amazon ($ . including tax). i opened the case, undid the two screws holding the battery, replaced it and the screws, and reassembled the case. it works fine. october , at : pm david. said... this comment has been removed by the author. october , at : pm david. said... mint kernels up to and including . . - -generic go to sleep normally when the lid is closed, and wake up normally when it is opened. subsequent kernels . . - and . . - appear to go to sleep normally when the lid is closed but when it is opened do not wake up, they do a cold boot. this is possibly related to the known problem in the latest mint that prevents it working with encrypted swap, with which my c s are configured. october , at : pm post a comment newer post older post home subscribe to: post comments (atom) blog rules posts and comments are copyright of their respective authors who, by posting or commenting, license their work under a creative commons attribution-share alike . united states license. off-topic or unsuitable comments will be deleted. dshr dshr in anwr recent comments full comments blog archive ► ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ▼ ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ▼ january ( ) preservable emulations rick whitt on digital preservation the long tail of non-english science gresham's law star wars storage media transition (personal) error : blogger is bloggered travels with a chromebook ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► july ( ) ► june ( ) ► february ( ) ► ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) lockss system has permission to collect, preserve, and serve this archival unit. simple theme. powered by blogger. dshr's blog: securing the software supply chain dshr's blog i'm david rosenthal, and this is a place to discuss the work i'm doing in digital preservation. tuesday, december , securing the software supply chain this is the second part of a series about trust in digital content that might be called: is this the real life? is this just fantasy? the first part was certificate transparency, about how we know we are getting content from the web site we intended to. this part is about how we know we're running the software we intended to. this question, how to defend against software supply chain attacks, has been in the news recently: a hacker or hackers sneaked a backdoor into a widely used open source code library with the aim of surreptitiously stealing funds stored in bitcoin wallets, software developers said monday. the malicious code was inserted in two stages into event-stream, a code library with million downloads that's used by fortune companies and small startups alike. in stage one, version . . , published on september , included a benign module known as flatmap-stream. stage two was implemented on october when flatmap-stream was updated to include malicious code that attempted to steal bitcoin wallets and transfer their balances to a server located in kuala lumpur. see also here and here. the good news is that this was a highly specific attack against a particular kind of cryptocurrency wallet software; things could have been much worse. the bad news is that, however effective they may be against some supply chain attacks, none of the techniques i discuss below the fold would defend against this particular attack. in an important paper entitled software distribution transparency and auditability, benjamin hof and georg carle from tu munich use debian's advanced package tool (apt) as an example of a state-of-the-art software supply chain, and: describe how apt works to maintain up-to-date software on clients by distributing signed packages. review previous efforts to improve the security of this process. propose to enhance apt's security by layering a system similar to certificate transparency (ct) on top. detail the operation of their systems' logs, auditors and monitors, which are similar to ct's in principle but different in detail. describe and measure the performance of an implementation of their layer on top of apt using the trillian software underlying some ct implementations. there are two important "missing pieces" in their system, and all the predecessors, which are the subjects of separate efforts: reproducible builds. bootstrappable compilers. how apt works a system running debian or other apt-based linux distribution runs software it received in "packages" that contain the software files, and metadata that includes dependencies. their hashes can be verified against those in a release file, signed by the distribution publisher. packages come in two forms, source and compiled. the source of a package is signed by the official package maintainer and submitted to the distribution publisher. the publisher verifies the signature and builds the source to form the compiled package, whose hash is then included in the release file. the signature on the source package verifies that the package maintainer approves this combination of files for the distributor to build. the signature on the release file verifies that the distributor built the corresponding set of packages from approved sources and that the combination is approved for users to install. previous work it is, of course, possible for the private keys on which the maintainer's and distributor's signatures depend to be compromised: samuel et al. consider compromise of signing keys in the design of the update framework (tuf), a secure application updater. to guard against key compromise, tuf introduces a number of different roles in the update release process, each of which operates cryptographic signing keys. the following three properties are protected by tuf. the content of updates is secured, meaning its integrity is preserved. securing the availability of updates protects against freeze attacks, where an outdated version with known vulnerabilities is served in place of a security update. the goal of maintaining the correct combination of updates implies the security of meta data. the goal of introducing multiple roles each with its own key is to limit the damage a single compromised key can do. an orthogonal approach is to implement multiple keys for each role, with users requiring a quorum of verified signatures before accepting a package: nikitin et al. develop chainiac, a system for software update transparency. software developers create a merkle tree over a software package and the corresponding binaries. this tree is then signed by the developer, constituting release approval. the signed trees are submitted to co-signing witness servers. the witnesses require a threshold of valid developer signatures to accept a package for release. additionally, the mapping between source and binary is verified by some of the witnesses. if these two checks succeed, the release is accepted and collectively signed by the witnesses. the system allows to rotate developer keys and witness keys, while the root of trust is an offline key. it also functions as a timestamping service, allowing for verification of update timeliness. ct-like layer hof and carle's proposal is to use verifiable logs, similar to those in ct, to ensure that malfeasance is detectable. they write: compromise of components and collusion of participants must not result in a violation of the following security goals remaining undetected. a goal of our system is to make it infeasible for the attacker to deliver targeted backdoors. for every binary, the system can produce the corresponding source code and the authorizing maintainer. defined irregularities, such as a failure to correctly increment version numbers, also can be detected by the system. as i understand it, this is accurate but somewhat misleading. their system adds a transparency layer on top of apt: the apt release file identifies, by cryptographic hash, the packages, sources, and meta data which includes dependencies. this release file, meta data, and source packages are submitted to a log server operating an appendonly merkle tree, as shown in figure . the log adds a new leaf for each file. we assume maintainers may only upload signed source packages to the archive, not binary packages. the archive submits source packages to one or more log servers. we further assume that the buildinfo files capturing the build environment are signed and are made public, e.g. by them being covered by the release file, together with other meta data. in order to make the maintainers uploading a package accountable, a source package containing all maintainer keys is created and submitted into the archive. this constitutes the declaration by the archive, that these keys were authorized to upload for this release. the key ring is required to be append-only, where keys are marked with an expiry date instead of being removed. this allows verification of source packages submitted long ago, using the keys valid at the respective point in time. just as with ct, the log replies to each valid submission with a signed commitment, guaranteeing that it will shortly produce the signed root of a merkle tree that includes the submission: at release time, meta data and release file are submitted into the log as well. the log server produces a commitment for each submission, which constitutes its promise to include the submitted item into a future version of the tree. the log only accepts authenticated submissions from the archive. the commitment includes a timestamp, hash of the release file, log identifier and the log's signature over these. the archive should then verify that the log has produced a signed tree root that resolves the commitment. to complete the release, the archive publishes the commitments together with the updates. clients can then proceed with the verification of the release file. the log regularly produces signed merkle tree roots after receiving a valid inclusion request. the signed tree root produced by the log includes the merkle tree hash, tree size, timestamp, log identifier, and the log's signature. the client now obtains from the distribution mirror not just the release file, but also one or more inclusion commitments showing that the release file has been submitted to one or more of the logs trusted both by the distributor and the client: given the release file and inclusion commitment, the client can verify by hashing that the commitment belongs to this release file and also verify the signature. the client can now query the log, demanding a current tree root and an inclusion proof for this release file. per standard merkle tree proofs, the inclusion proof consists of a list of hashes to recompute the received root hash. for the received tree root, a consistency proof is demanded to a previous known tree root. the consistency proof is again a list of hashes. for the two given tree roots, it shows that the log only added items between them. clients store the signed tree root for the largest tree they have seen, to be used in any later consistency proofs. set aside split view attacks, which will be discussed later, clients verifying the log inclusion of the release file will detect targeted modifications of the release. like ct, in addition to logs their system includes auditors, typically integrated with clients, and independent monitors regularly checking the logs for anomalies. for details, you need to read the paper, but some idea can be gained from their description of how the system detects two kinds of attack: the hidden version attack the split view attack the hidden version attack hof and carle describe this attack thus: the hidden version attack attempts to hide a targeted backdoor by following correct signing and log submission procedures. it may require collusion by the archive and an authorized maintainer. the attacker prepares targeted malicious update to a package, say version v . . , and a clean update v . . . the archive presents the malicious package only to the victim when it wishes to update. the clean version v. . . will be presented to everybody immediately afterwards. a non-targeted user is unlikely to ever observe the backdoored version, thereby drawing a minimal amount of attention to it. the attack however leaves an audit trail in the log, so the update itself can be detected by auditing. a package maintainer monitoring uploads for their packages using the log would notice an additional version being published. a malicious package maintainer would however not alert the public when this happens. this could be construed as a targeted backdoor in violation of the stated security goals. it is true that the backdoored package would be in the logs, but that in and of itself does not indicate that it is malign: to mitigate this problem a minimum time between package updates can be introduced. this can be achieved by a fixing the issuance of release files and their log submission to a static frequency, or by alerting on quick subsequent updates to one package. there may be good reasons for releasing a new update shortly after its predecessor; for example a vulnerability might be discovered in the predecessor shortly after release. in the hidden version attack, the attacker increases a version number in order to get the victim to update a package. the victim will install this backdoored update. the monitor detects the hidden version attack due to the irregular release file publication. there are now two cases to be considered. the backdoor may be in the binary package, or it may be in the source package. the first case will be detected by monitors verifying the reproducible builds property. a monitor can rebuild all changed source packages on every update and check if the resulting binary matches. if not, the blame falls clearly on the archive, because the source does not correspond to the binary, which can be demonstrated by exploiting reproducible builds. the second case requires investigation of the packages modified by the update. the source code modifications can be investigated for the changed packages, because all source code is logged. the fact that source code can be analyzed, and no analysis on binaries is required, makes the investigation of the hidden version alert simpler. the blame for this case falls on the maintainer, who can be identified by their signature on the source package. if the upload was signed by a key not in the allowed set, the blame falls on the archive for failing to authorize correctly. if the package version numbers in the meta data are inconsistent, this constitutes a misbehavior by the submitting archive. it can easily be detected by a monitor. using the release file the monitor can also easily ensure, by demanding inclusion proofs, that all required files have been logged. note that although their system's monitors detect this attack, and can correctly attribute it, they do so asynchronously. they do not prevent the victim installing the backdoored update. the split view attack the logs cannot be assumed to be above suspicion. hof and carle describe a log-based attack: the most significant attack by the log or with the collusion of the log is equivocation. in a split-view or equivocation attack, a malicious log presents different versions of the merkle tree to the victim and to everybody else. each tree version is kept consistent in itself. the tree presented to the victim will include a leaf that is malicious in some way, such as an update with a backdoor. it might also omit a leaf in order to hide an update. this is a powerful attack within the threat model that violates the security goals and must therefore be defended. a defense against this attack requires the client to learn if they are served from the same tree as the others. their defense requires that their be multiple logs under independent administration, perhaps run by different linux distributions. each time a "committing" log generated a new tree root containing new package submissions, it would be required to submit a signed copy of the root to one or more "witness" logs under independent administration. the "committing" log will obtain commitments from the "witness" logs, and supply them to clients. clients can then verify that the root they obtain from the "committing" log matches that obtained directly from the "witness" logs: when the client now verifies a log entry with the committing log, it also has to verify that a tree root covering this entry was submitted into the witnessing log. additionally, the client verifies the append-only property of the witnessing log. the witnessing log introduces additional monitoring requirements. next to the usual monitoring of the append-only operation, we need to check that no equivocating tree roots are included. to this end, a monitor follows all new log entries of the witnessing log that are tree roots of the committing log. the monitor verifies that they are all valid extensions of the committing log's tree history. reproducible builds one weakness in hof and carle's actual implementation is in the connection between the signed package of source and the hashes of the result of compiling it. it is in general impossible to verify that the binaries are the result of compiling the source. in many cases, even if the source is re-compiled in the same environment the resulting binaries will not be bit-for-bit identical, and thus their hashes will differ. the differences have many causes, including timestamps, randomized file names, and so on. of course, changes in the build environment can also introduce differences. to enable binaries to be securely connected to their source, a reproducible builds effort has been under way for more than years. debian project lead chris lamb's -minute talk think you're not a target? a tale of developers ... provides an overview of the problem and the work to solve it using three example compromises: alice, a package developer who is blackmailed to distribute binaries that don't match the public source. bob, a build farm sysadmin whose personal computer has been compromised, leading to a compromised build toolchain in the build farm that inserts backdoors into the binaries. carol, a free software enthusiast who distributes binaries to friends. an evil maid attack has compromised her laptop. as lamb describes, eliminating all sources of irreproducibility from a package is a painstaking process because there are so many possibilities. they include non-deterministic behaviors such as iterating over hashmaps, parallel builds, timestamps, build paths, file system directory name order, and so on. the work started in with % of debian packages building reproducibly. currently, over % of the debian packages are now reproducible. that is good, but % coverage is really necessary to provide security. bootstrappable compilers one of the most famous of the acm's annual turing award lectures was ken thompson's reflections on trusting trust (also here). in , bruce schneier summarized its message thus: way back in , paul karger and roger schell discovered a devastating attack against computer systems. ken thompson described it in his classic speech, "reflections on trusting trust." basically, an attacker changes a compiler binary to produce malicious versions of some programs, including itself. once this is done, the attack perpetuates, essentially undetectably. thompson demonstrated the attack in a devastating way: he subverted a compiler of an experimental victim, allowing thompson to log in as root without using a password. the victim never noticed the attack, even when they disassembled the binaries -- the compiler rigged the disassembler, too. schneier was discussing david a. wheeler's countering trusting trust through diverse double-compiling. wheeler's subsequent work led to his ph.d. thesis. to oversimpify, his technique involves the suspect compiler compiling its source twice, and comparing the output to that from a "trusted" compiler compiling the same source twice. he writes: ddc uses a second “trusted” compiler ct, which is trusted in the sense that we have a justified confidence that ct does not have triggers or payloads there are two issues here. the first is an assumption that the suspect compiler's build is reproducible. the second is the issue of where the "justified confidence" comes from. this is the motivation for the bootstrappable builds project, whose goal is to create a process for building a complete toolchain starting from a "seed" binary that is simple enough to be certified "by inspection". one sub-project is stage : stage starts with just a byte hex monitor and builds up the infrastructure required to start some serious software development. with zero external dependencies, with the most painful work already done and real langauges such as assembly, forth and garbage collected lisp already implemented the current . . release of stage : marks the first c compiler hand written in assembly with structs, unions, inline assembly and the ability to self-host it's c version, which is also self-hosting there is clearly a long way still to go to a bootstrapped full toolchain. a more secure software supply chain a software supply chain based on apt enhanced with hof and carle's transparency layer, distributing packages reproducibly built with bootstrapped compilers, would be much more difficult to attack than current technology. users of the software could have much higher confidence that the binaries they installed had been built from the corresponding source, and that no attacker had introduced functionality not evident in the source. these checks would take place during software installation or update. users would still need to verify that the software had not been modified after installation, perhaps using a tripwire-like mechanism, but this mechanism would have a trustworthy source of the hashes it needs to do its job. remaining software problems despite all these enhancements, the event-stream attack would still have succeeded. the attackers targeted a widely-used, fairly old package that was still being maintained by the original author, a volunteer. they offered to take over what had become a burdensome task, and the offer was accepted. now, despite the fact that the attacker was just an e-mail address, they were the official maintainer of the package and could authorize changes. their changes, being authorized by the official package maintainer, would pass unimpeded through even the enhanced supply chain. first, it is important to observe the goal of hof and carle's system is to detect targeted attacks, those delivered to a (typically small) subset of user systems. the event-stream attack was not targeted; it was delivered to all systems updating the package irrespective of whether they contained the wallet to be compromised. that their system is designed only to detect targeted attacks seems to me to be a significant weakness. it is very easy to design an attack, like the event-stream one, that is broadcast to all systems but is harmless on all but the targets. second, hof and carle's system operates asynchronously, so is intended to detect rather than prevent victim compromise. of course, once the attack was detected it could be unambiguously attributed. but: the attack would already have succeeded in purloining cryptocurrency from the target wallets. this seems to me to be a second weakness; in many cases the malign package would only need to be resident on the victim for a short time to exfiltrate critical data, or install further malware providing persistence. strictly speaking, the attribution would be to a private key. more realistically, it would be to a key and an e-mail address. in the case of an attack, linking these to a human malefactor would likely be difficult, leaving the perpetrators free to mount further attacks. even if the maintainer had not, as in the event-stream attack, been replaced via social engineering, it is possible that their e-mail and private key could have been compromised. the event-stream attack can be thought of as the organization-level analog of a sybil attack on a peer-to-peer system. creating an e-mail identity is almost free. the defense against sybil attacks is to make maintaining and using an identity in the system expensive. as with proof-of-work in bitcoin, the idea is that the white hats will spend more (compute more useless hashes) than the black hats. even this has limits. eric budish's analysis shows that, if the potential gain from an attack on a blockchain is to be outweighed by its cost, the value of transactions in a block must be less than the block reward. would a similar defense against "sybil" attacks on the software supply chain be possible? there are a number of issues: the potential gains from such attacks are large, both because they can compromise very large numbers of systems quickly (event-stream had m downloads), and because the banking credentials, cryptocurrency wallets, and other data these systems contain can quickly be converted into large amounts of cash. thus the penalty for mounting an attack would have to be an even larger amount of cash. package maintainers would need to be bonded or insured for large sums, which implies that distributions and package libraries would need organizational structures capable of enforcing these requirements. bonding and insurance would be expensive for package maintainers, who are mostly unpaid volunteers. there would have to be a way of paying them for their efforts, at least enough to cover the costs of bonding and insurance. thus users of the packages would need to pay for their use, which means the packages could neither be free, nor open source. the foss (free open source software) movement will need to find other ways to combat sybil attacks, which will be hard if the reward for a successful attack greatly exceeds the cost of mounting it. how to adequately reward maintainers for their essential but under-appreciated efforts is a fundamental problem for foss. hof and carle's system shares one more difficulty with ct. both systems are layered on top of an existing infrastructure, respectively apt and tls with certificate authorities. in both cases there is a bootstrap problem, an assumption that as the system starts up there is not an attack already underway. in ct's case the communications between the ca's, web sites, logs, auditors and monitors all use the very tls infrastructure that is being secured (see here and here). this is also the case for hof and carle, plus they have to assume the lack of malware in the initial state of the packages. hardware supply chain problems all this effort to secure the software supply chain will be for naught if the hardware it runs on is compromised: much of what we think of as "hardware" contains software to which what we think of as "software" has no access or visibility. examples include intel's management engine, the baseband processor in mobile devices, complex i/o devices such as nics and gpus. even if this "firmware" is visible to the system cpu, it is likely supplied as a "binary blob" whose source code is inaccessible. attacks on the hardware supply chain have been in the news recently, with the firestorm of publicity sparked by bloomberg's, probably erroneous reports, of a chinese attack on supermicro motherboards that added "rice-grain" sized malign chips. the details will have to wait for a future post. posted by david. at : am labels: security comments: bryan newbold said... nit: in the last bullet point, i think you mean "bloomberg", not "motherboard". december , at : pm david. said... thanks for correcting my fused neurons, bryan! december , at : pm david. said... i really should have pointed out that this whole post is about software that is installed on your device. these days, much of the software that runs on your device is not installed, it is delivered via ad networks and runs inside your browser. as blissex wrote in this comment, we are living: "in an age in which every browser gifts a free-to-use, unlimited-usage, fast vm to every visited web site, and these vms can boot and run quite responsive d games or linux distributions" ad blockers, essential equipment in this age, merely reduce the incidence of malware delivered via ad networks. brannon dorsey's fascinating experiments in malvertising are described by cory doctorow thus: "anyone can make an account, create an ad with god-knows-what javascript in it, then pay to have the network serve that ad up to thousands of browser. ... within about three hours, his code (experimental, not malicious, apart from surreptitiously chewing up processing resources) was running on , web browsers, on , unique ip addresses. adtech, it turns out, is a superb vector for injecting malware around the planet. some other fun details: dorsey found that when people loaded his ad, they left the tab open an average of minutes. that gave him huge amounts of compute time -- full days, in fact, for about $ in ad purchase." december , at : pm david. said... i regret not citing john leyden's open-source software supply chain vulns have doubled in months to illustrate the scope of the problem: "miscreants have even started to inject (or mainline) vulnerabilities directly into open source projects, according to sonatype, which cited recent examples of this type of malfeasance in its study. el reg has reported on several such incidents including a code hack on open-source utility eslint-scope back in july." and: "organisations are still downloading vulnerable versions of the apache struts framework at much the same rate as before the equifax data breach, at around , downloads per month. downloads of buggy versions of another popular web application framework called spring were also little changed since a september vulnerability, sonatype added. the , average in september has declined only per cent to , over the last months." december , at : am david. said... catalin cimpanu's users report losing bitcoin in clever hack of electrum wallets describes a software supply chain attack that started around st december and netted around $ k "worth" of btc. december , at : am david. said... popular wordpress plugin hacked by angry former employee is like the event-stream hack in that no amount of transparency would have prevented it. the disgruntled perpetrator apparently had valid credentials for the official source of the software: "the plugin in question is wpml (or wp multilingual), the most popular wordpress plugin for translating and serving wordpress sites in multiple languages. according to its website, wpml has over , paying customers and is one of the very few wordpress plugins that is so reputable that it doesn't need to advertise itself with a free version on the official wordpress.org plugins repository." january , at : am david. said... the fourth annual report for the national security adviser from the huawei cyber security evaluation centre oversight board in the uk is interesting. the centre has access to the source code for huawei products, and is working with huawei to make the builds reproducible: " . hcsec have worked with huawei r&d to try to correct the deficiencies in the underlying build and compilation process for these four products. this has taken significant effort from all sides and has resulted in a single product that can be built repeatedly from source to the general availability (ga) version as distributed. this particular build has yet to be deployed by any uk operator, but we expect deployment by uk operators in the future, as part of their normal network release cycle. the remaining three products from the pilot are expected to be made commercially available in h , with each having reproducible binaries." january , at : am david. said... huawei says fixing "the deficiencies in the underlying build and compilation process" in its carrier products will take five years. february , at : pm david. said... in cyber-mercenary groups shouldn't be trusted in your browser or anywhere else, the eff's cooper quintin describes the latest example showing why certificate authorities can't be trusted: "darkmatter, the notorious cyber-mercenary firm based in the united arab emirates, is seeking to become approved as a top-level certificate authority in mozilla’s root certificate program. giving such a trusted position to this company would be a very bad idea. darkmatter has a business interest in subverting encryption, and would be able to potentially decrypt any https traffic they intercepted. one of the things https is good at is protecting your private communications from snooping governments—and when governments want to snoop, they regularly hire darkmatter to do their dirty work. ... darkmatter was already given an "intermediate" certificate by another company, called quovadis, now owned by digicert. that's bad enough, but the "intermediate" authority at least comes with ostensible oversight by digicert." hat tip to cory doctorow. february , at : pm david. said... gareth corfield's just android things: m phones, gadgets installed 'adware-ridden' mobe simulator games reports on a very successful software supply chain attack: "android adware found its way into as many as million devices – after it was stashed inside a large number of those bizarre viral mundane job simulation games, we're told. ... although researchers believed that the titles were legitimate, they said they thought the devs were “scammed” into using a “malicious sdk, unaware of its content, leading to the fact that this campaign was not targeting a specific country or developed by the same developer.” march , at : am david. said... kim zetter's hackers hijacked asus software updates to install backdoors on thousands of computers is an excellent example of a software supply chain attack: "researchers at cybersecurity firm kaspersky lab say that asus, one of the world’s largest computer makers, was used to unwittingly install a malicious backdoor on thousands of its customers’ computers last year after attackers compromised a server for the company’s live software update tool. the malicious file was signed with legitimate asus digital certificates to make it appear to be an authentic software update from the company, kaspersky lab says." march , at : am david. said... sean gallagher's uk cyber security officials report huawei’s security practices are a mess reports on the latest report from the hcsec oversight board. they still can't do reproducible builds: "hcsec reported that the software build process used by huawei results in inconsistencies between software images. in other words, products ship with software with widely varying fingerprints, so it’s impossible to determine whether the code is the same based on checksums." which isn't a surprise, huawei already said it'd take another years. but i'd be more concerned that: "one major problem cited by the report is that a large portion of huawei’s network gear still relies on version . of wind river’s vxworks real-time operating system (rtos), which has reached its “end of life” and will soon no longer be supported. huawei has bought a premium long-term support license from vxworks, but that support runs out in ." and huawei is rolling its own rtos based on linux. what could possibly go wrong? march , at : pm david. said... the latest software supply chain attack victim is bootstrap-sass via rubygems, with about m downloads. april , at : am david. said... it turns out that shadowhammer targets multiple companies, asus just one of them: "asus was not the only company targeted by supply-chain attacks during the shadowhammer hacking operation as discovered by kaspersky, with at least six other organizations having been infiltrated by the attackers. as further found out by kaspersky's security researchers, asus' supply chain was successfully compromised by trojanizing one of the company's notebook software updaters named asus live updater which eventually was downloaded and installed on the computers of tens of thousands of customers according to experts' estimations." april , at : pm david. said... who owns huawei? by christopher balding and donald c. clarke concludes that: "huawei calls itself “employee-owned,” but this claim is questionable, and the corporate structure described on its website is misleading." april , at : pm david. said... david a. wheeler reports on another not-very-successful software supply chain attack: "a malicious backdoor has been found in the popular open source software library bootstrap-sass. this was done by someone who created an unauthorized updated version of the software on the rubygems software hosting site. the good news is that it was quickly detected (within the day) and updated, and that limited the impact of this subversion. the backdoored version ( . . . ) was only downloaded , times. for comparison, as of april the previous version in that branch ( . . . ) was downloaded . million times, and the following version . . . (which duplicated . . . ) was downloaded , times (that’s more than the subverted version!). so it is likely that almost all subverted systems have already been fixed." wheeler has three lessons from this: . maintainers need fa. . don't update your dependencies in the same day they're released. . reproducible builds! may , at : am david. said... andy greenberg's a mysterious hacker gang is on a supply-chain hacking spree ties various software supply chain attacks together and attributes them: "over the past three years, supply-chain attacks that exploited the software distribution channels of at least six different companies have now all been tied to a single group of likely chinese-speaking hackers. the group is known as barium, or sometimes shadowhammer, shadowpad, or wicked panda, depending on which security firm you ask. more than perhaps any other known hacker team, barium appears to use supply-chain attacks as its core tool. its attacks all follow a similar pattern: seed out infections to a massive collection of victims, then sort through them to find espionage targets." may , at : am david. said... someone is spamming and breaking a core component of pgp’s ecosystem by lorenzo franceschi-bicchierai reports on an attack on two of the core pgp developers,robert j. hansen and daniel kahn gillmor : "last week, contributors to the pgp protocol gnupg noticed that someone was “poisoning” or “flooding” their certificates. in this case, poisoning refers to an attack where someone spams a certificate with a large number of signatures or certifications. this makes it impossible for the the pgp software that people use to verify its authenticity, which can make the software unusable or break. in practice, according to one of the gnupg developers targeted by this attack, the hackers could make it impossible for people using linux to download updates, which are verified via pgp." the problem lies in the sks keyserver: "the sks software was written in an obscure language by a phd student for his thesis. and because of that, according to hansen, “there is literally no one in the keyserver community who feels qualified to do a serious overhaul on the codebase.” in other words, these attacks are here to stay." july , at : pm david. said... dan goodin's the year-long rash of supply chain attacks against open source is getting worse is a useful overview of the recent incidents pointing to the need for verifiable logs and reproducible builds. and, of course, for requiring developers to use multi--factor authentication. august , at : pm david. said... catalin cimpanu's hacking high-profile dev accounts could compromise half of the npm ecosystem is based on small world with high risks:a study of security threats in the npm ecosystem by marcus zimmerman et al: "their goal was to get an idea of how hacking one or more npm maintainer accounts, or how vulnerabilities in one or more packages, reverberated across the npm ecosystem; along with the critical mass needed to cause security incidents inside tens of thousands of npm projects at a time. ... the normal npm javascript package has an abnormally large number of dependencies -- with a package loading third-party packages from different maintainers, on average. this number is lower for popular packages, which only rely on code from other maintainers, on average, but the research team found that some popular npm packages ( ) relied on code written by more than maintainers. ... " highly influential maintainers affect more than , packages, making them prime targets for attacks," the research team said. "if an attacker manages to compromise the account of any of the most influential maintainers, the community will experience a serious security incident." furthermore, in a worst-case scenario where multiple maintainers collude, or a hacker gains access to a large number of accounts, the darmstadt team said that it only takes access to popular npm maintainer accounts to deploy malicious code impacting more than half of the npm ecosystem." october , at : am david. said... five years after the equation group hdd hacks, firmware security still sucks by catalin cimpanu illustrates how far disk drive firmware security is ahead of the rest of the device firmware world: "in , security researchers from kaspersky discovered a novel type of malware that nobody else had seen before until then. the malware, known as nls_ .dll, had the ability to rewrite hdd firmware for a dozen of hdd brands to plant persistent backdoors. kaspersky said the malware was used in attacks against systems all over the world. kaspersky researchers claimed the malware was developed by a hacker group known as the equation group, a codename that was later associated with the us national security agency (nsa). knowing that the nsa was spying on their customers led many hdd and ssd vendors to improve the security of their firmware, eclypsium said. however, five years since the equation group's hdd implants were found in the wild and introduced the hardware industry to the power of firmware hacking, eclypsium says vendors have only partially addressed this problem. "after the disclosure of the equation group's drive implants, many hdd and ssd vendors made changes to ensure their components would only accept valid firmware. however, many of the other peripheral components have yet to follow suit," researchers said." february , at : pm david. said... marc ohm et al analyze supply chain attacks via open source packages in three reposiotries in backstabber’s knife collection: a review of open source software supply chain attacks: "this paper presents a dataset of malicious software packages that were used in real-world attacks on open source software supply chains,and which were distributed via the popular package repositories npm, pypi, and rubygems. those packages, dating from november to november , were manually collected and analyzed. the paper also presents two general attack trees to provide a structured overview about techniques to inject malicious code into the dependency tree of downstream users, and to execute such code at different times and under different conditions." may , at : am david. said... bruce schneier's survey of supply chain attacks starts: "the atlantic council has a released a report that looks at the history of computer supply chain attacks." the atlantic council also has a summary of the report entitled breaking trust: shades of crisis across an insecure software supply chain: "software supply chain security remains an under-appreciated domain of national security policymaking. working to improve the security of software supporting private sector enterprise as well as sensitive defense and intelligence organizations requires more coherent policy response together industry and open source communities. this report profiles attacks and disclosures against the software supply chain from the past decade to highlight the need for action and presents recommendations to both raise the cost of these attacks and limit their harm." july , at : pm david. said... via my friend jim gettys, we learn of a major milestone in the development of a truly reproducible build environment. last june jan nieuwenhuizen posted guix further reduces bootstrap seed to %. the tl;dr is: "gnu mes is closely related to the bootstrappable builds project. mes aims to create an entirely source-based bootstrapping path for the guix system and other interested gnu/linux distributions. the goal is to start from a minimal, easily inspectable binary (which should be readable as source) and bootstrap into something close to r rs scheme. currently, mes consists of a mutual self-hosting scheme interpreter and c compiler. it also implements a c library. mes, the scheme interpreter, is written in about , lines of code of simple c. mescc, the c compiler, is written in scheme. together, mes and mescc can compile a lightly patched tinycc that is self-hosting. using this tinycc and the mes c library, it is possible to bootstrap the entire guix system for i -linux and x _ -linux." the binary they plan to start from is: "our next target will be a third reduction by ~ %; the full-source bootstrap will replace the mescc-tools and gnu mes binaries by stage and m -planet. the stage project by jeremiah orians starts everything from ~ bytes; virtually nothing. have a look at this incredible project if you haven’t already done so." in mid november nieuwenhuizen tweeted: "we just compiled the first working program using a reduced binary seed bootstrap'ped*) tinycc for arm" and on december he tweeted: "the reduced binary seed bootstrap is coming to arm: tiny c builds on @guixhpc wip-arm-bootstrap branch" starting from a working tinycc, you can build the current compiler chain. december , at : pm post a comment newer post older post home subscribe to: post comments (atom) blog rules posts and comments are copyright of their respective authors who, by posting or commenting, license their work under a creative commons attribution-share alike . united states license. off-topic or unsuitable comments will be deleted. dshr dshr in anwr recent comments full comments blog archive ► ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ▼ ( ) ▼ december ( ) securing the hardware supply chain meta: impending blog slowdown securing the software supply chain software preservation network blockchain: what's not to like? irina bolychevsky on solid selective amnesia ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► july ( ) ► june ( ) ► february ( ) ► ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) lockss system has permission to collect, preserve, and serve this archival unit. simple theme. powered by blogger. none none blog.cbeer.info blog.cbeer.info autoscaling aws elastic beanstalk worker tier based on sqs queue length ldpath in examples building a pivotal tracker irc bot with sinatra and cinch real-time statistics with graphite, statsd, and gdash icemelt: a stand-in for integration tests against aws glacier none archival face recognition for fun and nonprofit – andromeda yelton skip to content andromeda yelton menu home about contact resume hamlet lita talks machine learning (ala midwinter ) boston python meetup (august , ) swib libtechconf code lib keynote texas library association online northwest : five conversations about code new jersey esummit (may , ) westchester library association (january , ) bridging the digital divide with mobile services (webjunction, july ) archival face recognition for fun and nonprofit andromeda uncategorized february , in , dominique luster gave a super good code lib talk about applying ai to metadata for the charles “teenie” harris collection at the carnegie museum of art — more than , photographs of black life in pittsburgh. they experimented with solutions to various metadata problems, but the one that’s stuck in my head since is the face recognition one. it sure would be cool if you could throw ai at your digitized archival photos to find all the instances of the same person, right? or automatically label them, given that any of them are labeled correctly? sadly, because we cannot have nice things, the data sets used for pretrained face recognition embeddings are things like lots of modern photos of celebrities, a corpus which wildly underrepresents ) archival photos and ) black people. so the results of the face recognition process are not all that great. i have some extremely technical ideas for how to improve this — ideas which, weirdly, some computer science phds i’ve spoken with haven’t seen in the field. so i would like to experiment with them. but i must first invent the universe set up a data processing pipeline. three steps here: fetch archival photographs; do face detection (draw bounding boxes around faces and crop them out for use in the next step); do face recognition. for step , i’m using dpla, which has a super straightforward and well-documented api and an easy-to-use python wrapper (which, despite not having been updated in a while, works just fine with python . , the latest version compatible with some of my dependencies). for step , i’m using mtcnn, because i’ve been following this tutorial. for step , face recognition, i’m using the steps in the same tutorial, but purely for proof-of-concept — the results are garbage because archival photos from mid-century don’t actually look anything like modern-day celebrities. (neural net: “i have % confidence this is stevie wonder!” how nice for you.) clearly i’m going to need to build my own corpus of people, which i have a plan for (i.e. i spent some quality time thinking about numpy) but haven’t yet implemented. so far the gotchas have been: gotcha : if you fetch a page from the api and assume you can treat its contents as an image, you will be sad. you have to treat them as a raw data stream and interpret that as an image, thusly: from pil import image import requests response = requests.get(url, stream=true) response.raw.decode_content = true data = requests.get(url).content image.open(io.bytesio(data)) this code is, of course, hilariously lacking in error handling, despite fetching content from a cesspool of untrustworthiness, aka the internet. it’s a first draft. gotcha : you see code snippets to convert images to pixel arrays (suitable for ai ingestion) that look kinda like this: np.array(image).astype('uint '). except they say astype('float ') instead of astype('uint '). i got a creepy photonegative effect when i used floats. gotcha : although pil was happy to manipulate the .pngs fetched from the api, it was not happy to write them to disk; i needed to convert formats first (image.convert('rgb')). gotcha : the suggested keras_vggface library doesn’t have a pipfile or requirements.txt, so i had to manually install keras and tensorflow. luckily the setup.py documented the correct versions. sadly the tensorflow version is only compatible with python up to . (hence the comment about dpyla compatibility above). i don’t love this, but it got me up and running, and it seems like an easy enough part of the pipeline to rip out and replace if it’s bugging me too much. the plan from here, not entirely in order, subject to change as i don’t entirely know what i’m doing until after i’ve done it: build my own corpus of identified people this means the numpy thoughts, above it also means spending more quality time with the api to see if i can automatically apply names from photo metadata rather than having to spend too much of my own time manually labeling the corpus decide how much metadata i need to pull down in my data pipeline and how to store it figure out some kind of benchmark and measure it try out my idea for improving recognition accuracy benchmark again hopefully celebrate awesomeness share this: twitter facebook like this: like loading... tagged archival photos face recognition fridai published by andromeda romantic analytical technologist librarian. view all posts by andromeda published february , post navigation previous post sequence models of language: slightly irksome leave a reply cancel reply enter your comment here... fill in your details below or click an icon to log in: email (required) (address never made public) name (required) website you are commenting using your wordpress.com account. ( log out / change ) you are commenting using your google account. ( log out / change ) you are commenting using your twitter account. ( log out / change ) you are commenting using your facebook account. ( log out / change ) cancel connecting to %s notify me of new comments via email. notify me of new posts via email. create a free website or blog at wordpress.com. privacy & cookies: this site uses cookies. by continuing to use this website, you agree to their use. to find out more, including how to control cookies, see here: cookie policy %d bloggers like this: bibliographic wilderness skip to content bibliographic wilderness menu about contact product management in my career working in the academic sector, i have realized that one thing that is often missing from in-house software development is “product management.” but what does that mean exactly? you don’t know it’s missing if you don’t even realize it’s a thing and people can use different terms to mean different roles/responsibilities. basically, deciding what the software should do. this is not about colors on screen or margins (what our stakeholderes often enjoy micro-managing) — i’d consider those still the how of doing it, rather than the what to do. the what is often at a much higher level, about what features or components to develop at all. when done right, it is going to be based on both knowledge of the end-user’s needs and preferences (user research); but also knowledge of internal stakeholder’s desires and preferences (overall organiational strategy, but also just practically what is going to make the right people happy to keep us resourced). also knowledge of the local capacity, what pieces do we need to put in place to get these things developed. when done seriously, it will necessarily involve prioritization — there are many things we could possibly done, some subset of them we very well may do eventually, but which ones should we do now? my experience tells me it is a very big mistake to try to have a developer doing this kind of product management. not because a developer can’t have the right skillset to do them. but because having the same person leading development and product management is a mistake. the developer is too close to the development lense, and there’s just a clarification that happens when these roles are separate. my experience also tells me that it’s a mistake to have a committee doing these things, much as that is popular in the academic sector. because, well, just of course it is. but okay this is all still pretty abstract. things might become more clear if we get more specific about the actual tasks and work of this kind of product management role. i found damilola ajiboye blog post on “product manager vs product marketing manager vs product owner” very clear and helpful here. while it is written so as to distinguish between three different product management related roles, but ajiboye also acknowledges that in a smaller organization “a product manager is often tasked with the duty of these roles. regardless of if the responsibilities are to be done by one or two or three person, ajiboye’s post serves as a concise listing of the work to be done in managing a product — deciding the what of the product, in an ongoing iterative and collaborative manner, so that developers and designers can get to the how and to implementation. i recommend reading the whole article, and i’ll excerpt much of it here, slightly rearranged. the product manager these individuals are often referred to as mini ceos of a product. they conduct customer surveys to figure out the customer’s pain and build solutions to address it. the pm also prioritizes what features are to be built next and prepares and manages a cohesive and digital product roadmap and strategy. the product manager will interface with the users through user interviews/feedback surveys or other means to hear directly from the users. they will come up with hypotheses alongside the team and validate them through prototyping and user testing. they will then create a strategy on the feature and align the team and stakeholders around it. the pm who is also the chief custodian of the entire product roadmap will, therefore, be tasked with the duty of prioritization. before going ahead to carry out research and strategy, they will have to convince the stakeholders if it is a good choice to build the feature in context at that particular time or wait a bit longer based on the content of the roadmap. the product marketing manager the pmm communicates vital product value — the “why”, “what” and “when” of a product to intending buyers. he manages the go-to-market strategy/roadmap and also oversees the pricing model of the product. the primary goal of a pmm is to create demand for the products through effective messaging and marketing programs so that the product has a shorter sales cycle and higher revenue. the product marketing manager is tasked with market feasibility and discovering if the features being built align with the company’s sales and revenue plan for the period. they also make research on how sought-after the feature is being anticipated and how it will impact the budget. they communicate the values of the feature; the why, what, and when to potential buyers — in this case users in countries with poor internet connection. [while expressed in terms of a for-profit enterprise selling something, i think it’s not hard to translate this to a non-profit or academic environment. you still have an audience whose uptake you need to be succesful, whether internal or external. — jrochkind ] the product owner a product owner (po) maximizes the value of a product through the creation and management of the product backlog, creation of user stories for the development team. the product owner is the customer’s representative to the development team. he addresses customer’s pain points by managing and prioritizing a visible product backlog. the po is the first point of call when the development team needs clarity about interpreting a product feature to be implemented. the product owner will first have to prioritize the backlog to see if there are no important tasks to be executed and if this new feature is worth leaving whatever is being built currently. they will also consider the development effort required to build the feature i.e the time, tools, and skill set that will be required. they will be the one to tell if the expertise of the current developers is enough or if more engineers or designers are needed to be able to deliver at the scheduled time. the product owner is also armed with the task of interpreting the product/feature requirements for the development team. they serve as the interface between the stakeholders and the development team. when you have someone(s) doing these roles well, it ensures that the development team is actually spending time on things that meet user and business needs. i have found that it makes things so much less stressful and more rewarding for everyone involved. when you have nobody doing these roles, or someone doing it in a cursory or un-intentional way not recognized as part of their core job responsibilities, or have a lead developer trying to do it on top of develvopment, i find it leads to feelings of: spinning wheels, everything-is-an-emergency, lack of appreciation, miscommunication and lack of shared understanding between stakeholders and developers, general burnout and dissatisfaction — and at the root, a product that is not meeting user or business needs well, leading to these inter-personal and personal problems. jrochkind general leave a comment february , rails auto-scaling on heroku we are investigating moving our medium-small-ish rails app to heroku. we looked at both the rails autoscale add-on available on heroku marketplace, and the hirefire.io service which is not listed on heroku marketplace and i almost didn’t realize it existed. i guess hirefire.io doesn’t have any kind of a partnership with heroku, but still uses the heroku api to provide an autoscale service. hirefire.io ended up looking more fully-featured and lesser priced than rails autoscale; so the main service of this post is just trying to increase visibility of hirefire.io and therefore competition in the field, which benefits us consumers. background: interest in auto-scaling rails background jobs at first i didn’t realize there was such a thing as “auto-scaling” on heroku, but once i did, i realized it could indeed save us lots of money. i am more interested in scaling rails background workers than i a web workers though — our background workers are busiest when we are doing “ingests” into our digital collections/digital asset management system, so the work is highly variable. auto-scaling up to more when there is ingest work piling up can give us really nice inget throughput while keeping costs low. on the other hand, our web traffic is fairly low and probably isn’t going to go up by an order of magnitude (non-profit cultural institution here). and after discovering that a “standard” dyno is just too slow, we will likely be running a performance-m or performance-l anyway — which likely can handle all anticipated traffic on it’s own. if we have an auto-scaling solution, we might configure it for web dynos, but we are especially interested in good features for background scaling. there is a heroku built-in autoscale feature, but it only works for performance dynos, and won’t do anything for rails background job dynos, so that was right out. that could work for rails bg jobs, the rails autoscale add-on on the heroku marketplace; and then we found hirefire.io. pricing: pretty different hirefire as of now january , hirefire.io has pretty simple and affordable pricing. $ /month/heroku application. auto-scaling as many dynos and process types as you like. hirefire.io by default can only check into your apps metrics to decide if a scaling event can occur once per minute. if you want more frequent than that (up to once every seconds), you have to pay an additional $ /month, for $ /month/heroku application. even though it is not a heroku add-on, hirefire does advertise that they bill pro-rated to the second, just like heroku and heroku add-ons. rails autoscale rails autoscale has a more tiered approach to pricing that is based on number and type of dynos you are scaling. starting at $ /month for - standard dynos, the next tier up is $ for up to standard dynos, all the way up to $ (!) for to dynos. if you have performance dynos involved, from $ /month for - performance dynos, up to $ /month for up to performance dynos. for our anticipated uses… if we only scale bg dynos, i might want to scale from (low) or to (high) or standard dynos, so we’d be at $ /month. our web dynos are likely to be performance and i wouldn’t want/need to scale more than probably , but that puts us into performance dyno tier, so we’re looking at $ /month. this is of course significantly more expensive than hirefire.io’s flat rate. metric resolution since hirefire had an additional charge for finer than -minute resolution on checks for autoscaling, we’ll discuss resolution here in this section too. rails autoscale has same resolution for all tiers, and i think it’s generally seconds, so approximately the same as hirefire if you pay the extra $ for increased resolution. configuration let’s look at configuration screens to get a sense of feature-sets. rails autoscale web dynos to configure web dynos, here’s what you get, with default values: the metric rails autoscale uses for scaling web dynos is time in heroku routing queue, which seems right to me — when things are spending longer in heroku routing queue before getting to a dyno, it means scale up. worker dynos for scaling worker dynos, rails autoscale can scale dyno type named “worker” — it can understand ruby queuing libraries sidekiq, resque, delayed job, or que. i’m not certain if there are options for writing custom adapter code for other backends. here’s what the configuration options are — sorry these aren’t the defaults, i’ve already customized them and lost track of what defaults are. you can see that worker dynos are scaled based on the metric “number of jobs queued”, and you can tell it to only pay attention to certain queues if you want. hirefire hirefire has far more options for customization than rails autoscale, which can make it a bit overwhelming, but also potentially more powerful. web dynos you can actually configure as many heroku process types as you have for autoscale, not just ones named “web” and “worker”. and for each, you have your choice of several metrics to be used as scaling triggers. for web, i think queue time (percentile, average) matches what rails autoscale does, configured to percentile, , and is probably the best to use unless you have a reason to use another. (“rails autoscale tracks the th percentile queue time, which for most applications will hover well below the default threshold of ms.“) here’s what configuration hirefire makes available if you are scaling on “queue time” like rails autoscale, configuration may vary for other metrics. i think if you fill in the right numbers, you can configure to work equivalently to rails autoscale. worker dynos if you have more than one heroku process type for workers — say, working on different queues — hirefire can scale the independently, with entirely separate configuration. this is pretty handy, and i don’t think rails autoscale offers this. (update i may be wrong, rails autoscale says they do support this, so check on it yourself if it matters to you). for worker dynos, you could choose to scale based on actual “dyno load”, but i think this is probably mostly for types of processes where there isn’t the ability to look at “number of jobs”. a “number of jobs in queue” like rails autoscale does makes a lot more sense to me as an effective metric for scaling queue-based bg workers. hirefire’s metric is slightly difererent than rails autoscale’s “jobs in queue”. for recognized ruby queue systems (a larger list than rails autoscale’s; and you can write your own custom adapter for whatever you like), it actually measures jobs in queue plus workers currently busy. so queued+in-progress, rather than rails autoscale’s just queued. i actually have a bit of trouble wrapping my head around the implications of this, but basically, it means that hirefire’s “jobs in queue” metric strategy is intended to try to scale all the way to emptying your queue, or reaching your max scale limit, whichever comes first. i think this may make sense and work out at least as well or perhaps better than rails autoscale’s approach? here’s what configuration hirefire makes available for worker dynos scaling on “job queue” metric. since the metric isn’t the same as rails autosale, we can’t configure this to work identically. but there are a whole bunch of configuration options, some similar to rails autoscale’s. the most important thing here is that “ratio” configuration. it may not be obvious, but with the way the hirefire metric works, you are basically meant to configure this to equal the number of workers/threads you have on each dyno. i have it configured to because my heroku worker processes use resque, with resque_pool, configured to run resque workers on each dyno. if you use sidekiq, set ratio to your configured concurrency — or if you are running more than one sidekiq process, processes*concurrency. basically how many jobs your dyno can be concurrently working is what you should normally set for ‘ratio’. hirefire not a heroku plugin hirefire isn’t actually a heroku plugin. in addition to that meaning separate invoicing, there can be some other inconveniences. since hirefire only can interact with heroku api, for some metrics (including the “queue time” metric that is probably optimal for web dyno scaling) you have to configure your app to log regular statistics to heroku’s “logplex” system. this can add a lot of noise to your log, and for heroku logging add-ons that are tired based on number of log lines or bytes, can push you up to higher pricing tiers. if you use paperclip, i think you should be able to use the log filtering feature to solve this, keep that noise out of your logs and avoid impacting data log transfer limits. however, if you ever have cause to look at heroku’s raw logs, that noise will still be there. support and docs i asked a couple questions of both hirefire and rails autoscale as part of my evaluation, and got back well-informed and easy-to-understand answers quickly from both. support for both seems to be great. i would say the documentation is decent-but-not-exhaustive for both products. hirefire may have slightly more complete documentation. other features? there are other things you might want to compare, various kinds of observability (bar chart or graph of dynos or observed metrics) and notification. i don’t have time to get into the details (and didn’t actually spend much time exploring them to evaluate), but they seem to offer roughly similar features. conclusion rails autoscale is quite a bit more expensive than hirefire.io’s flat rate, once you get past rails autoscale’s most basic tier (scaling no more than standard dynos). it’s true that autoscaling saves you money over not, so even an expensive price could be considered a ‘cut’ of that, and possibly for many ecommerce sites even $ a month might a drop in the bucket (!)…. but this price difference is so significant with hirefire (which has flat rate regardless of dynos), that it seems to me it would take a lot of additional features/value to justify. and it’s not clear that rails autoscale has any feature advantage. in general, hirefire.io seems to have more features and flexibility. until , hirefire.io could only analyze metrics with -minute resolution, so perhaps that was a “killer feature”? honestly i wonder if this price difference is sustained by rails autoscale only because most customers aren’t aware of hirefire.io, it not being listed on the heroku marketplace? single-invoice billing is handy, but probably not worth $ + a month. i guess hirefire’s logplex noise is a bit inconvenient? or is there something else i’m missing? pricing competition is good for the consumer. and are there any other heroku autoscale solutions, that can handle rails bg job dynos, that i still don’t know about? update a day after writing djcp on a reddit thread writes: i used to be a principal engineer for the heroku add-ons program. one issue with hirefire is they request account level oauth tokens that essentially give them ability to do anything with your apps, where rails autoscaling worked with us to create a partnership and integrate with our “official” add-on apis that limits security concerns and are scoped to the application that’s being scaled. part of the reason for hirefire working the way it does is historical, but we’ve supported the endpoints they need to scale for “official” partners for years now. a lot of heroku customers use hirefire so please don’t think i’m spreading fud, but you should be aware you’re giving a third party very broad rights to do things to your apps. they probably won’t, of course, but what if there’s a compromise? “official” add-on providers are given limited scoped tokens to (mostly) only the actions / endpoints they need, minimizing blast radius if they do get compromised. you can read some more discussion at that thread. jrochkind general comments january , january , managed solr saas options i was recently looking for managed solr “software-as-a-service” (saas) options, and had trouble figuring out what was out there. so i figured i’d share what i learned. even though my knowledge here is far from exhaustive, and i have only looked seriously at one of the ones i found. the only managed solr options i found were: websolr; searchstax; and opensolr. of these, i think websolr and searchstax are more well-known, i couldn’t find anyone with experience with opensolr, which perhaps is newer. of them all, searchstax is the only one i actually took for a test drive, so will have the most to say about. why we were looking we run a fairly small-scale app, whose infrastructure is currently self-managed aws ec instances, running respectively: ) a rails web app ) bg workers for the rails web app ) postgres, and ) solr. oh yeah, there’s also a redis running one of those servers, on # with pg or # with solr, i forget. currently we manage this all ourselves, right on the ec . but we’re looking to move as much as we can into “managed” servers. perhaps we’ll move to heroku. perhaps we’ll use hatchbox. or if we do stay on aws resources we manage directly, we’d look at things like using an aws rds postgres instead of installing it on an ec ourselves, an aws elasticache for redis, maybe look into elastic beanstalk, etc. but no matter what we do, we need a solr, and we’d like to get it managed. hatchbox has no special solr support, aws doesn’t have a solr service, heroku does have a solr add-on but you can also use any solr with it and we’ll get to that later. our current solr use is pretty small scale. we don’t run “solrcloud mode“, just legacy ordinary solr. we only have around , documents in there (tiny for solr), our index size is only mb. our traffic is pretty low — when i tried to figure out how low, it doesn’t seem we have sufficient logging turned on to answer that specifically but using proxy metrics to guess i’d say k- k requests a day, query as well as add. this is a pretty small solr installation, although it is used centrally for the primary functions of the (fairly low-traffic) app. it currently runs on an ec t a.small, which is a “burstable” ec type with only g of ram. it does have two vcpus (that is one core with ‘hyperthreading’). the t a.small ec instance only costs $ /month on-demand price! we know we’ll be paying more for managed solr, but we want to do get out of the business of managing servers — we no longer really have the staff for it. websolr (didn’t actually try out) websolr is the only managed solr currently listed as a heroku add-on. it is also available as a managed solr independent of heroku. the pricing in the heroku plans vs the independent plans seems about the same. as a heroku add-on there is a $ “staging” plan that doesn’t exist in the independent plans. (unlike some other heroku add-ons, no time-limited free plan is available for websolr). but once we go up from there, the plans seem to line up. starting at: $ /month for: million document limit k requests/day index mb storage concurrent requests limit (this limit is not mentioned on the independent pricing page?) next level up is $ /month for: million document limit k requests/day . gb storage concurrent request limit (again concurrent request limits aren’t mentioned on independent pricing page) as you can see, websolr has their plans metered by usage. $ /month is around the price range we were hoping for (we’ll need two, one for staging one for production). our small solr is well under million documents and ~ gb storage, and we do only use one index at present. however, the k requests/day limit i’m not sure about, even if we fit under it, we might be pushing up against it. and the “concurrent request” limit simply isn’t one i’m even used to thinking about. on a self-managed solr it hasn’t really come up. what does “concurrent” mean exactly in this case, how is it measured? with puma web workers and sometimes a possibly multi-threaded batch index going on, could we exceed a limit of ? seems plausible. what happens when they are exceeded? your solr request results in an http error! do i need to now write the app to rescue those gracefully, or use connection pooling to try to avoid them, or something? having to rewrite the way our app functions for a particular managed solr is the last thing we want to do. (although it’s not entirely clear if those connection limits exist on the non-heroku-plugin plans, i suspect they do?). and in general, i’m not thrilled with the way the pricing works here, and the price points. i am positive for a lot of (eg) heroku customers an additional $ * =$ /month is peanuts not even worth accounting for, but for us, a small non-profit whose app’s traffic does not scale with revenue, that starts to be real money. it is not clear to me if websolr installations (at “standard” plans) are set up in “solrcloud mode” or not; i’m not sure what api’s exist for uploading your custom schema.xml (which we’d need to do), or if they expect you to do this only manually through a web ui (that would not be good); i’m not sure if you can upload custom solrconfig.xml settings (this may be running on a shared solr instance with standard solrconfig.xml?). basically, all of this made websolr not the first one we looked at. does it matter if we’re on heroku using a managed solr that’s not a heroku plugin? i don’t think so. in some cases, you can get a better price from a heroku plug-in than you could get from that same vendor not on heroku or other competitors. but that doesn’t seem to be the case here, and other that that does it matter? well, all heroku plug-ins are required to bill you by-the-minute, which is nice but not really crucial, other forms of billing could also be okay at the right price. with a heroku add-on, your billing is combined into one heroku invoice, no need to give a credit card to anyone else, and it can be tracked using heroku tools. which is certainly convenient and a plus, but not essential if the best tool for the job is not a heroku add-on. and as a heroku add-on, websolr provides a websolr_url heroku config/env variable automatically to code running on heroku. ok, that’s kind of nice, but it’s not a big deal to set a solr_url heroku config manually referencing the appropriate address. i suppose as a heroku add-on, websolr also takes care of securing and authenticating connections between the heroku dynos and the solr, so we need to make sure we have a reasonable way to do this from any alternative. searchstax (did take it for a spin) searchstax’s pricing tiers are not based on metering usage. there are no limits based on requests/day or concurrent connections. searchstax runs on dedicated-to-you individual solr instances (i would guess running on dedicated-to-you individual (eg) ec , but i’m not sure). instead the pricing is based on size of host running solr. you can choose to run on instances deployed to aws, google cloud, or azure. we’ll be sticking to aws (the others, i think, have a slight price premium). while searchstax gives you a pricing pages that looks like the “new-way-of-doing-things” transparent pricing, in fact there isn’t really enough info on public pages to see all the price points and understand what you’re getting, there is still a kind of “talk to a salesperson who has a price sheet” thing going on. what i think i have figured out from talking to a salesperson and support, is that the “silver” plans (“starting at $ a month”, although we’ll say more about that in a bit) are basically: we give you a solr, we don’t don’t provide any technical support for solr. while the “gold” plans “from $ /month” are actually about paying for solr consultants to set up and tune your schema/index etc. that is not something we need, and $ +/month is way more than the price range we are looking for. while the searchstax pricing/plan pages kind of imply the “silver” plan is not suitable for production, in fact there is no real reason not to use it for production i think, and the salesperson i talked to confirmed that — just reaffirming that you were on your own managing the solr configuration/setup. that’s fine, that’s what we want, we just don’t want to mangage the os or set up the solr or upgrade it etc. the silver plans have no sla, but as far as i can tell their uptime is just fine. the silver plans only guarantees -hour support response time — but for the couple support tickets i filed asking questions while under a free -day trial (oh yeah that’s available), i got prompt same-day responses, and knowledgeable responses that answered my questions. so a “silver” plan is what we are interested in, but the pricing is not actually transparent. $ /month is for the smallest instance available, and if you prepay/contract for a year. they call that small instance an ndn and it has gb of ram and gb of storage. if you pay-as-you-go instead of contracting for a year, that already jumps to $ /month. (that price is available on the trial page). when you are paying-as-you-go, you are actually billed per-day, which might not be as nice as heroku’s per-minute, but it’s pretty okay, and useful if you need to bring up a temporary solr instance as part of a migration/upgrade or something like that. the next step up is an “ndn ” which has g of ram and gb of storage, and has an ~$ /month pay-as-you-go — you can find that price if you sign-up for a free trial. the discount price price for an annual contract is a discount similar to the ndn %, $ /month — that price i got only from a salesperson, i don’t know if it’s always stable. it only occurs to me now that they don’t tell you how many cpus are available. i’m not sure if i can fit our solr in the g ndn , but i am sure i can fit it in the g ndn with some headroom, so i didn’t look at plans above that — but they are available, still under “silver”, with prices going up accordingly. all searchstax solr instances run in “solrcloud” mode — these ndn and ndn ones we’re looking at just run one node with one zookeeper, but still in cloud mode. there are also “silver” plans available with more than one node in a “high availability” configuration, but the prices start going up steeply, and we weren’t really interested in that. because it’s solrcloud mode though, you can use the standard solr api for uploading your configuration. it’s just solr! so no arbitrary usage limits, no features disabled. the searchstax web console seems competently implemented; it let’s you create and delete individual solr “deployments”, manage accounts to login to console (on “silver” plan you only get two, or can pay $ /month/account for more, nah), and set up auth for a solr deployment. they support ip-based authentication or http basic auth to the solr (no limit to how many solr basic auth accounts you can create). http basic auth is great for us, because trying to do ip-based from somewhere like heroku isn’t going to work. all solrs are available over https/ssl — great! searchstax also has their own proprietary http api that lets you do most anything, including creating/destroying deployments, managing solr basic auth users, basically everything. there is some api that duplicates the solr cloud api for adding configsets, i don’t think there’s a good reason to use it instead of standard solrcloud api, although their docs try to point you to it. there’s even some kind of webhooks for alerts! (which i haven’t really explored). basically, searchstax just seems to be a sane and rational managed solr option, it has all the features you’d expect/need/want for dealing with such. the prices seem reasonable-ish, generally more affordable than websolr, especially if you stay in “silver” and “one node”. at present, we plan to move forward with it. opensolr (didn’t look at it much) i have the least to say about this, have spent the least time with it, after spending time with searchstax and seeing it met our needs. but i wanted to make sure to mention it, because it’s the only other managed solr i am even aware of. definitely curious to hear from any users. here is the pricing page. the prices seem pretty decent, perhaps even cheaper than searchstax, although it’s unclear to me what you get. does “ solr clusters” mean that it’s not solrcloud mode? after seeing how useful solrcloud apis are for management (and having this confirmed by many of my peers in other libraries/museums/archives who choose to run solrcloud), i wouldn’t want to do without it. so i guess that pushes us to “executive” tier? which at $ /month (billed yearly!) is still just fine, around the same as searchstax. but they do limit you to one solr index; i prefer searchstax’s model of just giving you certain host resources and do what you want with it. it does say “shared infrastructure”. might be worth investigating, curious to hear more from anyone who did. now, what about elasticsearch? we’re using solr mostly because that’s what various collaborative and open source projects in the library/museum/archive world have been doing for years, since before elasticsearch even existed. so there are various open source libraries and toolsets available that we’re using. but for whatever reason, there seem to be so many more managed elasticsearch saas available. at possibly much cheaper pricepoints. is this because the elasticsearch market is just bigger? or is elasticsearch easier/cheaper to run in a saas environment? or what? i don’t know. but there’s the controversial aws elasticsearch service; there’s the elastic cloud “from the creators of elasticsearch”. on heroku that lists one solr add-on, there are three elasticsearch add-ons listed: elasticcloud, bonsai elasticsearch, and searchbox elasticsearch. if you just google “managed elasticsearch” you immediately see or other names. i don’t know enough about elasticsearch to evaluate them. there seem on first glance at pricing pages to be more affordable, but i may not know what i’m comparing and be looking at tiers that aren’t actually usable for anything or will have hidden fees. but i know there are definitely many more managed elasticsearch saas than solr. i think elasticsearch probably does everything our app needs. if i were to start from scratch, i would definitely consider elasticsearch over solr just based on how many more saas options there are. while it would require some knowledge-building (i have developed a lot of knowlege of solr and zero of elasticsearch) and rewriting some parts of our stack, i might still consider switching to es in the future, we don’t do anything too too complicated with solr that would be too too hard to switch to es, probably. jrochkind general leave a comment january , january , gem authors, check your release sizes most gems should probably be a couple hundred kb at most. i’m talking about the package actually stored in and downloaded from rubygems by an app using the gem. after all, source code is just text, and it doesn’t take up much space. ok, maybe some gems have a couple images in there. but if you are looking at your gem in rubygems and realize that it’s mb or bigger… and that it seems to be getting bigger with every release… something is probably wrong and worth looking into it. one way to look into it is to look at the actual gem package. if you use the handy bundler rake task to release your gem (and i recommend it), you have a ./pkg directory in your source you last released from. inside it are “.gem” files for each release you’ve made from there, unless you’ve cleaned it up recently. .gem files are just .tar files it turns out. that have more tar and gz files inside them etc. we can go into it, extract contents, and use the handy unix utility du -sh to see what is taking up all the space. how i found the bytes jrochkind-chf kithe (master ?) $ cd pkg jrochkind-chf pkg (master ?) $ ls kithe- . . .beta .gem kithe- . . .pre.rc .gem kithe- . . .gem kithe- . . .gem kithe- . . .pre.beta .gem kithe- . . .gem jrochkind-chf pkg (master ?) $ mkdir exploded jrochkind-chf pkg (master ?) $ cp kithe- . . .gem exploded/kithe- . . .tar jrochkind-chf pkg (master ?) $ cd exploded jrochkind-chf exploded (master ?) $ tar -xvf kithe- . . .tar x metadata.gz x data.tar.gz x checksums.yaml.gz jrochkind-chf exploded (master ?) $ mkdir unpacked_data_tar jrochkind-chf exploded (master ?) $ tar -xvf data.tar.gz -c unpacked_data_tar/ jrochkind-chf exploded (master ?) $ cd unpacked_data_tar/ /users/jrochkind/code/kithe/pkg/exploded/unpacked_data_tar jrochkind-chf unpacked_data_tar (master ?) $ du -sh * . k mit-license k readme.md . k rakefile k app . k config k db k lib m spec jrochkind-chf unpacked_data_tar (master ?) $ cd spec jrochkind-chf spec (master ?) $ du -sh * . k derivative_transformers m dummy k factories k indexing k models . k rails_helper.rb k shrine k simple_form_enhancements . k spec_helper.rb k test_support . k validators jrochkind-chf spec (master ?) $ cd dummy/ jrochkind-chf dummy (master ?) $ du -sh * . k rakefile k app k bin k config . k config.ru . k db m log . k package.json k public . k tmp doh! in this particular gem, i have a dummy rails app, and it has mb of logs, cause i haven’t b bothered trimming them in a while, that are winding up including in the gem release package distributed to rubygems and downloaded by all consumers! even if they were small, i don’t want these in the released gem package at all! that’s not good! it only turns into mb instead of mb, because log files are so compressable and there is compression involved in assembling the rubygems package. but i have no idea how much space it’s actually taking up on consuming applications machines. this is very irresponsible! what controls what files are included in the gem package? your .gemspec file of course. the line s.files = is an array of every file to include in the gem package. well, plus s.test_files is another array of more files, that aren’t supposed to be necessary to run the gem, but are to test it. (rubygems was set up to allow automated *testing* of gems after download, is why test files are included in the release package. i am not sure how useful this is, and who if anyone does it; although i believe that some linux distro packagers try to make use of it, for better or worse). but nobody wants to list every file in your gem individually, manually editing the array every time you add, remove, or move one. fortunately, gemspec files are executable ruby code, so you can use ruby as a shortcut. i have seen two main ways of doing this, with different “gem skeleton generators” taking one of two approaches. sometimes a shell out to git is used — the idea is that everything you have checked into your git should be in the gem release package, no more or no less. for instance, one of my gems has this in it, not sure where it came from or who/what generated it. spec.files = `git ls-files -z`.split("\x ").reject do |f| f.match(%r{^(test|spec|features)/}) end in that case, it wouldn’t have included anything in ./spec already, so this obviously isn’t actually the gem we were looking at before. but in this case, in addition to using ruby logic to manipulate the results, nothing excluded by your .gitignore file will end up included in your gem package, great! in kithe we were looking at before, those log files were in the .gitignore (they weren’t in my repo!), so if i had been using that git-shellout technique, they wouldn’t have been included in the ruby release already. but… i wasn’t. instead this gem has a gemspec that looks like: s.test_files = dir["spec/*/"] just include every single file inside ./spec in the test_files list. oops. then i get all those log files! one way to fix i don’t really know which is to be preferred of the git-shellout approach vs the dir-glob approach. i suspect it is the subject of historical religious wars in rubydom, when there were still more people around to argue about such things. any opinions? or another approach? without being in the mood to restructure this gemspec in anyway, i just did the simplest thing to keep those log files out… dir["spec/*/"].delete_if {|a| a =~ %r{/dummy/log/}} build the package without releasing with the handy bundler supplied rake build task… and my gem release package size goes from mb to k. (which actually kind of sounds like a minimum block size or something, right?) phew! that’s a big difference! sorry for anyone using previous versions and winding up downloading all that cruft! (actually this particular gem is mostly a proof of concept at this point and i don’t think anyone else is using it). check your gem sizes! i’d be willing to be there are lots of released gems with heavily bloated release packages like this. this isn’t the first one i’ve realized was my fault. because who pays attention to gem sizes anyway? apparently not many! but rubygems does list them, so it’s pretty easy to see. are your gem release packages multiple megs, when there’s no good reason for them to be? do they get bigger every release by far more than the bytes of lines of code you think were added? at some point in gem history was there a big jump from hundreds of kb to multiple mb? when nothing particularly actually happened to gem logic to lead to that? all hints that you might be including things you didn’t mean to include, possibly things that grow each release. you don’t need to have a dummy rails app in your repo to accidentally do this (i accidentally did it once with a gem that had nothing to do with rails). there could be other kind of log files. or test coverage or performance metric files, or any other artifacts of your build or your development, especially ones that grow over time — that aren’t actually meant to or needed as part of the gem release package! it’s good to sanity check your gem release packages now and then. in most cases, your gem release package should be hundreds of kb at most, not mbs. help keep your users’ installs and builds faster and slimmer! jrochkind general leave a comment january , every time you decide to solve a problem with code… every time you decide to solve a problem with code, you are committing part of your future capacity to maintaining and operating that code. software is never done. software is drowning the world by james abley jrochkind general leave a comment january , updating solrcloud configuration in ruby we have an app that uses solr. we currently run a solr in legacy “not cloud” mode. our solr configuration directory is on disk on the solr server, and it’s up to our processes to get our desired solr configuration there, and to update it when it changes. we are in the process of moving to a solr in “solrcloud mode“, probably via the searchstax managed solr service. our solr “cloud” might only have one node, but “solrcloud mode” gives us access to additional api’s for managing your solr configuration, as opposed to writing it directly to disk (which may not be possible at all in solrcloud mode? and certainly isn’t using managed searchstax). that is, the solr configsets api, although you might also want to use a few pieces of the collection management api for associating a configset with a solr collection. basically, you are taking your desired solr config directory, zipping it up, and uploading it to solr as a “config set” [or “configset”] with a certain name. then you can create collections using this config set, or reassign which named configset an existing collection uses. i wasn’t able to find any existing ruby gems for interacting with these solr api’s. rsolr is a “ruby client for interacting with solr”, but was written before most of these administrative api’s existed for solr, and doesn’t seem to have been updated to deal with them (unless i missed it), rsolr seems to be mostly/only about querying solr, and some limited indexing. but no worries, it’s not too hard to wrap the specific api i want to use in some ruby. which did seem far better to me than writing the specific http requests each time (and making sure you are dealing with errors etc!). (and yes, i will share the code with you). i decided i wanted an object that was bound to a particular solr collection at a particular solr instance; and was backed by a particular local directory with solr config. that worked well for my use case, and i wound up with an api that looks like this: updater = solrconfigsetupdater.new( solr_url: "https://example.com/solr", conf_dir: "./solr/conf", collection_name: "mycollection" ) # will zip up ./solr/conf and upload it as named myconfigset: updater.upload("myconfigset") updater.list #=> ["myconfigset"] updater.config_name # what configset name is mycollection currently configured to use? # => "oldconfigset" # what if we try to delete the one it's using? updater.delete("oldconfigset") # => raises solrconfigsetupdater::solrerror with message: # "can not delete configset as it is currently being used by collection [myconfigset]" # okay let's change it to use the new one and delete the old one updater.update_config_name("myconfigset") # now mycollection uses this new configset, although we possibly # need to reload the collection to make that so updater.reload # now let's delete the one we're not using updater.delete("oldconfigset") ok, great. there were some tricks in there in trying to catch the apparently multiple ways solr can report different kinds of errors, to make sure solr-reported errors turn into exceptions ideally with good error messages. now, in addition to uploading a configset initially for a collection you are creating to use, the main use case i have is wanting to update the configuration to new values in an existing collection. sure, this often requires a reindex afterwards. if you have the recently released solr . , it will let you overwrite an existing configset, so this can be done pretty easily. updater.upload(updater.config_name, overwrite: true) updater.reload but prior to solr . you can not overwrite an existing configset. and searchstax doesn’t yet have solr . . so one way or another, we need to do a dance where we upload the configset under a new name than switch the collection to use it. having this updater object that lets us easily execute relevant solr api lets us easily experiment with different logic flows for this. for instance in a solr listserv thread, alex halovnic suggests a somewhat complicated -step process workaround, which we can implement like so: current_name = updater.config_name temp_name = "#{current_name}_temp" updater.create(from: current_name, to: temp_name) updater.change_config_name(temp_name) updater.reload updater.delete(current_name) updater.upload(configset_name: current_name) updater.change_config_name(current_name) updater.reload updater.delete(temp_name) that works. but talking to dann bohn at penn state university, he shared a different algorithm, which goes like: make a cryptographic digest hash of the entire solr directory, which we’re going to use in the configset name. check if the collection is already using a configset named $name_$digest, which if it already is, you’re done, no change needed. otherwise, upload the configset with the fingerprint-based name, switch the collection to use it, reload, delete the configset that the collection used to use. at first this seemed like overkill to me, but after thinking and experimenting with it, i like it! it is really quick to make a digest of a handful of files, that’s not a big deal. (i use first chars of hex sha ). and even if we had solr . , i like that we can avoid doing any operation on solr at all if there had been no changes — i really want to use this operation much like a rails db:migrate, running it on every deploy to make sure the solr schema matches the one in the repo for the depoy. dann also shared his open source code with me, which was helpful for seeing how to make the digest, how to make a zip file in ruby, etc. thanks dann! sharing my code so i also wrote some methods to implement those variant updating stragies, dann’s, and alex halovnic’s from the list etc. i thought about wrapping this all up as a gem, but i didn’t really have the time to make it really good enough for that. my api is a little bit janky, i didn’t spend the extra time think it out really well to minimize the need for future backwards incompat changes like i would if it were a gem. i also couldn’t figure out a great way to write automated tests for this that i would find particularly useful; so in my code base it’s actually not currently test-covered (shhhhh) but in a gem i’d want to solve that somehow. but i did try to write the code general purpose/flexible so other people could use it for their use cases; i tried to document it to my highest standards; and i put it all in one file which actually might not be the best oo abstraction/design, but makes it easier for you to copy and paste the single file for your own use. :) so you can find my code here; it is apache-licensed; and you are welcome to copy and paste it and do whatever you like with it, including making a gem yourself if you want. maybe i’ll get around to making it a gem in the future myself, i dunno, curious if there’s interest. the searchstax proprietary api’s searchstax has it’s own api’s that can i think be used for updating configsets and setting collections to use certain configsets etc. when i started exploring them, they are’t the worst vendor api’s i’ve seen, but i did find them a bit cumbersome to work with. the auth system involves a lot of steps (why can’t you just create an api key from the searchstax web gui?). overall i found them harder to use than just the standard solr cloud api’s, which worked fine in the searchstax deployment, and have the added bonus of being transferable to any solrcloud deployment instead of being searchstax-specific. while the searchstax docs and support try to steer you to the searchstax specific api’s, i don’t think there’s really any good reason for this. (perhaps the custom searchstax api’s were written long ago when solr api’s weren’t as complete?) searchstax support suggested that the searchstax apis were somehow more secure; but my searchstax solr api’s are protected behind http basic auth, and if i’ve created basic auth credentials (or ip addr allowlist) those api’s will be available to anyone with auth to access solr whether i use em or not! and support also suggested that the searchstax api use would be logged, whereas my direct solr api use would not be, which seems to be true at least in default setup, i can probably configure solr logging differently, but it just isn’t that important to me for these particular functions. so after some initial exploration with searchstax api, i realized that solrcloud api (which i had never used before) could do everything i need and was more straightforward and transferable to use, and i’m happy with my decision to go with that. jrochkind general comments december , december , are you talking to heroku redis in cleartext or ssl? in “typical” redis installation, you might be talking to redis on localhost or on a private network, and clients typically talk to redis in cleartext. redis doesn’t even natively support communications over ssl. (or maybe it does now with redis ?) however, the heroku redis add-on (the one from heroku itself) supports ssl connections via “stunnel”, a tool popular with other redis users use to get ssl redis connections too. (or maybe via native redis with redis ? not sure if you’d know the difference, or if it matters). there are heroku docs on all of this which say: while you can connect to heroku redis without the stunnel buildpack, it is not recommend. the data traveling over the wire will be unencrypted. perhaps especially because on heroku your app does not talk to redis via localhost or on a private network, but on a public network. but i think i’ve worked on heroku apps before that missed this advice and are still talking to heroku in the clear. i just happened to run across it when i got curious about the redis_tls_url env/config variable i noticed heroku setting. which brings us to another thing, that heroku doc on it is out of date, it doesn’t mention the redis_tls_url config variable, just the redis_url one. the difference? the tls version will be a url beginning with rediss:// instead of redis:// , note extra s, which many redis clients use as a convention for “ssl connection to redis probably via stunnel since redis itself doens’t support it”. the redis docs provide ruby and go examples which instead use redis_url and writing code to swap the redis:// for rediss:// and even hard-code port number adjustments, which is silly! (while i continue to be very impressed with heroku as a product, i keep running into weird things like this outdated documentation, that does not match my experience/impression of heroku’s all-around technical excellence, and makes me worry if heroku is slipping…). the docs also mention a weird driver: ruby arg for initializing the redis client that i’m not sure what it is and it doesn’t seem necessary. the docs are correct that you have to tell the ruby redis client not to try to verify ssl keys against trusted root certs, and this implementation uses a self-signed cert. otherwise you will get an error that looks like: openssl::ssl::sslerror: ssl_connect returned= errno= state=error: certificate verify failed (self signed certificate in certificate chain) so, can be as simple as: redis_client = redis.new(url: env['redis_tls_url'], ssl_params: { verify_mode: openssl::ssl::verify_none }) $redis = redis_client # and/or resque.redis = redis_client i don’t use sidekiq on this project currently, but to get the ssl connection with verify_none, looking at sidekiq docs maybe on sidekiq docs you might have to(?): redis_conn = proc { redis.new(url: env['redis_tls_url'], ssl_params: { verify_mode: openssl::ssl::verify_none }) } sidekiq.configure_client do |config| config.redis = connectionpool.new(size: , &redis_conn) end sidekiq.configure_server do |config| config.redis = connectionpool.new(size: , &redis_conn) end (not sure what values you should pick for connection pool size). while the sidekiq docs mention heroku in passing, they don’t mention need for ssl connections — i think awareness of this heroku feature and their recommendation you use it may not actually be common! update: beware redis_url can also be rediss on one of my apps i saw a redis_url which used redis: and a redis_tls_url which uses (secure) rediss:. but on another app, it provides *only* a redis_url, which is rediss — meaning you have to set the verify_mode: openssl::ssl::verify_none when passing it to ruby redis client. so you have to be prepared to do this with redis_url values too — i think it shouldn’t hurt to set the ssl_params option even if you pass it a non-ssl redis: url, so just set it all the time? this second app was heroku- stack, and the first was heroku- stack, is that the difference? no idea. documented anywhere? i doubt it. definitely seems sloppy for what i expect of heroku, making me get a bit suspicious of whether heroku is sticking to the really impressive level of technical excellence and documentation i expect from them. so, your best bet is to check for both redis_tls_url and redis_url, prefering the tls one if present, realizing the redis_url can have a rediss:// value in it too. the heroku docs also say you don’t get secure tls redis connection on “hobby” plans, but i”m not sure that’s actually true anymore on heroku- ? not trusting the docs is not a good sign. jrochkind general comments november , november , comparing performance of a rails app on different heroku formations i develop a “digital collections” or “asset management” app, which manages and makes digitized historical objects and their descriptions available to the public, from the collections here at the science history institute. the app receives relatively low level of traffic (according to google analytics, around k pageviews a month), although we want it to be able to handle spikes without falling down. it is not the most performance-optimized app, it does have some relatively slow responses and can be ram-hungry. but it works adequately on our current infrastructure: web traffic is handled on a single aws ec t .medium instance, with passenger processes (free version of passenger, so no multi-threading). we are currently investigating the possibility of moving our infrastructure to heroku. after realizing that heroku standard dynos did not seem to have the performance characteristics i had expected, i decided to approach performance testing more methodically, to compare different heroku dyno formations to each other and to our current infrastructure. our basic research question is probably what heroku formation do we need to have similar performance to our existing infrastructure? i am not an expert at doing this — i did some research, read some blog posts, did some thinking, and embarked on this. i am going to lead you through how i approached this and what i found. feedback or suggestions are welcome. the most surprising result i found was much poorer performance from heroku standard dynos than i expected, and specifically that standard dynos would not match performance of present infrastructure. what urls to use in test some older load-testing tools only support testing one url over and over. i decided i wanted to test a larger sample list of urls — to be a more “realistic” load, and also because repeatedly requesting only one url might accidentally use caches in ways you aren’t expecting giving you unrepresentative results. (our app does not currently use fragment caching, but caches you might not even be thinking about include postgres’s built-in automatic caches, or passenger’s automatic turbocache (which i don’t think we have turned on)). my initial thought to get a list of such urls from our already-in-production app from production logs, to get a sample of what real traffic looks like. there were a couple barriers for me to using production logs as urls: some of those urls might require authentication, or be post requests. the bulk of our app’s traffic is get requests available without authentication, and i didn’t feel like the added complexity of setting up anything else in a load traffic was worthwhile. our app on heroku isn’t fully functional yet. without having connected it to a solr or background job workers, only certain urls are available. in fact, a large portion of our traffic is an “item” or “work” detail page like this one. additionally, those are the pages that can be the biggest performance challenge, since the current implementation includes a thumbnail for every scanned page or other image, so response time unfortunately scales with number of pages in an item. so i decided a good list of urls was simply a representative same of those “work detail” pages. in fact, rather than completely random sample, i took the largest/slowest work pages, and then added in another randomly chosen from our current ~ k pages. and gave them all a randomly shuffled order. in our app, every time a browser requests a work detail page, the js on that page makes an additional request for a json document that powers our page viewer. so for each of those work detail pages, i added the json request url, for a more “realistic” load, and total urls. performance: “base speed” vs “throughput under load” thinking about it, i realized there were two kinds of “performance” or “speed” to think about. you might just have a really slow app, to exagerate let’s say typical responses are seconds. that’s under low/no-traffic, a single browser is the only thing interacting with the app, it makes a single request, and has to wait seconds for a response. that number might be changed by optimizations or performance regressions in your code (including your dependencies). it might also be changed by moving or changing hardware or virtualization environment — including giving your database more cpu/ram resources, etc. but that number will not change by horizontally scaling your deployment — adding more puma or passenger processes or threads, scaling out hosts with a load balancer or heroku dynos. none of that will change this base speed because it’s just how long the app takes to prepare a response when not under load, how slow it is in a test only one web worker , where adding web workers won’t matter because they won’t be used. then there’s what happens to the app actually under load by multiple users at once. the base speed is kind of a lower bound on throughput under load — page response time is never going to get better than s for our hypothetical very slow app (without changing the underlying base speed). but it can get a lot worse if it’s hammered by traffic. this throughput under load can be effected not only by changing base speed, but also by various forms of horizontal scaling — how many puma or passenger processes you have with how many threads each, and how many cpus they have access to, as well as number of heroku dynos or other hosts behind a load balancer. (i had been thinking about this distinction already, but nate berkopec’s great blog post on scaling rails apps gave me the “speed” vs “throughout” terminology to use). for my condition, we are not changing the code at all. but we are changing the host architecture from a manual ec t .medium to heroku dynos (of various possible types) in a way that could effect base speed, and we’re also changing our scaling architecture in a way that could change throughput under load on top of that — from one t .medium with passenger process to possibly multiple heroku dynos behind heroku’s load balancer, and also (for reasons) switching from free passenger to trying puma with multiple threads per process. (we are running puma with new experimental performance features turned on). so we’ll want to get a sense of base speed of the various host choices, and also look at how throughput under load changes based on various choices. benchmarking tool: wrk we’re going to use wrk. there are lots of choices for http benchmarking/load testing, with really varying complexity and from different eras of web history. i got a bit overwhelmed by it, but settled on wrk. some other choices didn’t have all the features we need (some way to test a list of urls, with at least some limited percentile distribution reporting). others were much more flexible and complicated and i had trouble even figuring out how to use them! wrk does need a custom lua script in order to handle a list of urls. i found a nice script here, and modified it slightly to take filename from an env variable, and not randomly shuffle input list. it’s a bit confusing understanding the meaning of “threads” vs “connections” in wrk arguments. this blog post from appfolio clears it up a bit. i decided to leave threads set to , and vary connections for load — so -c -t is a “one url at a time” setting we can use to test “base speed”, and we can benchmark throughput under load by increasing connections. we want to make sure we run the test for long enough to touch all urls in our list at least once, even in the slower setups, to have a good comparison — ideally it would be go through the list more than once, but for my own ergonomics i had to get through a lot of tests so ended up less tha ideal. (should i have put fewer than urls in? not sure). conclusions in advance as benchmarking posts go (especially when i’m the one writing them), i’m about to drop a lot of words and data on you. so to maximize the audience that sees the conclusions (because they surprise me, and i want feedback/pushback on them), i’m going to give you some conclusions up front. our current infrastructure has web app on a single ec t .medium, which is a burstable ec type — our relatively low-traffic app does not exhaust it’s burst credits. measuring base speed (just one concurrent request at a time), we found that performance dynos seem to have about the cpu speed of a bursting t .medium (just a hair slower). but standard dynos are as a rule to times slower; additionally they are highly variable, and that variability can be over hours/days. a minute period can have measured response times or more times slower than another minute period a couple hours later. but they seem to typically be - x slower than our current infrastructure. under load, they scale about how you’d expect if you knew how many cpus are present, no real surprises. our existing t .medium has two cpus, so can handle simultaneous requests as fast as , and after that degrades linearly. a single performance-l ($ /month) has cpus ( hyperthreads), so scales under load much better than our current infrastructure. a single performance-m ($ /month) has only cpu (!), so scales pretty terribly under load. testing scaling with standard- x’s ($ /month total), we see that it scales relatively evenly. although lumpily because of variability, and it starts out so much worse performing that even as it scales “evenly” it’s still out-performed by all other arcchitectures. :( (at these relatively fast median response times you might say it’s still fast enough who cares, but in our fat tail of slower pages it gets more distressing). now we’ll give you lots of measurements, or you can skip all that to my summary discussion or conclusions for our own project at the end. let’s compare base speed ok, let’s get to actual measurements! for “base speed” measurements, we’ll be telling wrk to use only one connection and one thread. existing t .medium: base speed our current infrastructure is one ec t .medium. this ec instance type has two vcpus and gb of ram. on that single ec instance, we run passenger (free not enterprise) set to have passenger processes, although the base speed test with only one connection should only touch one of the workers. the t is a “burstable” type, and we do always have burst credits (this is not a high traffic app; verified we never exhausted burst credits in these tests), so our test load may be taking advantage of burst cpu. $ urls=./sample_works.txt wrk -c -t -d m --timeout s --latency -s load_test/multiplepaths.lua.txt https://[current staging server] multiplepaths: found paths multiplepaths: found paths running m test @ https://staging-digital.sciencehistory.org threads and connections thread stats avg stdev max +/- stdev latency . ms . ms . s . % req/sec . . . . % latency distribution % . ms % . ms % . ms % . s requests in . m, . mb read requests/sec: . transfer/sec: . mb i’m actually feeling pretty good about those numbers on our current infrastructure! ms median, not bad, and even ms th percentile is not too bad. now, our test load involves some json responses that are quicker to deliver than corresponding html page, but still pretty good. the th/ th/and max request ( . s) aren’t great, but i knew i had some slow pages, this matches my previous understanding of how slow they are in our current infrastructure. th percentile is ~ times th percenile. i don’t have an understanding of why the two different req/sec and requests/sec values are so different, and don’t totally understand what to do with the stdev and +/- stdev values, so i’m just going to be sticking to looking at the latency percentiles, i think “latency” could also be called “response times” here. but ok, this is our baseline for this workload. and doing this minute test at various points over the past few days, i can say it’s nicely regular and consistent, occasionally i got a slower run, but th percentile was usually ms– ms, right around there. heroku standard- x: base speed from previous mucking about, i learned i can only reliably fit one puma worker in a standard- x, and heroku says “we typically recommend a minimum of processes, if possible” (for routing algorithmic reasons when scaled to multiple dynos), so i am just starting at a standard- x with two puma workers each with threads, matching heroku recommendations for a standard- x dyno. so one thing i discovered is that bencharks from a heroku standard dyno are really variable, but here are typical ones: $ heroku dyno:resize type size qty cost/mo ─────── ─────────── ─── ─────── web standard- x $ heroku config:get --shell web_concurrency rails_max_threads web_concurrency= rails_max_threads= $ urls=./sample_works.txt wrk -c -t -d m --timeout s --latency -s load_test/multiplepaths.lua.txt https://scihist-digicoll.herokuapp.com/ multiplepaths: found paths multiplepaths: found paths running m test @ https://scihist-digicoll.herokuapp.com/ threads and connections thread stats avg stdev max +/- stdev latency . ms . ms . s . % req/sec . . . . % latency distribution % . ms % . ms % . s % . s requests in . m, . mb read requests/sec: . transfer/sec: . kb i had heard that heroku standard dynos would have variable performance, because they are shared multi-tenant resources. i had been thinking of this like during a minute test i might see around the same median with more standard deviation — but instead, what it looks like to me is that running this benchmark on monday at am might give very different results than at : am or tuesday at pm. the variability is over a way longer timeframe than my minute test — so that’s something learned. running this here and there over the past week, the above results seem to me typical of what i saw. (to get better than “seem typical” on this resource, you’d have to run a test, over several days or a week i think, probably not hammering the server the whole time, to get a sense of actual statistical distribution of the variability). i sometimes saw tests that were quite a bit slower than this, up to a ms median. i rarely if ever saw results too much faster than this on a standard- x. th percentile is ~ x median, less than my current infrastructure, but that still gets up there to . instead of ms. this typical one is quite a bit slower than than our current infrastructure, our median response time is x the latency, with th and max being around x. this was worse than i expected. heroku performance-m: base speed although we might be able to fit more puma workers in ram, we’re running a single-connection base speed test, so it shouldn’t matter to, and we won’t adjust it. $ heroku dyno:resize type size qty cost/mo ─────── ───────────── ─── ─────── web performance-m $ heroku config:get --shell web_concurrency rails_max_threads web_concurrency= rails_max_threads= $ urls=./sample_works.txt wrk -c -t -d m --timeout s --latency -s load_test/multiplepaths.lua.txt https://scihist-digicoll.herokuapp.com/ multiplepaths: found paths multiplepaths: found paths running m test @ https://scihist-digicoll.herokuapp.com/ threads and connections thread stats avg stdev max +/- stdev latency . ms . ms . s . % req/sec . . . . % latency distribution % . ms % . ms % . s % . s requests in . m, . mb read requests/sec: . transfer/sec: . kb this is a lot closer to the ballpark of our current infrastructure. it’s a bit slower ( ms median intead of ms median), but in running this now and then over the past week it was remarkably, thankfully, consistent. median and th percentile are both % slower (makes me feel comforted that those numbers are the same in these two runs!), that doesn’t bother me so much if it’s predictable and regular, which it appears to be. the max appears to me still a little bit less regular on heroku for some reason, since performance is supposed to be non-shared aws resources, you wouldn’t expect it to be, but slow requests are slow, ok. th percentile is ~ x median, about the same as my current infrastructure. heroku performance-l: base speed $ heroku dyno:resize type size qty cost/mo ─────── ───────────── ─── ─────── web performance-l $ heroku config:get --shell web_concurrency rails_max_threads web_concurrency= rails_max_threads= urls=./sample_works.txt wrk -c -t -d m --timeout s --latency -s load_test/multiplepaths.lua.txt https://scihist-digicoll.herokuapp.com/ multiplepaths: found paths multiplepaths: found paths running m test @ https://scihist-digicoll.herokuapp.com/ threads and connections thread stats avg stdev max +/- stdev latency . ms . ms . s . % req/sec . . . . % latency distribution % . ms % . ms % . s % . s requests in . m, . mb read requests/sec: . transfer/sec: . kb no news is good news, it looks very much like performance-m, which is exactly what we expected, because this isn’t a load test. it tells us that performance-m and performance-l seem to have similar cpu speeds and similar predictable non-variable regularity, which is what i find running this test periodically over a week. th percentile is ~ x median, about the same as current infrastructure. the higher max speed is just evidence of what i mentioned, the speed of slowest request did seem to vary more than on our manual t .medium, can’t really explain why. summary: base speed not sure how helpful this visualization is, charting th, th, and th percentile responses across architectures. but basically: performance dynos perform similarly to my (bursting) t .medium. can’t explain why performance-l seems slightly slower than performance-m, might be just incidental variation when i ran the tests. the standard- x is about twice as slow as my (bursting) t .medium. again recall standard- x results varied a lot every time i ran them, the one i reported seems “typical” to me, that’s not super scientific, admittedly, but i’m confident that standard- x are a lot slower in median response times than my current infrastructure. throughput under load ok, now we’re going to test using wrk to use more connections. in fact, i’ll test each setup with various number of connections, and graph the result, to get a sense of how each formation can handle throughput under load. (this means a lot of minutes to get all these results, at minutes per number of connection test, per formation!). an additional thing we can learn from this test, on heroku we can look at how much ram is being used after a load test, to get a sense of the app’s ram usage under traffic to understand the maximum number of puma workers we might be able to fit in a given dyno. existing t .medium: under load a t .medium has g of ram and cpus. we run passenger workers (no multi-threading, since we are free, rather than enterprise, passenger). so what do we expect? with cpus and more than workers, i’d expect it to handle simultaneous streams of requests almost as well as ; - should be quite a bit slower because they are competing for the cpus. over , performance will probably become catastrophic. connections are exactly flat with , as expected for our two cpus, hooray! then it goes up at a strikingly even line. going over (to ) simultaneous connections doesn’t matter, even though we’ve exhausted our workers, i guess at this point there’s so much competition for the two cpus already. the slope of this curve is really nice too actually. without load, our median response time is ms, but even at a totally overloaded overloaded connections, it’s only ms, which actually isn’t too bad. we can make a graph that in addition to median also has th, th, and th percentile response time on it: it doesn’t tell us too much; it tells us the upper percentiles rise at about the same rate as the median. at simultaneous connection th percentile of ms is about times the median of ms; at requests the th percentile of . seconds is about times the median of ms. this does remind us that under load when things get slow, this has more of a disastrous effect on already slow requests than fast requests. when not under load, even our th percentile was kind of sort of barley acceptable at ms, but under load at . seconds it really isn’t. single standard- x dyno: under load a standard- x dyno has g of ram. the (amazing, excellent, thanks schneems) heroku puma guide suggests running two puma workers with threads each. at first i wanted to try running three workers, which seemed to fit into available ram — but under heavy load-testing i was getting heroku r memory quota exceeded errors, so we’ll just stick with the heroku docs recommendations. two workers with threads each fit with plenty of headroom. a standard- x dyno is runs on shared (multi-tenant) underlying amazon virtual hardware. so while it is running on hardware with cpus (each of which can run two “hyperthreads“), the puma doc suggests “it is best to assume only one process can execute at a time” on standard dynos. what do we expect? well, if it really only had one cpu, it would immediately start getting bad at simulataneous connections, and just get worse from there. when we exceed the two worker count, will it get even worse? what about when we exceed the thread ( workers * threads) count? you’d never run just one dyno if you were expecting this much traffic, you’d always horizontally scale. this very artificial test is just to get a sense of it’s characteristics. also, we remember that standard- x’s are just really variable; i could get much worse or better runs than this, but graphed numbers from a run that seemed typical. well, it really does act like cpu, simultaneous connections is immediately a lot worse than . the line isn’t quite as straight as in our existing t .medium, but it’s still pretty straight; i’d attribute the slight lumpiness to just the variability of shared-architecture standard dyno, and figure it would get perfectly straight with more data. it degrades at about the same rate of our baseline t .medium, but when you start out slower, that’s more disastrous. our t .medium at an overloaded simultaneous requests is ms (pretty tolerable actually), times the median at one request only. this standard- x has a median response time of ms at only one simultaneous request, and at an overloaded requests has a median response time also about x worse, but that becomes a less tolerable ms. does also graphing the th, th, and th percentile tell us much? eh, i think the lumpiness is still just standard shared-architecture variability. the rate of “getting worse” as we add more overloaded connections is actually a bit better than it was on our t .medium, but since it already starts out so much slower, we’ll just call it a wash. (on t .medium, th percentile without load is ms and under an overloaded connections . s. on this single standard- x, it’s . s and . s). i’m not sure how much these charts with various percentiles on them tell us, i’ll not include them for every architecture hence. standard- x, dynos: under load ok, realistically we already know you shouldn’t have just one standard- x dyno under that kind of load. you’d scale out, either manually or perhaps using something like the neat rails autoscale add-on. let’s measure with dynos. each is still running puma workers, with threads each. what do we expect? hm, treating each dyno as if it has only one cpu, we’d expect it to be able to handle traffic pretty levelly up to simultenous connections, distributed to dynos. it’s going to do worse after that, but up to there is still one puma worker per connection so it might get even worse after ? well… i think that actually is relatively flat from to simultaneous connections, except for lumpiness from variability. but lumpiness from variability is huge! we’re talking ms median measured at connection, up to ms measured median at , down to ms at . and then maybe yeah, a fairly shallow slope up to simutaneous connections than steeper. but it’s all fairly shallow slope compared to our base t .medium. at connections (after which we pretty much max out), the standard- x median of ms is only . times the median at conection. compared to the t .median increase of . times. as we’d expect, scaling out to dynos (with four cpus/ hyperthreads) helps us scale well — the problem is the baseline is so slow to begin (with very high bounds of variability making it regularly even slower). performance-m: under load a performance-m has . gb of memory. it only has one physical cpu, although two “vcpus” (two hyperthreads) — and these are all your apps, it is not shared. by testing under load, i demonstrated i could actually fit workers on there without any memory limit errors. but is there any point to doing that with only / cpus? under a bit of testing, it appeared not. the heroku puma docs recommend only processes with threads. you could do a whole little mini-experiment just trying to measure/optimize process/thread count on performance-m! we’ve already got too much data here, but in some experimentation it looked to me like processes with threads each performed better (and certainly no worse) than processes with threads — if you’ve got the ram just sitting there anyway (as we do), why not? i actually tested with puma processes with threads each. there is still a large amount of ram headroom we aren’t going to use even under load. what do we expect? well, with the “hyperthreads” perhaps it can handle simultaneous requests nearly as well as (or not?); after that, we expect it to degrade quickly same as our original t .medium did. it an handle connections slightly better than you’d expect if there really was only cpu, so i guess a hyperthread does give you something. then the slope picks up, as you’d expect; and it looks like it does get steeper after simultaneous connections, yup. performance-l: under load a performance-l ($ /month) costs twice as much as a performance-m ($ /month), but has far more than twice as much resources. performance-l has a whopping gb of ram compared to performance-m’s . gb; and performance-l has real cpus/hyperthreads available to use (visible using the nproc technique in the heroku puma article. because we have plenty of ram to do so, we’re going to run worker processes to match our original t .medium’s. we still ran with threads, just cause it seems like maybe you should never run a puma worker with only one thread? but who knows, maybe workers with thread each would perform better; plenty of room (but not plenty of my energy) for yet more experimentation. what do we expect? the graph should be pretty flat up to simultaneous connections, then it should start getting worse, pretty evenly as simultaneous connections rise all the way up to . it is indeed pretty flat up to simultaneous connections. then up to it’s still not too bad — median at is only ~ . median at (!). then it gets worse after (oh yeah, hyperthreads?). but the slope is wonderfully shallow all the way. even at simultaneous connections, the median response time of ms is only . x what it was at one connection. (in our original t .medium, at simultaneous connections median response time was over x what it was at connection). this thing is indeed a monster. summary comparison: under load we showed a lot of graphs that look similar, but they all had different sclaes on the y-axis. let’s plot median response times under load of all architectures on the same graph, and see what we’re really dealing with. the blue t .medium is our baseline, what we have now. we can see that there isn’t really a similar heroku option, we have our choice of better or worse. the performance-l is just plain better than what we have now. it starts out performing about the same as what we have now for or simultaneous connections, but then scales so much flatter. the performance-m also starts out about thesame, but sccales so much worse than even what we have now. (it’s that real cpu instead of , i guess?). the standard- x scaled to dynos… has it’s own characteristics. it’s baseline is pretty terrible, it’s to times as slow as what we have now even not under load. but then it scales pretty well, since it’s dynos after all, it doesn’t get worse as fast as performance-m does. but it started out so bad, that it remains far worse than our original t .medium even under load. adding more dynos to standard- x will help it remain steady under even higher load, but won’t help it’s underlying problem that it’s just slower than everyone else. discussion: thoughts and surprises i had been thinking of a t .medium (even with burst) as “typical” (it is after all much slower than my macbook), and has been assuming (in retrospect with no particular basis) that a heroku standard dyno would perform similarly. most discussion and heroku docs, as well as the naming itself, suggest that a ‘standard’ dyno is, well, standard, and performance dynos are for “super scale, high traffic apps”, which is not me. but in fact, heroku standard dynos are much slower and more variable in performance than a bursting t .medium. i suspect they are slower than other options you might consider non-heroku “typical” options. my conclusion is honestly that “standard” dynos are really “for very fast, well-optimized apps that can handle slow and variable cpu” and “performance” dynos are really “standard, matching the cpu speeds you’d get from a typical non-heroku option”. but this is not how they are documented or usually talked about. are other people having really different experiences/conclusions than me? if so, why, or where have i gone wrong? this of course has implications for estimating your heroku budget if considering switching over. :( if you have a well-optimized fast app, say even th percentile is ms (on bursting t .medium), then you can handle standard slowness — so what your th percentile is now ms (and during some time periods even much slower, s or worse, due to variability). that’s not so bad for a th percentile. one way to get a very fast is of course caching. there is lots of discussion of using caching in rails, sometimes the message (explicit or implicit) is “you have to use lots of caching to get reasonable performance cause rails is so slow.” what if many of these people are on heroku, and it’s really you have to use lots of caching to get reasonable performance on heroku standard dyno?? i personally don’t think caching is maintenance free; in my experience properly doing cache invalidation and dealing with significant processing spikes needed when you choose to invalidate your entire cache (cause cached html needs to change) lead to real maintenance/development cost. i have not needed caching to meet my performance goals on present architecture. everyone doesn’t necessarily have the same performance goals/requirements. mine of a low-traffic non-commercial site are are maybe more modest, i just need users not to be super annoyed. but whatever your performance goals, you’re going to have to spend more time on optimization on a heroku standard than something with much faster cpu — like a standard affordable mid-tier ec . am i wrong? one significant factor on heroku standard dyno performance is that they use shared/multi-tenant infrastructure. i wonder if they’ve actually gotten lower performance over time, as many customers (who you may be sharing with) have gotten better at maximizing their utilization, so the shared cpus are typically more busy? like a frog boiling, maybe nobody noticed that standard dynos have become lower performance? i dunno, brainstorming. or maybe there are so many apps that start on heroku instead of switcching from somewhere else, that people just don’t realize that standard dynos are much slower than other low/mid-tier options? i was expecting to pay a premium for heroku — but even standard- x’s are a significant premium over paying for t .medium ec yourself, one i found quite reasonable…. performance dynos are of course even more premium. i had a sort of baked-in premise that most rails apps are “io-bound”, they spend more time waiting on io than using cpu. i don’t know where i got that idea, i heard it once a long time ago and it became part of my mental model. i now do not believe this is true true of my app, and i do not in fact believe it is true of most rails apps in . i would hypothesize that most rails apps today are in fact cpu-bound. the performance-m dyno only has one cpu. i had somehow also been assuming that it would have two cpus — i’m not sure why, maybe just because at that price! it would be a much better deal with two cpus. instead we have a huge jump from $ performance-m to $ performance-l that has x the cpus and ~ x the ram. so it doesn’t make financial sense to have more than one performance-m dyno, you might as well go to performance-l. but this really complicates auto-scaling, whether using heroku’s feature , or the awesome rails autoscale add-on. i am not sure i can afford a performance-l all the time, and a performance-m might be sufficient most of the time. but if % of the time i’m going to need more (or even %, or even unexpectedly-mentioned-in-national-media), it would be nice to set things up to autoscale up…. i guess to financially irrational or more performance-m’s? :( the performance-l is a very big machine, that is significantly beefier than my current infrastructure. and has far more ram than i need/can use with only physical cores. if i consider standard dynos to be pretty effectively low tier (as i do), heroku to me is kind of missing mid-tier options. a cpu option at . g or g of ram would make a lot of sense to me, and actually be exactly what i need… really i think performance-m would make more sense with cpus at it’s existing already-premium price point, and to be called a “performance” dyno. . maybe heroku is intentionally trying set options to funnel people to the highest-priced performance-l. conclusion: what are we going to do? in my investigations of heroku, my opinion of the developer ux and general service quality only increases. it’s a great product, that would increase our operational capacity and reliability, and substitute for so many person-hours of sysadmin/operational time if we were self-managing (even on cloud architecture like ec ). but i had originally been figuring we’d use standard dynos (even more affordably, possibly auto-scaled with rails autoscale plugin), and am disappointed that they end up looking so much lower performance than our current infrastructure. could we use them anyway? response time going from ms to ms — hey, ms is still fine, even if i’m sad to lose those really nice numbers i got from a bit of optimization. but this app has a wide long-tail ; our th percentile going from ms to s, our th percentile going from ms to . s and our th going from . s to . s — a lot harder to swallow. especially when we know that due to standard dyno variability, a slow-ish page that on my present architecture is reliably . s, could really be anywhere from to (!) on heroku. i would anticipate having to spend a lot more developer time on optimization on heroku standard dynos — or, i this small over-burdened non-commercial shop, not prioritizing that (or not having the skills for it), and having our performance just get bad. so i’m really reluctant to suggest moving our app to heroku with standard dynos. a performance-l dyno is going to let us not have to think about performance any more than we do now, while scaling under high-traffic better than we do now — i suspect we’d never need to scale to more than one performance-l dyno. but it’s pricey for us. a performance-m dyno has a base-speed that’s fine, but scales very poorly and unaffordably. doesn’t handle an increase in load very well as one dyno, and to get more cpus you have to pay far too much (especially compared to standard dynos i had been assuming i’d use). so i don’t really like any of my options. if we do heroku, maybe we’ll try a performance-m, and “hope” our traffic is light enough that a single one will do? maybe with rails autoscale for traffic spikes, even though performance-m dynos isn’t financially efficient? if we are scaling to (or more!) performance-m’s more than very occasionally, switch to performance-l, which means we need to make sure we have the budget for it? jrochkind general leave a comment november , november , deep dive: moving ruby projects from travis to github actions for ci so this is one of my super wordy posts, if that’s not your thing abort now, but some people like them. we’ll start with a bit of context, then get to some detailed looks at github actions features i used to replace my travis builds, with example config files and examination of options available. for me, by “continuous integration” (ci), i mostly mean “running automated tests automatically, on your code repo, as you develop”, on every pr and sometimes with scheduled runs. other people may mean more expansive things by “ci”. for a lot of us, our first experience with ci was when travis-ci started to become well-known, maybe years ago or so. travis was free for open source, and so darn easy to set up and use — especially for rails projects, it was a time when it still felt like most services focused on docs and smooth fit for ruby and rails specifically. i had heard of doing ci, but as a developer in a very small and non-profit shop, i want to spend time writing code not setting up infrastructure, and would have had to get any for-cost service approved up the chain from our limited budget. but it felt like i could almost just flip a switch and have travis on ruby or rails projects working — and for free! free for open source wasn’t entirely selfless, i think it’s part of what helped travis literally define the market. (btw, i think they were the first to invent the idea of a “badge” url for a github readme?) along with an amazing developer ux (which is today still a paragon), it just gave you no reason not to use it. and then once using it, it started to seem insane to not have ci testing, nobody would ever again want to develop software without the build status on every pr before merge. travis really set a high bar for ease of use in a developer tool, you didn’t need to think about it much, it just did what you needed, and told you what you needed to know in it’s read-outs. i think it’s an impressive engineering product. but then. end of an era travis will no longer be supporting open source projects with free ci. the free open source travis projects originally ran on travis-ci.org, with paid commercial projects on travis-ci.com. in may , they announced they’d be unifying these on travis-ci.com only, but with no announced plan that the policy for free open source would change. this migration seemed to proceed very slowly though. perhaps because it was part of preparing the company for a sale, in jan it was announced private equity firm idera had bought travis. at the time the announcement said “we will continue to maintain a free, hosted service for open source projects,” but knowing what “private equity” usually means, some were concerned for the future. (hn discussion). while the faq on the migration to travis-ci.com still says that travis-ci.org should remain reliable until projects are fully migrated, in fact over the past few months travis-ci.org projects largely stopped building, as travis apparently significantly reduced resources on the platform. some people began manually migrating their free open source projects to travis-ci.com where builds still worked. but, while the faq also still says “will travis ci be getting rid of free users? travis ci will continue to offer a free tier for public or open-source repositories on travis-ci.com” — in fact, travis announced that they are ending the free service for open source. the “free tier” is a limited trial (available not just to open source), and when it expires, you can pay, or apply to a special program for an extension, over and over again. they are contradicting themselves enough that while i’m not sure exactly what is going to happen, but no longer trust them as a service. enter github actions i work mostly on ruby and rails projects. they are all open source, almost all of them use travis. so while (once moved to travis-ci.com) they are all currently working, it’s time to start moving them somewhere else, before i have dozens of projects with broken ci and still don’t know how to move them. and the new needs to be free — many of these projects are zero-budget old-school “volunteer” or “informal multi-institutional collaboration” open source. there might be several other options, but the one i chose is github actions — my sense that it had gotten mature enough to start approaching travis level of polish, and all of my projects are github-hosted, and github actions is free for unlimited use for open source. (pricing page; aug announcement of free for open source). and we are really fortunate that it became mature and stable in time for travis to withdraw open source support (if travis had been a year earlier, we’d be in trouble). github actions is really powerful. it is built to do probably way more than travis does, definitely way beyond “automated testing” to various flows for deployment and artifact release, to really just about any kind of process for managing your project you want. the logic you can write almost unlimited, all running on github’s machines. as a result though…. i found it a bit overwhelming to get started. the github actions docs are just overwhelmingly abstract, there is so much there, you can almost anything — but i don’t actually want to learn a new platform, i just want to get automated test ci for my ruby project working! there are some language/project speccific guides available, for node.js, python, a few different java setups — but not for ruby or rails! my how rails has fallen, from when most services like this would be focusing on rails use cases first. :( there are some third part guides available that might focus on ruby/rails, but one of the problems is that actions has been evolving for a few years with some pivots, so it’s easy to find outdated instructions. one i found helpful orientation was this drifting ruby screencast. this screencast showed me there is a kind of limited web ui with integrated docs searcher — but i didn’t end up using it, i just created the text config file by hand, same as i would have for travis. github provides templates for “ruby” or “ruby gem”, but the drifting ruby sccreencast said “these won’t really work for our ruby on rails application so we’ll have to set up one manually”, so that’s what i did too. ¯\_(ツ)_/¯ but the cost of all the power github actions provides is… there are a lot more switches and dials to understand and get right (and maintain over time and across multiple projects). i’m not someone who likes copy-paste without understanding it, so i spent some time trying to understand the relevant options and alternatives; in the process i found some things i might have otherwise copy-pasted from other people’s examples that could be improved. so i give you the results of my investigations, to hopefully save you some time, if wordy comprehensive reports are up your alley. a simple test workflow: ruby gem, test with multiple ruby versions here’s a file for a fairly simple test workflow. you can see it’s in the repo at .github/workflows. the name of the file doesn’t matter — while this one is called ruby.yml, i’ve since moved over to naming the file to match the name: key in the workflow for easier traceability, so i would have called it ci.yml instead. triggers you can see we say that this workflow should be run on any push to master branch, and also for any pull_request at all. many other examples i’ve seen define pull_request: branches: ["main"], which seems to mean only run on pull requests with main as the base. while that’s most of my pr’s, if there is ever a pr that uses another branch as a base for whatever reason, i still want to run ci! while hypothetically you should be able leave branches out to mean “any branch”, i only got it to work by explicitly saying branches: ["**"] matrix for this gem, we want to run ci on multiple ruby versions. you can see we define them here. this works similarly to travis matrixes. if you have more than one matrix variable defined, the workflow will run for every combination of variables (hence the name “matrix”). matrix: ruby: [ ' . . ', ' . . ', ' . . ', ' . . ', 'jruby- . . . ', 'jruby- . . . ' ] in a given run, the current value of the matrix variables is available in github actions “context”, which you can acccess as eg ${{ matrix.ruby }}. you can see how i use that in the name, so that the job will show up with it’s ruby version in it. name: ruby ${{ matrix.ruby }} ruby install while github itself provides an action for ruby install, it seems most people are using this third-party action. which we reference as `ruby/setup-ruby@v `. you can see we use the matrix.ruby context to tell the setup-ruby action what version of ruby to install, which works because our matrix values are the correct values recognized by the action. which are documented in the readme, but note that values like jruby-head are also supported. note, although it isn’t clearly documented, you can say . to mean “latest available . .x” (rather than it meaning “ . . ”), which is hugely useful, and i’ve switched to doing that. i don’t believe that was available via travis/rvm ruby install feature. for a project that isn’t testing under multiple rubies, if we left out the with: ruby-version, the action will conveniently use a .ruby-version file present in the repo. note you don’t need to put a gem install bundler into your workflow yourself, while i’m not sure it’s clearly documented, i found the ruby/setup-ruby action would do this for you (installing the latest available bundler, instead of using whatever was packaged with ruby version), btw regardless of whether you are using the bundler-cache feature (see below). note on how matrix jobs show up to github with travis, testing for multiple ruby or rails versions with a matrix, we got one (or, well, actually two) jobs showing up on the github pr: each of those lines summaries a collection of matrix jobs (eg different ruby versions). if any of the individual jobs without the matrix failed, the whole build would show up as failed. success or failure, you could click on “details” to see each job and it’s status: i thought this worked pretty well — especially for “green” builds i really don’t need to see the details on the pr, the summary is great, and if i want to see the details i can click through, great. with github actions, each matrix job shows up directly on the pr. if you have a large matrix, it can be… a lot. some of my projects have way more than . on pr: maybe it’s just because i was used to it, but i preferred the travis way. (this also makes me think maybe i should change the name key in my workflow to say eg ci: ruby . . to be more clear? oops, tried that, it just looks even weirder in other gh contexts, not sure.) oh, also, that travis way of doing the build twice, once for “pr” and once for “push”? github actions doesn’t seem to do that, it just does one, i think corresponding to travis “push”. while the travis feature seemed technically smart, i’m not sure i ever actually saw one of these builds pass while the other failed in any of my projects, i probably won’t miss it. badge did you have a readme badge for travis? don’t forget to swap it for equivalent in github actions. the image url looks like: https://github.com/$owner/$repository/workflows/$workflow_name/badge.svg?branch=master, where $workflow_name of course has to be url-escaped if it ocntains spaces etc. the github page at https://github.com/owner/repo/actions, if you select a particular workflow/branch, does, like travis, give you a badge url/markdown you can copy/paste if you click on the three-dots and then “create status badge”. unlike travis, what it gives you to copy/paste is just image markdown, it doesn’t include a link. but i definitely want the badge to link to viewing the results of the last build in the ui. so i do it manually. limit to the speciifc workflow and branch that you made the badge for in the ui then just copy and paste the url from the browser. a bit confusing markdown to construct manually, here’s what it ended up looking like for me: [![ci status](https://github.com/jrochkind/attr_json/workflows/ci/badge.svg?branch=master)% d(https://github.com/jrochkind/attr_json/actions?query=workflow% aci+branch% amaster) view raw gh_badge_markdown_example.txt hosted with ❤ by github i copy and paste that from an existing project when i need it in a new one. :shrug: require ci to merge pr? however, that difference in how jobs show up to github, the way each matrix job shows up separately now, has an even more negative impact on requiring ci success to merge a pr. if you want to require that ci passes before merging a pr, you configure that at https://github.com/acct/project/settings/branches under “branch protection rules”.when you click “add rule”, you can/must choose which jobs are “required”. for travis, that’d be those two “master” jobs, but for the new system, every matrix job shows up separately — in fact, if you’ve been messing with job names trying to get it right as i have, you have any job name that was ever used in the last days, and they don’t have the github workflow name appended to them or anything (another reason to put github workflow name in the job name?). but the really problematic part is that if you edit your list of jobs in the matrix — adding or removing ruby versions as one does, or even just changing the name that shows up for a job — you have to go back to this screen to add or remove jobs as a “required status check”. that seems really unworkable to me, i’m not sure how it hasn’t been a major problem already for users. it would be better if we could configure “all the checks in the workflow, whatever they may be”, or perhaps best of all if we could configure a check as required in the workflow yml file, the same place we’re defining it, just a required_before_merge key you could set to true or use a matrix context to define or whatever. i’m currently not requiring status checks for merge on most of my projects (even though i did with travis), because i was finding it unmanageable to keep the job names sync’d, especially as i get used to github actions and kept tweaking things in a way that would change job names. so that’s a bit annoying. fail_fast: false by default, if one of the matrix jobs fails, github acitons will cancel all remaining jobs, not bother to run them at all. after all, you know the build is going to fail if one job fails, what do you need those others for? well, for my use case, it is pretty annoying to be told, say, “job for ruby . . failed, we can’t tell you whether the other ruby versions would have passed or failed or not” — the first thing i want to know is if failed on all ruby versions or just . . , so now i’d have to spend extra time figuring that out manually? no thanks. so i set `fail_fast: false` on all of my workflows, to disable this behavior. note that travis had a similar (opt-in) fast_finish feature, which worked subtly different: travis would report failure to github on first failure (and notify, i think), but would actually keep running all jobs. so when i saw a failure, i could click through to ‘details’ to see which (eg) ruby versions passed, from the whole matrix. this does work for me, so i’d chose to opt-in to that travis feature. unfortunately, the github actions subtle difference in effect makes it not desirable to me. note you may see some people referencing a github actions continue-on-error feature. i found the docs confusing, but after experimentation what this really does is mark a job as successful even when it fails. it shows up in all gh ui as succeeeded even when it failed, the only way to know it failed would be to click through to the actual build log to see failure in the logged console. i think “continue on error” is a weird name for this; it is not useful to me with regard to fine-tuning fail-fast; or honestly in any other use case i can think of that i have. bundle cache? bundle install can take + seconds, and be a significant drag on your build (not to mention a lot of load on rubygems servers from all these builds). so when travis introduced a feature to cache: bundler: true, it was very popular. true to form, github actions gives you a generic caching feature you can try to configure for your particular case (npm, bundler, whatever), instead of an out of the box feature “just do the right thing you for bundler, you figure it out”. the ruby/setup-ruby third-party action has a built-in feature to cache bundler installs for you, but i found that it does not work right if you do not have a gemfile.lock checked into the repo. (ie, for most any gem, rather than app, project). it will end up re-using cached dependencies even if there are new releases of some of your dependencies, which is a big problem for how i use ci for a gem — i expect it to always be building with latest releases of dependencies, so i can find out of one breaks the build. this may get fixed in the action. if you have an app (rather than gem) with a gemfile.lock checked into repo, the bundler-cache: true feature should be just fine. otherwise, github has some suggestions for using the generic cache feature for ruby bundler (search for “ruby – bundler” on this page) — but i actually don’t believe they will work right without a gemfile.lock checked into the repo either. starting from that example, and using the restore-keys feature, i think it should be possible to design a use that works much like travis’s bundler cache did, and works fine without a checked-in gemfile.lock. we’d want it to use a cache from the most recent previous (similar job), and then run bundle install anyway, and then cache the results again at the end always to be available for the next run. but i haven’t had time to work that out, so for now my gem builds are simply not using bundler caching. (my gem builds tend to take around seconds to do a bundle install, so that’s in every build now, could be worse). update nov : the ruby/ruby-setup action should be fixed to properly cache-bust when you don’t have a gemfile.lock checked in. if you are using a matrix for ruby version, as below, you must set the ruby version by setting the bundle_gemfile env variable rather than the way we did it below, and there is is a certain way github action requires/provides you do that, it’s not just export. see the issue in ruby/ruby-setup project. notifications: not great travis has really nice defaults for notifications: the person submitting the pr would get an email generally only on status changes (from pass to fail or fail to pass) rather than on every build. and travis would even figure out what email to send to based on what email you used in your git commits. (originally perhaps a workaround to lack of github api at travis’ origin, i found it a nice feature). and then travis has sophisticated notification customization available on a per-repo basis. github notifications are unfortunately much more basic and limited. the only notification settings avaialable are for your entire account at https://github.com/settings/notifications, “github actions”. so they apply to all github workflows in all projects, there are no workflow- or project-specific settings. you can set to receive notification via web push or email or both or neither. you can receive notifications for all builds or only failed builds. that’s it. the author of a pr is the one who receives the notifications, same as in travis. you will get notifications for every single build, even repeated successes or failures in a series. i’m not super happy with the notification options. i may end up just turning off github actions notifications entirely for my account. hypothetically, someone could probably write a custom github action to give you notifications exactly how travis offered — after all, travis was using public gh api that should be available to any other author, and i think should be usable from within an action. but when i started to think through it, while it seemed an interesting project, i realized it was definitely beyond the “spare hobby time” i was inclined to give to it at present, especially not being much of a js developer (the language of custom gh actions, generally). (while you can list third-party actions on the github “marketplace”, i don’t think there’s a way to charge for them). . there are custom third-party actions available to do things like notify slack for build completion; i haven’t looked too much into any of them, beyond seeing that i didn’t see any that would be “like travis defaults”. a more complicated gem: postgres, and rails matrix let’s move to a different example workflow file, in a different gem. you can see i called this one ci.yml, matching it’s name: ci, to have less friction for a developer (including future me) trying to figure out what’s going on. this gem does have rails as a dependency and does test against it, but isn’t actually a rails engine as it happens. it also needs to test against postgres, not just sqlite . scheduled builds at one point travis introduced a feature for scheduling (eg) weekly builds even when no pr/commit had been made. i enthusiastically adopted this for my gem projects. why? gem releases are meant to work on a variety of different ruby versions and different exact versions of dependencies (including rails). sometimes a new release of ruby or rails will break the build, and you want to know about that and fix it. with ci builds happening only on new code, you find out about this with some random new code that is unlikely to be related to the failure; and you only find out about it on the next “new code” that triggers a build after a dependency release, which on some mature and stable gems could be a long time after the actual dependency release that broke it. so scheduled builds for gems! (i have no purpose for scheduled test runs on apps). github actions does have this feature. hooray. one problem is that you will receive no notification of the result of the scheduled build, success or failure. :( i suppose you could include a third-party action to notify a fixed email address or slack or something else; not sure how you’d configure that to apply only to the scheduled builds and not the commit/pr-triggered builds if that’s what you wanted. (or make an custom action to file a gh issue on failure??? but make sure it doesn’t spam you with issues on repeated failures). i haven’t had the time to investigate this yet. also oops just noticed this: “in a public repository, scheduled workflows are automatically disabled when no repository activity has occurred in days.” which poses some challenges for relying on scheduled builds to make sure a stable slow-moving gem isn’t broken by dependency updates. i definitely am committer on gems that are still in wide use and can go - + months without a commit, because they are mature/done. i still have it configured in my workflow; i guess even without notifications it will effect the “badge” on the readme, and… maybe i’ll notice? very far from ideal, work in progress. :( rails matrix ok, this one needs to test against various ruby versions and various rails versions. a while ago i realized that an actual matrix of every ruby combined with every rails was far too many builds. fortunately, github actions supports the same kind of matrix/include syntax as travis, which i use. matrix: include: - gemfile: rails_ _ ruby: . - gemfile: rails_ _ ruby: . i use the appraisal gem to handle setting up testing under multiple rails versions, which i highly recommend. you could use it for testing variant versions of any dependencies, i use it mostly for varying rails. appraisal results in a separate gemfile committed to your repo for each (in my case) rails version, eg ./gemfiles/rails_ _ .gemfile. so those values i use for my gemfile matrix key are actually portions of the gemfile path i’m going to want to use for each job. then we just need to tell bundler, in a given matrix job, to use the gemfile we specified in the matrix. the old-school way to do this is with the bundle_gemfile environmental variable, but i found it error-prone to make sure it stayed consistently set in each workflow step. i found that the newer (although not that new!) bundle config set gemfile worked swimmingly! i just set it before the bundle install, it stays set for the rest of the run including the actual test run. steps: # [...] - name: bundle install run: | bundle config set gemfile "${github_workspace}/gemfiles/${{ matrix.gemfile }}.gemfile" bundle install --jobs --retry note that single braces are used for ordinary bash syntax to reference the env variable ${github_workspace}, but double braces for the github actions context value interpolation ${{ matrix.gemfile }}. works great! oh, note how we set the name of the job to include both ruby and rails matrix values, important for it showing up legibly in github ui: name: ${{ matrix.gemfile }}, ruby ${{ matrix.ruby }}. because of how we constructed our gemfile matrix, that shows up with job names rails_ _ , ruby . . still not using bundler caching in this workflow. as before, we’re concerned about the ruby/setup-ruby built-in bundler-cache feature not working as desired without a gemfile.lock in the repo. this time, i’m also not sure how to get that feature to play nicely with the variant gemfiles and bundle config set gemfile. github actions makes you put together a lot more pieces together yourself compared to travis, there are still things i just postponed figuring out for now. update jan : the ruby/setup-ruby action now includes a ruby version matrix example in it’s readme. https://github.com/ruby/setup-ruby#matrix-of-gemfiles it does require you use the bundle_gemfile env variable, rather than the bundle config set gemfile command i used here. this should ordinarily be fine, but is something to watch out for in case other instructions you are following tries to use bundle config set gemfile instead, for reasons or not. postgres this project needs to build against a real postgres. that is relatively easy to set up in github actions. postgres normally by default allows connections on localhost without a username/password set, and my past builds (in travis or locally) took advantage of this to not bother setting one, which then the app didn’t have to know about. but the postgres image used for github actions doesn’t allow this, you have to set a username/password. so the section of the workflow that sets up postgres looks like: jobs: tests: services: db: image: postgres: . env: postgres_user: postgres postgres_password: postgres ports: [' : '] is the default postgres port, we need to set it and map it so it will be available as expected. note you also can specify whatever version of postgres you want, this one is intentionally testing on one a bit old. ok now our rails app that will be executed under rspec needs to know that username and password to use in it’s postgres connection; when before it connected without a username/password. that env under the postgres service image is not actually available to the job steps. i didn’t find any way to dry the username/password in one place, i had to repeat it in another env block, which i put at the top level of the workflow so it would apply to all steps. and then i had to alter my database.yml to use those env variables, in the test environment. on a local dev machine, if your postgres doens’t have a username/password requirement and you don’t set the env variables, it keeps working as before. i also needed to add host: localhost to the database.yml; before, the absence of the host key meant it used a unix-domain socket (filesystem-located) to connect to postgres, but that won’t work in the github actions containerized environment. note, there are things you might see in other examples that i don’t believe you need: no need for an apt-get of pg dev libraries. i think everything you need is on the default gh actions images now. some examples i’ve seen do a thing with options: --health-cmd pg_isready, my builds seem to be working just fine without it, and less code is less code to maintain. allow_failures in travis, i took advantage of the travis allow_failures key in most of my gems. why? i am testing against various ruby and rails versions; i want to test against *future* (pre-release, edge) ruby and rails versions, cause its useful to know if i’m already with no effort passing on them, and i’d like to keep passing on them — but i don’t want to mandate it, or prevent pr merges if the build fails on a pre-release dependency. (after all, it could very well be a bug in the dependency too!) there is no great equivalent to allow_failures in github actions. (note again, continue_on_error just makes failed jobs look identical to successful jobs, and isn’t very helpful here). i investigated some alternatives, which i may go into more detail on in a future post, but on one project i am trying a separate workflow just for “future ruby/rails allowed failures” which only checks master commits (not prs), and has a separate badge on readme (which is actually pretty nice for advertising to potential users “yeah, we already work on rails edge/ . .rc !”). main downside there is having to copy/paste synchronize what’s really the same workflow in two files. a rails app i have many more number of projects i’m a committer on that are gems, but i spend more of my time on apps, one app in specific. so here’s an example github actions ci workflow for a rails app. it mostly remixes the features we’ve already seen. it doesn’t need any matrix. it does need a postgres. it does need some “os-level” dependencies — the app does some shell-out to media utilities like vips and ffmpeg, and there are integration tests that utilize this. easy enough to just install those with apt-get, works swimmingly. - name: install apt dependencies run: | sudo apt-get -y install libvips-tools ffmpeg mediainfo update nov: my apt-get that worked for a couple weeks started failing for some reason on trying to install a libpulse dependency of one of those packages, the solution was doing a sudo apt-get update before the sudo apt-get install. i guess this is always good practice? (that forum post also uses apt install and apt update instead of apt-get install and apt-get update, that i can’t tell you much about, i’m really not a linux admin). in addition to the bundle install, a modern rails app using webpacker needs yarn install. this just worked for me — no need to include lines for installing npm itself or yarn or any yarn dependencies, although some examples i find online have them. (my yarn installs seem to happen in ~ seconds, so i’m not motivated to try to figure out caching for yarn). and we need to create the test database in the postgres, which i do with rails_env=test bundle exec rails db:create — typical rails test setup will then automatically run migrations if needed. there might be other (better?) ways to prep the database, but i was having trouble getting rake db:prepare to work, and didn’t spend the time to debug it, just went with something that worked. - name: set up app run: | rails_env=test bundle exec rails db:create yarn install rails test setup usually ends up running migrations automatically is why i think this worked alone, but you could also throw in a rails_env=test bundle exec rake db:schema:load if you wanted. under travis i had to install chrome with addons: chrome: stable to have it available to use with capybara via the webdrivers gem. no need for installing chrome in github actions, some (recent-ish?) version of it is already there as part of the standard github actions build image. in this workflow, you can also see a custom use of the github “cache” action to cache a solr install that the test setup automatically downloads and sets up. in this case the cache doesn’t actually save us any build time, but is kinder on the apache foundation servers we are downloading from with every build otherwise (and have gotten throttled from in the past). conclusion github aciton sis a really impressively powerful product. and it’s totally going to work to replace travis for me. it’s also probably going to take more of my time to maintain. the trade-off of more power/flexibility and focusing on almost limitless use cases is more things th eindividual project has to get right for their use case. for instance figuring out the right configuration to get caching for bundler or yarn right, instead of just writing cache: { yarn: true, bundler: true}. and when you have to figure it out yourself, you can get it wrong, which when you are working on many projects at once means you have a bunch of places to fix. the amazingness of third-party action “marketplace” means you have to figure out the right action to use (the third-party ruby/setup-ruby instead of the vendor’s actions/setup-ruby), and again if you change your mind about that you have a bunch of projects to update. anyway, it is what it is — and i’m grateful to have such a powerful and in fact relatively easy to use service available for free! i could not really live without ci anymore, and won’t have to! oh, and github actions is giving me way more (free) simultaneous parallel workers than travis ever did, for my many-job builds! jrochkind general comments november , january , unexpected performance characteristics when exploring migrating a rails app to heroku i work at a small non-profit research institute. i work on a rails app that is a “digital collections” or “digital asset management” app. basically it manages and provides access (public as well as internal) to lots of files and description about those files, mostly images. it’s currently deployed on some self-managed amazon ec instances (one for web, one for bg workers, one in which postgres is installed, etc). it gets pretty low-traffic in general web/ecommerce/rails terms. the app is definitely not very optimized — we know it’s kind of a ram hog, we know it has many actions whose response time is undesirable. but it works “good enough” on it’s current infrastructure for current use, such that optimizing it hasn’t been the highest priority. we are considering moving it from self-managed ec to heroku, largely because we don’t really have the capacity to manage the infrastructure we currently have, especially after some recent layoffs. our rails app is currently served by passenger on an ec t .medium ( g of ram). i expected the performance characteristics moving to heroku “standard” dynos would be about the same as they are on our current infrastructure. but was surprised to see some degradation: responses seem much slower to come back when deployed, mainly for our slowest actions. quick actions are just as quick on heroku, but slower ones (or perhaps actions that involve more memory allocations?) are much slower on heroku. the application instances seem to take more ram running on heroku dynos than they do on our ec (this one in particular mystifies me). i am curious if anyone with more heroku experience has any insight into what’s going on here. i know how to do profiling and performance optimization (i’m more comfortable with profiling cpu time with ruby-prof than i am with trying to profile memory allocations with say derailed_benchmarks). but it’s difficult work, and i wasn’t expecting to have to do more of it as part of a migration to heroku, when performance characteristics were acceptable on our current infrastructure. response times (cpu) again, yep, know these are fairly slow response times. but they are “good enough” on current infrastruture (ec t .medium), wasn’t expecting them to get worse on heroku (standard- x dyno, backed by heroku pg standard- ). fast pages are about the same, but slow pages (that create a lot of objects in memory?) are a lot slower. this is not load testing, i am not testing under high traffic or for current requests. this is just accessing demo versions of the app manually one page a time, to see response times when the app is only handling one response at a time. so it’s not about how many web workers are running or fit into ram or anything; one is sufficient. action existing ec t .medium heroku standard- x dyno slow reporting page that does a few very expensive sql queries, but they do not return a lot of objects. rails logging reports: allocations: ~ ms ~ ms (faster pg?) fast page with a few ar/sql queries returning just a few objects each, a few partials, etc. rails logging reports: allocations: - ms ~ ms a fairly small “item” page, rails logging reports: allocations: ~ ms ~ ms a medium size item page, loads a lot more ar models, has a larger byte size page response. allocations: ~ ms - ms one of our largest pages, fetches a lot of ar instances, does a lot of allocations, returns a very large page response. allocations: - ms - ms fast-ish responses (and from this limited sample, actually responses with few allocations even if slow waiting on io?) are about the same. but our slowest/highest allocating actions are ~ % slower on heroku? again, i know these allocations and response times are not great even on our existing infrastructure; but why do they get so much worse on heroku? (no, there were no heroku memory errors or swapping happening). ram use of an app instance we currently deploy with passenger (free), running workers on our gb t .medium. to compare apples to apples, deployed using passenger on a heroku standard- x. just one worker instance (because that’s actually all i can fit on a standard- x!), to compare size of a single worker from one infrastructure to the other. on our legacy infrastructure, on a server that’s been up for days of production traffic, passenger-status looks something like this: requests in queue: * pid: sessions: processed: uptime: d h m s cpu: % memory : m last used: s * pid: sessions: processed: uptime: d h m s cpu: % memory : m last used: s * pid: sessions: processed: uptime: d h m s cpu: % memory : m last used: m * pid: sessions: processed: uptime: d h m s cpu: % memory : m last used: h * pid: sessions: processed: uptime: d h m s cpu: % memory : m last used: h * pid: sessions: processed: uptime: d h m s cpu: % memory : m last used: h * pid: sessions: processed: uptime: d h m s cpu: % memory : m last used: h * pid: sessions: processed: uptime: d h m s cpu: % memory : m last used: h * pid: sessions: processed: uptime: d h m s cpu: % memory : m last used: h * pid: sessions: processed: uptime: d h m s cpu: % memory : m last used: h we can see, yeah, this app is low traffic, most of those workers don’t see a lot of use. the first worker, which has handled by far the most traffic has a private rss of m. (other workers having handled fewer requests much slimmer). kind of overweight, not sure where all that ram is going, but it is what it is. i could maybe hope to barely fit workers on a heroku standard- ( m) instance, if these sizes were the same on heroku. this is after a week of production use — if i restart passenger on a staging server, and manually access some of my largest, hungriest, most-allocating pages a few times, i can only see private rss use of like mb. however, on the heroku standard- x, with one passenger worker, using the heroku log-runtime-metrics feature to look at memory… private rss is i believe what should correspond to passenger’s report, and what heroku uses for memory capacity limiting… immediately after restarting my app, it’s at sample#memory_total= . mb sample#memory_rss= . mb. after manually accessing a few of my “hungriest” actions, i see: sample#memory_total= . mb sample#memory_rss= . mb . just a few manual requests not a week of production traffic, and % more ram than on my legacy ec infrastructure after a week of production traffic. actually approaching the limits of what can fit in a standard- x ( mb) dyno as just one worker. now, is heroku’s memory measurements being done differently than passenger-status does them? possibly. it would be nice to compare apples to apples, and passenger hypothetically has a service that would let you access passenger-status results from heroku… but unfortunately i have been unable to get it to work. (ideas welcome). other variations tried on heroku trying the heroku gaffneyc/jemalloc build-pack with heroku config:set jemalloc_enabled=true (still with passenger, one worker instance) doesn’t seem to have made any significant differences, maybe % ram savings or maybe it’s a just a fluke. switching to puma (puma with the experimental possibly memory-saving features turned on; just one worker with one thread), doesn’t make any difference in response time performance (none expected), but… maybe does reduce ram usage somehow? after a few sample requests of some of my hungriest pages, i see sample#memory_total= . mb sample#memory_rss= . mb, still more than my baseline, but not drastically so. (with or without jemalloc buildpack seems to make no difference). odd. so what should i conclude? i know this app could use a fitness regime; but it performs acceptably on current infrastructure. we are exploring heroku because of staffing capacity issues, hoping to not to have to do so much ops. but if we trade ops for having to spend much time on challenging (not really suitable for junior dev) performance optimization…. that’s not what we were hoping for! but perhaps i don’t know what i’m doing, and this haphapzard anecdotal comparison is not actually data and i shoudn’t conclude much from it? let me know, ideally with advice of how to do it better? or… are there reasons to expect different performance chracteristics from heroku? might it be running on underlying aws infrastructure that has less resources than my t .medium? or, starting to make guess hypotheses, maybe the fact that heroku standard tier does not run on “dedicated” compute resources means i should expect a lot more variance compared to my own t .medium, and as a result when deploying on heroku you need to optimize more (so the worst case of variance isn’t so bad) than when running on your own ec? that’s maybe just part of what you get with heroku, unless paying for performance dynos, it is even more important to have an good performing app? (yeah, i know i could use more caching, but that of course brings it’s own complexities, i wasn’t expecting to have to add it in as part of a heroku migration). or… i find it odd that it seems like slower (or more allocating?) actions are the ones that are worse. is there any reason that memory allocations would be even more expensive on a heroku standard dyno than on my own ec t .medium? and why would the app workers seem to use so much more ram on heroku than on my own ec anyway? any feedback or ideas welcome! jrochkind general comments october , posts navigation older posts bibliographic wilderness is a blog by jonathan rochkind about digital library services, ruby, and web development. contact search for: email subscription enter your email address to subscribe to this blog and receive notifications of new posts by email. join other followers email address: subscribe recent posts product management february , rails auto-scaling on heroku january , managed solr saas options january , gem authors, check your release sizes january , every time you decide to solve a problem with code… january , archives archives select month february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) april ( ) march ( ) december ( ) october ( ) september ( ) august ( ) june ( ) april ( ) march ( ) february ( ) december ( ) november ( ) october ( ) september ( ) august ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) november ( ) october ( ) september ( ) august ( ) july ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) september ( ) august ( ) june ( ) may ( ) march ( ) february ( ) january ( ) november ( ) october ( ) september ( ) august ( ) july ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) september ( ) august ( ) june ( ) may ( ) april ( ) march ( ) feeds rss - posts rss - comments recent comments jrochkind on rails auto-scaling on heroku adam (rails autoscale) on rails auto-scaling on heroku on catalogers, programmers, and user tasks – gavia libraria on broad categories from class numbers replacing marc – gavia libraria on linked data caution jrochkind on deep dive: moving ruby projects from travis to github actions for ci jrochkind on deep dive: moving ruby projects from travis to github actions for ci jrochkind on deep dive: moving ruby projects from travis to github actions for ci eregontp on deep dive: moving ruby projects from travis to github actions for ci top posts yes, product owner and technical lead need to be different people bootstrap to : changes in how font size, line-height, and spacing is done. or "what happened to $line-height-computed." dealing with legacy and externally loaded code in webpack(er) activerecord: atomic check-and-update through optimistic locking are you talking to heroku redis in cleartext or ssl? top clicks bibwild.files.wordpress.c… apidock.com/rails/activer… github.com/mperham/sideki… bibwild.files.wordpress.c… opensolr.com a blog by jonathan rochkind. all original content licensed cc-by. create a website or blog at wordpress.com add your thoughts here... (optional) post to cancel privacy & cookies: this site uses cookies. by continuing to use this website, you agree to their use. to find out more, including how to control cookies, see here: cookie policy large-scale celebfaces attributes (celeba) dataset large-scale celebfaces attributes (celeba) dataset ziwei liu ping luo xiaogang wang xiaoou tang multimedia laboratory, the chinese university of hong kong news - - two related datasets, celebamask-hq and celeba-spoof, have been released. - - if dropbox is not accessible, please download the dataset using google drive or baidu drive (password: rp s). details celebfaces attributes dataset (celeba) is a large-scale face attributes dataset with more than k celebrity images, each with attribute annotations. the images in this dataset cover large pose variations and background clutter. celeba has large diversities, large quantities, and rich annotations, including , number of identities, , number of face images, and landmark locations, binary attributes annotations per image. the dataset can be employed as the training and test sets for the following computer vision tasks: face attribute recognition, face detection, landmark (or facial part) localization, and face editing & synthesis. sample images downloads in-the-wild images align&cropped images landmarks annotations attributes annotations identities annotations evaluation train/val/test partitions if the above links are not accessable, you could download the dataset using baidu drive. for more details of the dataset, please refer to the paper "deep learning face attributes in the wild". agreement the celeba dataset is available for non-commercial research purposes only. all images of the celeba dataset are obtained from the internet which are not property of mmlab, the chinese university of hong kong. the mmlab is not responsible for the content nor the meaning of these images. you agree not to reproduce, duplicate, copy, sell, trade, resell or exploit for any commercial purposes, any portion of the images and any portion of derived data. you agree not to further copy, publish or distribute any portion of the celeba dataset. except, for internal use at a single site within the same organization it is allowed to make copies of the dataset. the mmlab reserves the right to terminate your access to the celeba dataset at any time. the face identities are released upon request for research purposes only. please contact us for details. citation @inproceedings{liu faceattributes, title = {deep learning face attributes in the wild}, author = {liu, ziwei and luo, ping and wang, xiaogang and tang, xiaoou}, booktitle = {proceedings of international conference on computer vision (iccv)}, month = {december}, year = { } } related datasets lfwa+ dataset celebamask-hq dataset celeba-spoof dataset contact please contact ping luo and ziwei liu for questions about the dataset. rails auto-scaling on heroku – bibliographic wilderness skip to content bibliographic wilderness menu about contact rails auto-scaling on heroku jrochkind general january , january , we are investigating moving our medium-small-ish rails app to heroku. we looked at both the rails autoscale add-on available on heroku marketplace, and the hirefire.io service which is not listed on heroku marketplace and i almost didn’t realize it existed. i guess hirefire.io doesn’t have any kind of a partnership with heroku, but still uses the heroku api to provide an autoscale service. hirefire.io ended up looking more fully-featured and lesser priced than rails autoscale; so the main service of this post is just trying to increase visibility of hirefire.io and therefore competition in the field, which benefits us consumers. background: interest in auto-scaling rails background jobs at first i didn’t realize there was such a thing as “auto-scaling” on heroku, but once i did, i realized it could indeed save us lots of money. i am more interested in scaling rails background workers than i a web workers though — our background workers are busiest when we are doing “ingests” into our digital collections/digital asset management system, so the work is highly variable. auto-scaling up to more when there is ingest work piling up can give us really nice inget throughput while keeping costs low. on the other hand, our web traffic is fairly low and probably isn’t going to go up by an order of magnitude (non-profit cultural institution here). and after discovering that a “standard” dyno is just too slow, we will likely be running a performance-m or performance-l anyway — which likely can handle all anticipated traffic on it’s own. if we have an auto-scaling solution, we might configure it for web dynos, but we are especially interested in good features for background scaling. there is a heroku built-in autoscale feature, but it only works for performance dynos, and won’t do anything for rails background job dynos, so that was right out. that could work for rails bg jobs, the rails autoscale add-on on the heroku marketplace; and then we found hirefire.io. pricing: pretty different hirefire as of now january , hirefire.io has pretty simple and affordable pricing. $ /month/heroku application. auto-scaling as many dynos and process types as you like. hirefire.io by default can only check into your apps metrics to decide if a scaling event can occur once per minute. if you want more frequent than that (up to once every seconds), you have to pay an additional $ /month, for $ /month/heroku application. even though it is not a heroku add-on, hirefire does advertise that they bill pro-rated to the second, just like heroku and heroku add-ons. rails autoscale rails autoscale has a more tiered approach to pricing that is based on number and type of dynos you are scaling. starting at $ /month for - standard dynos, the next tier up is $ for up to standard dynos, all the way up to $ (!) for to dynos. if you have performance dynos involved, from $ /month for - performance dynos, up to $ /month for up to performance dynos. for our anticipated uses… if we only scale bg dynos, i might want to scale from (low) or to (high) or standard dynos, so we’d be at $ /month. our web dynos are likely to be performance and i wouldn’t want/need to scale more than probably , but that puts us into performance dyno tier, so we’re looking at $ /month. this is of course significantly more expensive than hirefire.io’s flat rate. metric resolution since hirefire had an additional charge for finer than -minute resolution on checks for autoscaling, we’ll discuss resolution here in this section too. rails autoscale has same resolution for all tiers, and i think it’s generally seconds, so approximately the same as hirefire if you pay the extra $ for increased resolution. configuration let’s look at configuration screens to get a sense of feature-sets. rails autoscale web dynos to configure web dynos, here’s what you get, with default values: the metric rails autoscale uses for scaling web dynos is time in heroku routing queue, which seems right to me — when things are spending longer in heroku routing queue before getting to a dyno, it means scale up. worker dynos for scaling worker dynos, rails autoscale can scale dyno type named “worker” — it can understand ruby queuing libraries sidekiq, resque, delayed job, or que. i’m not certain if there are options for writing custom adapter code for other backends. here’s what the configuration options are — sorry these aren’t the defaults, i’ve already customized them and lost track of what defaults are. you can see that worker dynos are scaled based on the metric “number of jobs queued”, and you can tell it to only pay attention to certain queues if you want. hirefire hirefire has far more options for customization than rails autoscale, which can make it a bit overwhelming, but also potentially more powerful. web dynos you can actually configure as many heroku process types as you have for autoscale, not just ones named “web” and “worker”. and for each, you have your choice of several metrics to be used as scaling triggers. for web, i think queue time (percentile, average) matches what rails autoscale does, configured to percentile, , and is probably the best to use unless you have a reason to use another. (“rails autoscale tracks the th percentile queue time, which for most applications will hover well below the default threshold of ms.“) here’s what configuration hirefire makes available if you are scaling on “queue time” like rails autoscale, configuration may vary for other metrics. i think if you fill in the right numbers, you can configure to work equivalently to rails autoscale. worker dynos if you have more than one heroku process type for workers — say, working on different queues — hirefire can scale the independently, with entirely separate configuration. this is pretty handy, and i don’t think rails autoscale offers this. (update i may be wrong, rails autoscale says they do support this, so check on it yourself if it matters to you). for worker dynos, you could choose to scale based on actual “dyno load”, but i think this is probably mostly for types of processes where there isn’t the ability to look at “number of jobs”. a “number of jobs in queue” like rails autoscale does makes a lot more sense to me as an effective metric for scaling queue-based bg workers. hirefire’s metric is slightly difererent than rails autoscale’s “jobs in queue”. for recognized ruby queue systems (a larger list than rails autoscale’s; and you can write your own custom adapter for whatever you like), it actually measures jobs in queue plus workers currently busy. so queued+in-progress, rather than rails autoscale’s just queued. i actually have a bit of trouble wrapping my head around the implications of this, but basically, it means that hirefire’s “jobs in queue” metric strategy is intended to try to scale all the way to emptying your queue, or reaching your max scale limit, whichever comes first. i think this may make sense and work out at least as well or perhaps better than rails autoscale’s approach? here’s what configuration hirefire makes available for worker dynos scaling on “job queue” metric. since the metric isn’t the same as rails autosale, we can’t configure this to work identically. but there are a whole bunch of configuration options, some similar to rails autoscale’s. the most important thing here is that “ratio” configuration. it may not be obvious, but with the way the hirefire metric works, you are basically meant to configure this to equal the number of workers/threads you have on each dyno. i have it configured to because my heroku worker processes use resque, with resque_pool, configured to run resque workers on each dyno. if you use sidekiq, set ratio to your configured concurrency — or if you are running more than one sidekiq process, processes*concurrency. basically how many jobs your dyno can be concurrently working is what you should normally set for ‘ratio’. hirefire not a heroku plugin hirefire isn’t actually a heroku plugin. in addition to that meaning separate invoicing, there can be some other inconveniences. since hirefire only can interact with heroku api, for some metrics (including the “queue time” metric that is probably optimal for web dyno scaling) you have to configure your app to log regular statistics to heroku’s “logplex” system. this can add a lot of noise to your log, and for heroku logging add-ons that are tired based on number of log lines or bytes, can push you up to higher pricing tiers. if you use paperclip, i think you should be able to use the log filtering feature to solve this, keep that noise out of your logs and avoid impacting data log transfer limits. however, if you ever have cause to look at heroku’s raw logs, that noise will still be there. support and docs i asked a couple questions of both hirefire and rails autoscale as part of my evaluation, and got back well-informed and easy-to-understand answers quickly from both. support for both seems to be great. i would say the documentation is decent-but-not-exhaustive for both products. hirefire may have slightly more complete documentation. other features? there are other things you might want to compare, various kinds of observability (bar chart or graph of dynos or observed metrics) and notification. i don’t have time to get into the details (and didn’t actually spend much time exploring them to evaluate), but they seem to offer roughly similar features. conclusion rails autoscale is quite a bit more expensive than hirefire.io’s flat rate, once you get past rails autoscale’s most basic tier (scaling no more than standard dynos). it’s true that autoscaling saves you money over not, so even an expensive price could be considered a ‘cut’ of that, and possibly for many ecommerce sites even $ a month might a drop in the bucket (!)…. but this price difference is so significant with hirefire (which has flat rate regardless of dynos), that it seems to me it would take a lot of additional features/value to justify. and it’s not clear that rails autoscale has any feature advantage. in general, hirefire.io seems to have more features and flexibility. until , hirefire.io could only analyze metrics with -minute resolution, so perhaps that was a “killer feature”? honestly i wonder if this price difference is sustained by rails autoscale only because most customers aren’t aware of hirefire.io, it not being listed on the heroku marketplace? single-invoice billing is handy, but probably not worth $ + a month. i guess hirefire’s logplex noise is a bit inconvenient? or is there something else i’m missing? pricing competition is good for the consumer. and are there any other heroku autoscale solutions, that can handle rails bg job dynos, that i still don’t know about? update a day after writing djcp on a reddit thread writes: i used to be a principal engineer for the heroku add-ons program. one issue with hirefire is they request account level oauth tokens that essentially give them ability to do anything with your apps, where rails autoscaling worked with us to create a partnership and integrate with our “official” add-on apis that limits security concerns and are scoped to the application that’s being scaled. part of the reason for hirefire working the way it does is historical, but we’ve supported the endpoints they need to scale for “official” partners for years now. a lot of heroku customers use hirefire so please don’t think i’m spreading fud, but you should be aware you’re giving a third party very broad rights to do things to your apps. they probably won’t, of course, but what if there’s a compromise? “official” add-on providers are given limited scoped tokens to (mostly) only the actions / endpoints they need, minimizing blast radius if they do get compromised. you can read some more discussion at that thread. share this: twitter facebook tagged ruby published by jrochkind view all posts by jrochkind published january , january , post navigation previous post managed solr saas options next post product management thoughts on “rails auto-scaling on heroku” adam (rails autoscale) says: january , at : pm this is a well-done and fair comparison. i appreciate the write-up and feedback. thanks also for the correction regarding multiple worker processes. that’s on me for not having it documented (which i have since updated). jrochkind says: february , at : am as far as hirefire’s cpu load metric (available on any type of dyno), note that google suggests: > in a majority of cases (although certainly not in all), we’ve found that simply using cpu consumption as the signal for provisioning works well, for the following reasons… https://sre.google/sre-book/handling-overload/ leave a reply cancel reply enter your comment here... fill in your details below or click an icon to log in: email (required) (address never made public) name (required) website you are commenting using your wordpress.com account. ( log out / change ) you are commenting using your google account. ( log out / change ) you are commenting using your twitter account. ( log out / change ) you are commenting using your facebook account. ( log out / change ) cancel connecting to %s notify me of new comments via email. notify me of new posts via email. bibliographic wilderness is a blog by jonathan rochkind about digital library services, ruby, and web development. contact search for: email subscription enter your email address to subscribe to this blog and receive notifications of new posts by email. join other followers email address: subscribe recent posts product management february , rails auto-scaling on heroku january , managed solr saas options january , gem authors, check your release sizes january , every time you decide to solve a problem with code… january , archives archives select month february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) april ( ) march ( ) december ( ) october ( ) september ( ) august ( ) june ( ) april ( ) march ( ) february ( ) december ( ) november ( ) october ( ) september ( ) august ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) november ( ) october ( ) september ( ) august ( ) july ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) september ( ) august ( ) june ( ) may ( ) march ( ) february ( ) january ( ) november ( ) october ( ) september ( ) august ( ) july ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) september ( ) august ( ) june ( ) may ( ) april ( ) march ( ) feeds rss - posts rss - comments recent comments jrochkind on rails auto-scaling on heroku adam (rails autoscale) on rails auto-scaling on heroku on catalogers, programmers, and user tasks – gavia libraria on broad categories from class numbers replacing marc – gavia libraria on linked data caution jrochkind on deep dive: moving ruby projects from travis to github actions for ci jrochkind on deep dive: moving ruby projects from travis to github actions for ci jrochkind on deep dive: moving ruby projects from travis to github actions for ci eregontp on deep dive: moving ruby projects from travis to github actions for ci top posts yes, product owner and technical lead need to be different people bootstrap to : changes in how font size, line-height, and spacing is done. or "what happened to $line-height-computed." dealing with legacy and externally loaded code in webpack(er) activerecord: atomic check-and-update through optimistic locking are you talking to heroku redis in cleartext or ssl? top clicks bibwild.files.wordpress.c… apidock.com/rails/activer… github.com/mperham/sideki… bibwild.files.wordpress.c… opensolr.com a blog by jonathan rochkind. all original content licensed cc-by. create a website or blog at wordpress.com add your thoughts here... (optional) post to cancel privacy & cookies: this site uses cookies. by continuing to use this website, you agree to their use. to find out more, including how to control cookies, see here: cookie policy planned maintenance: classify api javascript is currently not supported or is disabled by this browser. some features of this site will not be available. please enable javascript for full functionality. oclc.org oclc.org home research support & training community center webjunction skip to page content. about us contact us get started develop collaborate gallery news events settings menu search home news wms ncip api changes planned maintenance: classify api february oclc will be performing quarterly maintenance on the experimental classify api on february from : am – : am eastern us (utc - ). during this time the classify application and api will be unavailable. please contact oclc customer support if you have any questions. karen coombs senior product analyst oclc headquarters kilgour place dublin, ohio - us oclc@oclc.org - - - - (usa / canada only) find information for developers librarians partners visit related sites oclc.org oclc research support & training oclc system alerts contact the developer network browse the api explorer request a web service key subscribe to wc-devnet-l get started develop collaborate gallery news events follow the oclc developer network: twitter youtube flickr github © oclc domestic and international trademarks and/or service marks of oclc, inc. and its affiliates this site uses cookies. by continuing to browse the site, you are agreeing to our use of cookies. find out more about oclc's cookie notice. feedback privacy statement accessibility statement iso certificate news: virgil griffith wants charges tossed, microstrategy bets big on btc, ledger hacked – amy castor primary menu amy castor independent journalist about me selected clips contact me blog follow blog via email enter your email address to follow this blog and receive notifications of new posts by email. join , other followers email address: follow twitter updates rt @dgolumbia: isn't it about time women and minorities got to be equal partners in the #cryptocurrency "industry" effort to defraud people… minutes ago rt @tr llytr llface: the "bitcoin economy" is just illegal gambling by people who are under lockdown with nothing else to do, funded by a s… hours ago rt @cushbomb: ayn rand is useful because she’s one of the few thinkers who is the completely wrong. you don’t have to spend time separating… hours ago rt @tommorris: clubhouse is a shitshow. also, using phone numbers as a unique identifier is terrible for privacy. please stop. https://t.c… hours ago rt @digieconomist: the chart shows one #bitcoin transaction versus , visa transactions, because any less visa transactions wouldn’t e… day ago recent comments anonymous on confirmed: reggie fowler can… virginia viola on news: i’m speaking in vancouve… ico on news: money laundering in real… all on talkgold — the ponzi forum whe… amy castor on news: kraken sets out to raise… skip to content amy castor news: virgil griffith wants charges tossed, microstrategy bets big on btc, ledger hacked things are getting antsy here in the u.s. we’ve got two days till the election, and i’m stocking up on food and alcohol, just in case all hell breaks loose. meanwhile, here’s the crypto news for the week, starting with… “libra shrugged” is here! david gerard’s book “libra shrugged” is available on amazon starting monday. i bought a copy, and you should, too. the book covers everything from how facebook was lured into blockchain in the first place—even that part is crazy—to how its plans for a world cryptocurrency were slammed down by regulators. there’s even a section on central bank digital currencies, or cbdcs. you would think that a company as large as facebook would be savvy enough to know how to prep for regulators, but sadly, no. read the book. it is fabulous. (i helped edit an early draft.) virgil tries to get charges dropped virgil griffith, the former ethereum developer who was arrested last thanksgiving and charged with one count of conspiracy to violate the international emergency economic powers act, is trying to get the charges dropped. griffith, a u.s. citizen who was living in singapore at the time, flew to north korea in to give a talk at a conference. (the ieepa prohibits u.s. citizens from exporting goods, services, or technology to north korea without approval from the treasury department.) the motion, filed by attorney brian klein, is moving to dismiss based on grounds the indictment was unconstitutional. essentially the motion claims griffith didn’t do anything that horrible, like actually teach the dprk how to evade sanctions. he simply went to a conference there and gave a general speech based on publicly available information, “like he does almost monthly at conferences throughout the world,” klein wrote. (coindesk, cointelegraph, decrypt) "here, the government seeks to penalize speech…." virgil griffith, the developer indicted for sanctions violations after going to north korea, is moving to dismiss on first amendment grounds. this is an important challenge to us sanctions enforcement:https://t.co/hnwaf d y — jake chervinsky (@jchervinsky) october , i’m no lawyer, but i think trying to get the charges dismissed is a long-shot. griffith was pretty in-your-face about traveling to the sanctioned country, going so far as posting his visa for north korea on twitter and encouraging others to come to the conference with him. the u.s. takes sanctions “very seriously,” stephen rutenberg, an attorney at polsinelli law firm, told coindesk in january. “it wasn’t like he was going there to play music.” bitcoin hits $ , the price of bitcoin hit $ , (briefly) on october —for the first time since january , when it was in free fall from the biggest bitcoin bubble to date. bitcoiners (people who are invested in the popular virtual currency and want you to invest, too) are convinced we’ll have another bull run like , so buy now before it goes to the moon! it is really, really hard to ignore the correlation between bitcoin’s price and the latest fresh supply of tethers (usdt). tether issued $ million worth of tethers in one week and is fast on its way to a total of $ billion worth of tethers in circulation. take a look at this graph: enhance pic.twitter.com/hi odyqqb — nomics crypto (@nomicsfinance) october , most of those tethers, by the way, go straight to crypto exchanges bitfinex, binance, and huobi, according to whale alert. 🚨 , , #usdt ( , , usd) transferred from tether treasury to #binance tx: https://t.co/f gbppigp — whale alert (@whale_alert) november , microstrategy’s bitcoin bet in its q earnings call, business analytics firm microstrategy said it was putting its excess stockpiles of cash into stock buybacks and bitcoin—but mostly bitcoin. bitcoin is mentioned times in the call by microstrategy president phong le and ceo michael saylor, who spoke to investors. the publicly traded company purchased approximately , bitcoins for $ million during the quarter, for an average of around $ , . saylor also disclosed in a recent tweet that he personally “hodls” , btc, which he bought at $ , each on average for a total of $ million. he claims microstrategy knew of his personal investments before the company went ahead and bought btc on its own. as wall street journal columnist jason zweig notes, microstrategy stock was hot during the dot-com bubble of the late s, but after the sec accused the firm of accounting fraud in december , its stock never recovered. to settle the charges at the time, saylor paid $ , in civil penalties to the sec and disgorged $ . million. two other executives also paid $ , to the sec and returned $ . million and $ . million to shareholders. they did not admit or deny the charges. (sec litigation release) now, after complaining to the wsj about the low returns on cash, saylor said he is willing to take a risk on bitcoin. but what about the company’s stockholders? is this why they are buying a publicly traded stock? so they can gamble on bitcoin? “investors who wish to buy bitcoin could always do so themselves with the proceeds of a dividend or share buybacks,” zweig writes. “the point of buying a stock is to get a stake in a business, not to take a flier on cryptocurrency.” keto dietary hazards bitcoiners are famous for their weird dietary habits. last week, i mentioned soylent, the dreadful drink that is a meal replacement substitute, which some bitcoiners were investing in—and drinking. and a lot of bitcoiners follow a strict all-meat diet. but at least one has ended up in the hospital. “after about a month of a % strict carnivore diet, and years of a mostly [low carb, high fat] diet before that, i have now been hospitalized since sunday morning for diverticulitis of the large intestine. i won’t be able to eat anything but soups and mashed things for a while,” bitcoin advocate knut svanholm tweeted from his hospital bed. after about a month of a % strict carnivore diet, and years of a mostly lchf diet before that, i have now been hospitalized since sunday morning for diverticulitis of the large intestine. i won't be able to eat anything but soups and mashed things for a while. thread 👇 pic.twitter.com/efzkexbjrn — knut svanholm (@knutsvanholm) october , ledger hacked you would think a cryptocurrency wallet—meant to help you safely store your bitcoin—would be big on security right? think again. after a hack resulted in a leak of ledger’s customer emails (and phone numbers, too, apparently), owners of the hardware crypto wallet are being targeted by a phishing attack. a third-party is sending them emails and text messages that appear to come from ledger support telling them to download the latest version of the ledger app. (the block) one ledger customer posted on reddit a confusing email from ledger explaining the situation. customers are upset because they say ledger hasn’t been transparent about the breach and what exactly was stolen. so, if you want to buy bitcoin, but you’re worried about how to safely manage your keys, invest in a hardware wallet—but preferably not one that will lose your emails. ok @ledger, i used an email exclusively for your shop and i got a (very well done) phishing mail today, pointing to ledgersupport_dot_io download why didn't i get any info that my email was compromised? what data was compromised from me? do they know my physical address? / — andreas tasch ⚡ (@ndeet) october , @ledger, i received two phishing emails the other day but also just received a text 'from you' telling me to update the app via a link. how much of my data was stolen from you? — leo montgomery (@leo_montgomery) november , share this: twitter facebook linkedin like this: like loading... facebook libraledgermichael saylormicrostrategyvirgil griffith posted on november , december , by amy castor in blogging post navigation previous postnews: modern consensus co-founder arrested, paypal embraces bitcoin, peter schiff in the hot seat next postquadriga’s th report of the trustee—choosing a currency conversion date leave a reply cancel reply enter your comment here... fill in your details below or click an icon to log in: email (address never made public) name website you are commenting using your wordpress.com account. ( log out / change ) you are commenting using your google account. ( log out / change ) you are commenting using your twitter account. ( log out / change ) you are commenting using your facebook account. ( log out / change ) cancel connecting to %s notify me of new comments via email. notify me of new posts via email. create a website or blog at wordpress.com add your thoughts here... (optional) post to cancel %d bloggers like this: managed solr saas options – bibliographic wilderness skip to content bibliographic wilderness menu about contact managed solr saas options jrochkind general january , january , i was recently looking for managed solr “software-as-a-service” (saas) options, and had trouble figuring out what was out there. so i figured i’d share what i learned. even though my knowledge here is far from exhaustive, and i have only looked seriously at one of the ones i found. the only managed solr options i found were: websolr; searchstax; and opensolr. of these, i think websolr and searchstax are more well-known, i couldn’t find anyone with experience with opensolr, which perhaps is newer. of them all, searchstax is the only one i actually took for a test drive, so will have the most to say about. why we were looking we run a fairly small-scale app, whose infrastructure is currently self-managed aws ec instances, running respectively: ) a rails web app ) bg workers for the rails web app ) postgres, and ) solr. oh yeah, there’s also a redis running one of those servers, on # with pg or # with solr, i forget. currently we manage this all ourselves, right on the ec . but we’re looking to move as much as we can into “managed” servers. perhaps we’ll move to heroku. perhaps we’ll use hatchbox. or if we do stay on aws resources we manage directly, we’d look at things like using an aws rds postgres instead of installing it on an ec ourselves, an aws elasticache for redis, maybe look into elastic beanstalk, etc. but no matter what we do, we need a solr, and we’d like to get it managed. hatchbox has no special solr support, aws doesn’t have a solr service, heroku does have a solr add-on but you can also use any solr with it and we’ll get to that later. our current solr use is pretty small scale. we don’t run “solrcloud mode“, just legacy ordinary solr. we only have around , documents in there (tiny for solr), our index size is only mb. our traffic is pretty low — when i tried to figure out how low, it doesn’t seem we have sufficient logging turned on to answer that specifically but using proxy metrics to guess i’d say k- k requests a day, query as well as add. this is a pretty small solr installation, although it is used centrally for the primary functions of the (fairly low-traffic) app. it currently runs on an ec t a.small, which is a “burstable” ec type with only g of ram. it does have two vcpus (that is one core with ‘hyperthreading’). the t a.small ec instance only costs $ /month on-demand price! we know we’ll be paying more for managed solr, but we want to do get out of the business of managing servers — we no longer really have the staff for it. websolr (didn’t actually try out) websolr is the only managed solr currently listed as a heroku add-on. it is also available as a managed solr independent of heroku. the pricing in the heroku plans vs the independent plans seems about the same. as a heroku add-on there is a $ “staging” plan that doesn’t exist in the independent plans. (unlike some other heroku add-ons, no time-limited free plan is available for websolr). but once we go up from there, the plans seem to line up. starting at: $ /month for: million document limit k requests/day index mb storage concurrent requests limit (this limit is not mentioned on the independent pricing page?) next level up is $ /month for: million document limit k requests/day . gb storage concurrent request limit (again concurrent request limits aren’t mentioned on independent pricing page) as you can see, websolr has their plans metered by usage. $ /month is around the price range we were hoping for (we’ll need two, one for staging one for production). our small solr is well under million documents and ~ gb storage, and we do only use one index at present. however, the k requests/day limit i’m not sure about, even if we fit under it, we might be pushing up against it. and the “concurrent request” limit simply isn’t one i’m even used to thinking about. on a self-managed solr it hasn’t really come up. what does “concurrent” mean exactly in this case, how is it measured? with puma web workers and sometimes a possibly multi-threaded batch index going on, could we exceed a limit of ? seems plausible. what happens when they are exceeded? your solr request results in an http error! do i need to now write the app to rescue those gracefully, or use connection pooling to try to avoid them, or something? having to rewrite the way our app functions for a particular managed solr is the last thing we want to do. (although it’s not entirely clear if those connection limits exist on the non-heroku-plugin plans, i suspect they do?). and in general, i’m not thrilled with the way the pricing works here, and the price points. i am positive for a lot of (eg) heroku customers an additional $ * =$ /month is peanuts not even worth accounting for, but for us, a small non-profit whose app’s traffic does not scale with revenue, that starts to be real money. it is not clear to me if websolr installations (at “standard” plans) are set up in “solrcloud mode” or not; i’m not sure what api’s exist for uploading your custom schema.xml (which we’d need to do), or if they expect you to do this only manually through a web ui (that would not be good); i’m not sure if you can upload custom solrconfig.xml settings (this may be running on a shared solr instance with standard solrconfig.xml?). basically, all of this made websolr not the first one we looked at. does it matter if we’re on heroku using a managed solr that’s not a heroku plugin? i don’t think so. in some cases, you can get a better price from a heroku plug-in than you could get from that same vendor not on heroku or other competitors. but that doesn’t seem to be the case here, and other that that does it matter? well, all heroku plug-ins are required to bill you by-the-minute, which is nice but not really crucial, other forms of billing could also be okay at the right price. with a heroku add-on, your billing is combined into one heroku invoice, no need to give a credit card to anyone else, and it can be tracked using heroku tools. which is certainly convenient and a plus, but not essential if the best tool for the job is not a heroku add-on. and as a heroku add-on, websolr provides a websolr_url heroku config/env variable automatically to code running on heroku. ok, that’s kind of nice, but it’s not a big deal to set a solr_url heroku config manually referencing the appropriate address. i suppose as a heroku add-on, websolr also takes care of securing and authenticating connections between the heroku dynos and the solr, so we need to make sure we have a reasonable way to do this from any alternative. searchstax (did take it for a spin) searchstax’s pricing tiers are not based on metering usage. there are no limits based on requests/day or concurrent connections. searchstax runs on dedicated-to-you individual solr instances (i would guess running on dedicated-to-you individual (eg) ec , but i’m not sure). instead the pricing is based on size of host running solr. you can choose to run on instances deployed to aws, google cloud, or azure. we’ll be sticking to aws (the others, i think, have a slight price premium). while searchstax gives you a pricing pages that looks like the “new-way-of-doing-things” transparent pricing, in fact there isn’t really enough info on public pages to see all the price points and understand what you’re getting, there is still a kind of “talk to a salesperson who has a price sheet” thing going on. what i think i have figured out from talking to a salesperson and support, is that the “silver” plans (“starting at $ a month”, although we’ll say more about that in a bit) are basically: we give you a solr, we don’t don’t provide any technical support for solr. while the “gold” plans “from $ /month” are actually about paying for solr consultants to set up and tune your schema/index etc. that is not something we need, and $ +/month is way more than the price range we are looking for. while the searchstax pricing/plan pages kind of imply the “silver” plan is not suitable for production, in fact there is no real reason not to use it for production i think, and the salesperson i talked to confirmed that — just reaffirming that you were on your own managing the solr configuration/setup. that’s fine, that’s what we want, we just don’t want to mangage the os or set up the solr or upgrade it etc. the silver plans have no sla, but as far as i can tell their uptime is just fine. the silver plans only guarantees -hour support response time — but for the couple support tickets i filed asking questions while under a free -day trial (oh yeah that’s available), i got prompt same-day responses, and knowledgeable responses that answered my questions. so a “silver” plan is what we are interested in, but the pricing is not actually transparent. $ /month is for the smallest instance available, and if you prepay/contract for a year. they call that small instance an ndn and it has gb of ram and gb of storage. if you pay-as-you-go instead of contracting for a year, that already jumps to $ /month. (that price is available on the trial page). when you are paying-as-you-go, you are actually billed per-day, which might not be as nice as heroku’s per-minute, but it’s pretty okay, and useful if you need to bring up a temporary solr instance as part of a migration/upgrade or something like that. the next step up is an “ndn ” which has g of ram and gb of storage, and has an ~$ /month pay-as-you-go — you can find that price if you sign-up for a free trial. the discount price price for an annual contract is a discount similar to the ndn %, $ /month — that price i got only from a salesperson, i don’t know if it’s always stable. it only occurs to me now that they don’t tell you how many cpus are available. i’m not sure if i can fit our solr in the g ndn , but i am sure i can fit it in the g ndn with some headroom, so i didn’t look at plans above that — but they are available, still under “silver”, with prices going up accordingly. all searchstax solr instances run in “solrcloud” mode — these ndn and ndn ones we’re looking at just run one node with one zookeeper, but still in cloud mode. there are also “silver” plans available with more than one node in a “high availability” configuration, but the prices start going up steeply, and we weren’t really interested in that. because it’s solrcloud mode though, you can use the standard solr api for uploading your configuration. it’s just solr! so no arbitrary usage limits, no features disabled. the searchstax web console seems competently implemented; it let’s you create and delete individual solr “deployments”, manage accounts to login to console (on “silver” plan you only get two, or can pay $ /month/account for more, nah), and set up auth for a solr deployment. they support ip-based authentication or http basic auth to the solr (no limit to how many solr basic auth accounts you can create). http basic auth is great for us, because trying to do ip-based from somewhere like heroku isn’t going to work. all solrs are available over https/ssl — great! searchstax also has their own proprietary http api that lets you do most anything, including creating/destroying deployments, managing solr basic auth users, basically everything. there is some api that duplicates the solr cloud api for adding configsets, i don’t think there’s a good reason to use it instead of standard solrcloud api, although their docs try to point you to it. there’s even some kind of webhooks for alerts! (which i haven’t really explored). basically, searchstax just seems to be a sane and rational managed solr option, it has all the features you’d expect/need/want for dealing with such. the prices seem reasonable-ish, generally more affordable than websolr, especially if you stay in “silver” and “one node”. at present, we plan to move forward with it. opensolr (didn’t look at it much) i have the least to say about this, have spent the least time with it, after spending time with searchstax and seeing it met our needs. but i wanted to make sure to mention it, because it’s the only other managed solr i am even aware of. definitely curious to hear from any users. here is the pricing page. the prices seem pretty decent, perhaps even cheaper than searchstax, although it’s unclear to me what you get. does “ solr clusters” mean that it’s not solrcloud mode? after seeing how useful solrcloud apis are for management (and having this confirmed by many of my peers in other libraries/museums/archives who choose to run solrcloud), i wouldn’t want to do without it. so i guess that pushes us to “executive” tier? which at $ /month (billed yearly!) is still just fine, around the same as searchstax. but they do limit you to one solr index; i prefer searchstax’s model of just giving you certain host resources and do what you want with it. it does say “shared infrastructure”. might be worth investigating, curious to hear more from anyone who did. now, what about elasticsearch? we’re using solr mostly because that’s what various collaborative and open source projects in the library/museum/archive world have been doing for years, since before elasticsearch even existed. so there are various open source libraries and toolsets available that we’re using. but for whatever reason, there seem to be so many more managed elasticsearch saas available. at possibly much cheaper pricepoints. is this because the elasticsearch market is just bigger? or is elasticsearch easier/cheaper to run in a saas environment? or what? i don’t know. but there’s the controversial aws elasticsearch service; there’s the elastic cloud “from the creators of elasticsearch”. on heroku that lists one solr add-on, there are three elasticsearch add-ons listed: elasticcloud, bonsai elasticsearch, and searchbox elasticsearch. if you just google “managed elasticsearch” you immediately see or other names. i don’t know enough about elasticsearch to evaluate them. there seem on first glance at pricing pages to be more affordable, but i may not know what i’m comparing and be looking at tiers that aren’t actually usable for anything or will have hidden fees. but i know there are definitely many more managed elasticsearch saas than solr. i think elasticsearch probably does everything our app needs. if i were to start from scratch, i would definitely consider elasticsearch over solr just based on how many more saas options there are. while it would require some knowledge-building (i have developed a lot of knowlege of solr and zero of elasticsearch) and rewriting some parts of our stack, i might still consider switching to es in the future, we don’t do anything too too complicated with solr that would be too too hard to switch to es, probably. share this: twitter facebook published by jrochkind view all posts by jrochkind published january , january , post navigation previous post gem authors, check your release sizes next post rails auto-scaling on heroku leave a reply cancel reply enter your comment here... fill in your details below or click an icon to log in: email (required) (address never made public) name (required) website you are commenting using your wordpress.com account. ( log out / change ) you are commenting using your google account. ( log out / change ) you are commenting using your twitter account. ( log out / change ) you are commenting using your facebook account. ( log out / change ) cancel connecting to %s notify me of new comments via email. notify me of new posts via email. bibliographic wilderness is a blog by jonathan rochkind about digital library services, ruby, and web development. contact search for: email subscription enter your email address to subscribe to this blog and receive notifications of new posts by email. join other followers email address: subscribe recent posts product management february , rails auto-scaling on heroku january , managed solr saas options january , gem authors, check your release sizes january , every time you decide to solve a problem with code… january , archives archives select month february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) april ( ) march ( ) december ( ) october ( ) september ( ) august ( ) june ( ) april ( ) march ( ) february ( ) december ( ) november ( ) october ( ) september ( ) august ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) november ( ) october ( ) september ( ) august ( ) july ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) september ( ) august ( ) june ( ) may ( ) march ( ) february ( ) january ( ) november ( ) october ( ) september ( ) august ( ) july ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) september ( ) august ( ) june ( ) may ( ) april ( ) march ( ) feeds rss - posts rss - comments recent comments jrochkind on rails auto-scaling on heroku adam (rails autoscale) on rails auto-scaling on heroku on catalogers, programmers, and user tasks – gavia libraria on broad categories from class numbers replacing marc – gavia libraria on linked data caution jrochkind on deep dive: moving ruby projects from travis to github actions for ci jrochkind on deep dive: moving ruby projects from travis to github actions for ci jrochkind on deep dive: moving ruby projects from travis to github actions for ci eregontp on deep dive: moving ruby projects from travis to github actions for ci top posts yes, product owner and technical lead need to be different people bootstrap to : changes in how font size, line-height, and spacing is done. or "what happened to $line-height-computed." dealing with legacy and externally loaded code in webpack(er) activerecord: atomic check-and-update through optimistic locking are you talking to heroku redis in cleartext or ssl? top clicks bibwild.files.wordpress.c… apidock.com/rails/activer… github.com/mperham/sideki… bibwild.files.wordpress.c… opensolr.com a blog by jonathan rochkind. all original content licensed cc-by. create a website or blog at wordpress.com privacy & cookies: this site uses cookies. by continuing to use this website, you agree to their use. to find out more, including how to control cookies, see here: cookie policy comparing performance of a rails app on different heroku formations – bibliographic wilderness skip to content bibliographic wilderness menu about contact comparing performance of a rails app on different heroku formations jrochkind general november , november , i develop a “digital collections” or “asset management” app, which manages and makes digitized historical objects and their descriptions available to the public, from the collections here at the science history institute. the app receives relatively low level of traffic (according to google analytics, around k pageviews a month), although we want it to be able to handle spikes without falling down. it is not the most performance-optimized app, it does have some relatively slow responses and can be ram-hungry. but it works adequately on our current infrastructure: web traffic is handled on a single aws ec t .medium instance, with passenger processes (free version of passenger, so no multi-threading). we are currently investigating the possibility of moving our infrastructure to heroku. after realizing that heroku standard dynos did not seem to have the performance characteristics i had expected, i decided to approach performance testing more methodically, to compare different heroku dyno formations to each other and to our current infrastructure. our basic research question is probably what heroku formation do we need to have similar performance to our existing infrastructure? i am not an expert at doing this — i did some research, read some blog posts, did some thinking, and embarked on this. i am going to lead you through how i approached this and what i found. feedback or suggestions are welcome. the most surprising result i found was much poorer performance from heroku standard dynos than i expected, and specifically that standard dynos would not match performance of present infrastructure. what urls to use in test some older load-testing tools only support testing one url over and over. i decided i wanted to test a larger sample list of urls — to be a more “realistic” load, and also because repeatedly requesting only one url might accidentally use caches in ways you aren’t expecting giving you unrepresentative results. (our app does not currently use fragment caching, but caches you might not even be thinking about include postgres’s built-in automatic caches, or passenger’s automatic turbocache (which i don’t think we have turned on)). my initial thought to get a list of such urls from our already-in-production app from production logs, to get a sample of what real traffic looks like. there were a couple barriers for me to using production logs as urls: some of those urls might require authentication, or be post requests. the bulk of our app’s traffic is get requests available without authentication, and i didn’t feel like the added complexity of setting up anything else in a load traffic was worthwhile. our app on heroku isn’t fully functional yet. without having connected it to a solr or background job workers, only certain urls are available. in fact, a large portion of our traffic is an “item” or “work” detail page like this one. additionally, those are the pages that can be the biggest performance challenge, since the current implementation includes a thumbnail for every scanned page or other image, so response time unfortunately scales with number of pages in an item. so i decided a good list of urls was simply a representative same of those “work detail” pages. in fact, rather than completely random sample, i took the largest/slowest work pages, and then added in another randomly chosen from our current ~ k pages. and gave them all a randomly shuffled order. in our app, every time a browser requests a work detail page, the js on that page makes an additional request for a json document that powers our page viewer. so for each of those work detail pages, i added the json request url, for a more “realistic” load, and total urls. performance: “base speed” vs “throughput under load” thinking about it, i realized there were two kinds of “performance” or “speed” to think about. you might just have a really slow app, to exagerate let’s say typical responses are seconds. that’s under low/no-traffic, a single browser is the only thing interacting with the app, it makes a single request, and has to wait seconds for a response. that number might be changed by optimizations or performance regressions in your code (including your dependencies). it might also be changed by moving or changing hardware or virtualization environment — including giving your database more cpu/ram resources, etc. but that number will not change by horizontally scaling your deployment — adding more puma or passenger processes or threads, scaling out hosts with a load balancer or heroku dynos. none of that will change this base speed because it’s just how long the app takes to prepare a response when not under load, how slow it is in a test only one web worker , where adding web workers won’t matter because they won’t be used. then there’s what happens to the app actually under load by multiple users at once. the base speed is kind of a lower bound on throughput under load — page response time is never going to get better than s for our hypothetical very slow app (without changing the underlying base speed). but it can get a lot worse if it’s hammered by traffic. this throughput under load can be effected not only by changing base speed, but also by various forms of horizontal scaling — how many puma or passenger processes you have with how many threads each, and how many cpus they have access to, as well as number of heroku dynos or other hosts behind a load balancer. (i had been thinking about this distinction already, but nate berkopec’s great blog post on scaling rails apps gave me the “speed” vs “throughout” terminology to use). for my condition, we are not changing the code at all. but we are changing the host architecture from a manual ec t .medium to heroku dynos (of various possible types) in a way that could effect base speed, and we’re also changing our scaling architecture in a way that could change throughput under load on top of that — from one t .medium with passenger process to possibly multiple heroku dynos behind heroku’s load balancer, and also (for reasons) switching from free passenger to trying puma with multiple threads per process. (we are running puma with new experimental performance features turned on). so we’ll want to get a sense of base speed of the various host choices, and also look at how throughput under load changes based on various choices. benchmarking tool: wrk we’re going to use wrk. there are lots of choices for http benchmarking/load testing, with really varying complexity and from different eras of web history. i got a bit overwhelmed by it, but settled on wrk. some other choices didn’t have all the features we need (some way to test a list of urls, with at least some limited percentile distribution reporting). others were much more flexible and complicated and i had trouble even figuring out how to use them! wrk does need a custom lua script in order to handle a list of urls. i found a nice script here, and modified it slightly to take filename from an env variable, and not randomly shuffle input list. it’s a bit confusing understanding the meaning of “threads” vs “connections” in wrk arguments. this blog post from appfolio clears it up a bit. i decided to leave threads set to , and vary connections for load — so -c -t is a “one url at a time” setting we can use to test “base speed”, and we can benchmark throughput under load by increasing connections. we want to make sure we run the test for long enough to touch all urls in our list at least once, even in the slower setups, to have a good comparison — ideally it would be go through the list more than once, but for my own ergonomics i had to get through a lot of tests so ended up less tha ideal. (should i have put fewer than urls in? not sure). conclusions in advance as benchmarking posts go (especially when i’m the one writing them), i’m about to drop a lot of words and data on you. so to maximize the audience that sees the conclusions (because they surprise me, and i want feedback/pushback on them), i’m going to give you some conclusions up front. our current infrastructure has web app on a single ec t .medium, which is a burstable ec type — our relatively low-traffic app does not exhaust it’s burst credits. measuring base speed (just one concurrent request at a time), we found that performance dynos seem to have about the cpu speed of a bursting t .medium (just a hair slower). but standard dynos are as a rule to times slower; additionally they are highly variable, and that variability can be over hours/days. a minute period can have measured response times or more times slower than another minute period a couple hours later. but they seem to typically be - x slower than our current infrastructure. under load, they scale about how you’d expect if you knew how many cpus are present, no real surprises. our existing t .medium has two cpus, so can handle simultaneous requests as fast as , and after that degrades linearly. a single performance-l ($ /month) has cpus ( hyperthreads), so scales under load much better than our current infrastructure. a single performance-m ($ /month) has only cpu (!), so scales pretty terribly under load. testing scaling with standard- x’s ($ /month total), we see that it scales relatively evenly. although lumpily because of variability, and it starts out so much worse performing that even as it scales “evenly” it’s still out-performed by all other arcchitectures. :( (at these relatively fast median response times you might say it’s still fast enough who cares, but in our fat tail of slower pages it gets more distressing). now we’ll give you lots of measurements, or you can skip all that to my summary discussion or conclusions for our own project at the end. let’s compare base speed ok, let’s get to actual measurements! for “base speed” measurements, we’ll be telling wrk to use only one connection and one thread. existing t .medium: base speed our current infrastructure is one ec t .medium. this ec instance type has two vcpus and gb of ram. on that single ec instance, we run passenger (free not enterprise) set to have passenger processes, although the base speed test with only one connection should only touch one of the workers. the t is a “burstable” type, and we do always have burst credits (this is not a high traffic app; verified we never exhausted burst credits in these tests), so our test load may be taking advantage of burst cpu. $ urls=./sample_works.txt wrk -c -t -d m --timeout s --latency -s load_test/multiplepaths.lua.txt https://[current staging server] multiplepaths: found paths multiplepaths: found paths running m test @ https://staging-digital.sciencehistory.org threads and connections thread stats avg stdev max +/- stdev latency . ms . ms . s . % req/sec . . . . % latency distribution % . ms % . ms % . ms % . s requests in . m, . mb read requests/sec: . transfer/sec: . mb i’m actually feeling pretty good about those numbers on our current infrastructure! ms median, not bad, and even ms th percentile is not too bad. now, our test load involves some json responses that are quicker to deliver than corresponding html page, but still pretty good. the th/ th/and max request ( . s) aren’t great, but i knew i had some slow pages, this matches my previous understanding of how slow they are in our current infrastructure. th percentile is ~ times th percenile. i don’t have an understanding of why the two different req/sec and requests/sec values are so different, and don’t totally understand what to do with the stdev and +/- stdev values, so i’m just going to be sticking to looking at the latency percentiles, i think “latency” could also be called “response times” here. but ok, this is our baseline for this workload. and doing this minute test at various points over the past few days, i can say it’s nicely regular and consistent, occasionally i got a slower run, but th percentile was usually ms– ms, right around there. heroku standard- x: base speed from previous mucking about, i learned i can only reliably fit one puma worker in a standard- x, and heroku says “we typically recommend a minimum of processes, if possible” (for routing algorithmic reasons when scaled to multiple dynos), so i am just starting at a standard- x with two puma workers each with threads, matching heroku recommendations for a standard- x dyno. so one thing i discovered is that bencharks from a heroku standard dyno are really variable, but here are typical ones: $ heroku dyno:resize type size qty cost/mo ─────── ─────────── ─── ─────── web standard- x $ heroku config:get --shell web_concurrency rails_max_threads web_concurrency= rails_max_threads= $ urls=./sample_works.txt wrk -c -t -d m --timeout s --latency -s load_test/multiplepaths.lua.txt https://scihist-digicoll.herokuapp.com/ multiplepaths: found paths multiplepaths: found paths running m test @ https://scihist-digicoll.herokuapp.com/ threads and connections thread stats avg stdev max +/- stdev latency . ms . ms . s . % req/sec . . . . % latency distribution % . ms % . ms % . s % . s requests in . m, . mb read requests/sec: . transfer/sec: . kb i had heard that heroku standard dynos would have variable performance, because they are shared multi-tenant resources. i had been thinking of this like during a minute test i might see around the same median with more standard deviation — but instead, what it looks like to me is that running this benchmark on monday at am might give very different results than at : am or tuesday at pm. the variability is over a way longer timeframe than my minute test — so that’s something learned. running this here and there over the past week, the above results seem to me typical of what i saw. (to get better than “seem typical” on this resource, you’d have to run a test, over several days or a week i think, probably not hammering the server the whole time, to get a sense of actual statistical distribution of the variability). i sometimes saw tests that were quite a bit slower than this, up to a ms median. i rarely if ever saw results too much faster than this on a standard- x. th percentile is ~ x median, less than my current infrastructure, but that still gets up there to . instead of ms. this typical one is quite a bit slower than than our current infrastructure, our median response time is x the latency, with th and max being around x. this was worse than i expected. heroku performance-m: base speed although we might be able to fit more puma workers in ram, we’re running a single-connection base speed test, so it shouldn’t matter to, and we won’t adjust it. $ heroku dyno:resize type size qty cost/mo ─────── ───────────── ─── ─────── web performance-m $ heroku config:get --shell web_concurrency rails_max_threads web_concurrency= rails_max_threads= $ urls=./sample_works.txt wrk -c -t -d m --timeout s --latency -s load_test/multiplepaths.lua.txt https://scihist-digicoll.herokuapp.com/ multiplepaths: found paths multiplepaths: found paths running m test @ https://scihist-digicoll.herokuapp.com/ threads and connections thread stats avg stdev max +/- stdev latency . ms . ms . s . % req/sec . . . . % latency distribution % . ms % . ms % . s % . s requests in . m, . mb read requests/sec: . transfer/sec: . kb this is a lot closer to the ballpark of our current infrastructure. it’s a bit slower ( ms median intead of ms median), but in running this now and then over the past week it was remarkably, thankfully, consistent. median and th percentile are both % slower (makes me feel comforted that those numbers are the same in these two runs!), that doesn’t bother me so much if it’s predictable and regular, which it appears to be. the max appears to me still a little bit less regular on heroku for some reason, since performance is supposed to be non-shared aws resources, you wouldn’t expect it to be, but slow requests are slow, ok. th percentile is ~ x median, about the same as my current infrastructure. heroku performance-l: base speed $ heroku dyno:resize type size qty cost/mo ─────── ───────────── ─── ─────── web performance-l $ heroku config:get --shell web_concurrency rails_max_threads web_concurrency= rails_max_threads= urls=./sample_works.txt wrk -c -t -d m --timeout s --latency -s load_test/multiplepaths.lua.txt https://scihist-digicoll.herokuapp.com/ multiplepaths: found paths multiplepaths: found paths running m test @ https://scihist-digicoll.herokuapp.com/ threads and connections thread stats avg stdev max +/- stdev latency . ms . ms . s . % req/sec . . . . % latency distribution % . ms % . ms % . s % . s requests in . m, . mb read requests/sec: . transfer/sec: . kb no news is good news, it looks very much like performance-m, which is exactly what we expected, because this isn’t a load test. it tells us that performance-m and performance-l seem to have similar cpu speeds and similar predictable non-variable regularity, which is what i find running this test periodically over a week. th percentile is ~ x median, about the same as current infrastructure. the higher max speed is just evidence of what i mentioned, the speed of slowest request did seem to vary more than on our manual t .medium, can’t really explain why. summary: base speed not sure how helpful this visualization is, charting th, th, and th percentile responses across architectures. but basically: performance dynos perform similarly to my (bursting) t .medium. can’t explain why performance-l seems slightly slower than performance-m, might be just incidental variation when i ran the tests. the standard- x is about twice as slow as my (bursting) t .medium. again recall standard- x results varied a lot every time i ran them, the one i reported seems “typical” to me, that’s not super scientific, admittedly, but i’m confident that standard- x are a lot slower in median response times than my current infrastructure. throughput under load ok, now we’re going to test using wrk to use more connections. in fact, i’ll test each setup with various number of connections, and graph the result, to get a sense of how each formation can handle throughput under load. (this means a lot of minutes to get all these results, at minutes per number of connection test, per formation!). an additional thing we can learn from this test, on heroku we can look at how much ram is being used after a load test, to get a sense of the app’s ram usage under traffic to understand the maximum number of puma workers we might be able to fit in a given dyno. existing t .medium: under load a t .medium has g of ram and cpus. we run passenger workers (no multi-threading, since we are free, rather than enterprise, passenger). so what do we expect? with cpus and more than workers, i’d expect it to handle simultaneous streams of requests almost as well as ; - should be quite a bit slower because they are competing for the cpus. over , performance will probably become catastrophic. connections are exactly flat with , as expected for our two cpus, hooray! then it goes up at a strikingly even line. going over (to ) simultaneous connections doesn’t matter, even though we’ve exhausted our workers, i guess at this point there’s so much competition for the two cpus already. the slope of this curve is really nice too actually. without load, our median response time is ms, but even at a totally overloaded overloaded connections, it’s only ms, which actually isn’t too bad. we can make a graph that in addition to median also has th, th, and th percentile response time on it: it doesn’t tell us too much; it tells us the upper percentiles rise at about the same rate as the median. at simultaneous connection th percentile of ms is about times the median of ms; at requests the th percentile of . seconds is about times the median of ms. this does remind us that under load when things get slow, this has more of a disastrous effect on already slow requests than fast requests. when not under load, even our th percentile was kind of sort of barley acceptable at ms, but under load at . seconds it really isn’t. single standard- x dyno: under load a standard- x dyno has g of ram. the (amazing, excellent, thanks schneems) heroku puma guide suggests running two puma workers with threads each. at first i wanted to try running three workers, which seemed to fit into available ram — but under heavy load-testing i was getting heroku r memory quota exceeded errors, so we’ll just stick with the heroku docs recommendations. two workers with threads each fit with plenty of headroom. a standard- x dyno is runs on shared (multi-tenant) underlying amazon virtual hardware. so while it is running on hardware with cpus (each of which can run two “hyperthreads“), the puma doc suggests “it is best to assume only one process can execute at a time” on standard dynos. what do we expect? well, if it really only had one cpu, it would immediately start getting bad at simulataneous connections, and just get worse from there. when we exceed the two worker count, will it get even worse? what about when we exceed the thread ( workers * threads) count? you’d never run just one dyno if you were expecting this much traffic, you’d always horizontally scale. this very artificial test is just to get a sense of it’s characteristics. also, we remember that standard- x’s are just really variable; i could get much worse or better runs than this, but graphed numbers from a run that seemed typical. well, it really does act like cpu, simultaneous connections is immediately a lot worse than . the line isn’t quite as straight as in our existing t .medium, but it’s still pretty straight; i’d attribute the slight lumpiness to just the variability of shared-architecture standard dyno, and figure it would get perfectly straight with more data. it degrades at about the same rate of our baseline t .medium, but when you start out slower, that’s more disastrous. our t .medium at an overloaded simultaneous requests is ms (pretty tolerable actually), times the median at one request only. this standard- x has a median response time of ms at only one simultaneous request, and at an overloaded requests has a median response time also about x worse, but that becomes a less tolerable ms. does also graphing the th, th, and th percentile tell us much? eh, i think the lumpiness is still just standard shared-architecture variability. the rate of “getting worse” as we add more overloaded connections is actually a bit better than it was on our t .medium, but since it already starts out so much slower, we’ll just call it a wash. (on t .medium, th percentile without load is ms and under an overloaded connections . s. on this single standard- x, it’s . s and . s). i’m not sure how much these charts with various percentiles on them tell us, i’ll not include them for every architecture hence. standard- x, dynos: under load ok, realistically we already know you shouldn’t have just one standard- x dyno under that kind of load. you’d scale out, either manually or perhaps using something like the neat rails autoscale add-on. let’s measure with dynos. each is still running puma workers, with threads each. what do we expect? hm, treating each dyno as if it has only one cpu, we’d expect it to be able to handle traffic pretty levelly up to simultenous connections, distributed to dynos. it’s going to do worse after that, but up to there is still one puma worker per connection so it might get even worse after ? well… i think that actually is relatively flat from to simultaneous connections, except for lumpiness from variability. but lumpiness from variability is huge! we’re talking ms median measured at connection, up to ms measured median at , down to ms at . and then maybe yeah, a fairly shallow slope up to simutaneous connections than steeper. but it’s all fairly shallow slope compared to our base t .medium. at connections (after which we pretty much max out), the standard- x median of ms is only . times the median at conection. compared to the t .median increase of . times. as we’d expect, scaling out to dynos (with four cpus/ hyperthreads) helps us scale well — the problem is the baseline is so slow to begin (with very high bounds of variability making it regularly even slower). performance-m: under load a performance-m has . gb of memory. it only has one physical cpu, although two “vcpus” (two hyperthreads) — and these are all your apps, it is not shared. by testing under load, i demonstrated i could actually fit workers on there without any memory limit errors. but is there any point to doing that with only / cpus? under a bit of testing, it appeared not. the heroku puma docs recommend only processes with threads. you could do a whole little mini-experiment just trying to measure/optimize process/thread count on performance-m! we’ve already got too much data here, but in some experimentation it looked to me like processes with threads each performed better (and certainly no worse) than processes with threads — if you’ve got the ram just sitting there anyway (as we do), why not? i actually tested with puma processes with threads each. there is still a large amount of ram headroom we aren’t going to use even under load. what do we expect? well, with the “hyperthreads” perhaps it can handle simultaneous requests nearly as well as (or not?); after that, we expect it to degrade quickly same as our original t .medium did. it an handle connections slightly better than you’d expect if there really was only cpu, so i guess a hyperthread does give you something. then the slope picks up, as you’d expect; and it looks like it does get steeper after simultaneous connections, yup. performance-l: under load a performance-l ($ /month) costs twice as much as a performance-m ($ /month), but has far more than twice as much resources. performance-l has a whopping gb of ram compared to performance-m’s . gb; and performance-l has real cpus/hyperthreads available to use (visible using the nproc technique in the heroku puma article. because we have plenty of ram to do so, we’re going to run worker processes to match our original t .medium’s. we still ran with threads, just cause it seems like maybe you should never run a puma worker with only one thread? but who knows, maybe workers with thread each would perform better; plenty of room (but not plenty of my energy) for yet more experimentation. what do we expect? the graph should be pretty flat up to simultaneous connections, then it should start getting worse, pretty evenly as simultaneous connections rise all the way up to . it is indeed pretty flat up to simultaneous connections. then up to it’s still not too bad — median at is only ~ . median at (!). then it gets worse after (oh yeah, hyperthreads?). but the slope is wonderfully shallow all the way. even at simultaneous connections, the median response time of ms is only . x what it was at one connection. (in our original t .medium, at simultaneous connections median response time was over x what it was at connection). this thing is indeed a monster. summary comparison: under load we showed a lot of graphs that look similar, but they all had different sclaes on the y-axis. let’s plot median response times under load of all architectures on the same graph, and see what we’re really dealing with. the blue t .medium is our baseline, what we have now. we can see that there isn’t really a similar heroku option, we have our choice of better or worse. the performance-l is just plain better than what we have now. it starts out performing about the same as what we have now for or simultaneous connections, but then scales so much flatter. the performance-m also starts out about thesame, but sccales so much worse than even what we have now. (it’s that real cpu instead of , i guess?). the standard- x scaled to dynos… has it’s own characteristics. it’s baseline is pretty terrible, it’s to times as slow as what we have now even not under load. but then it scales pretty well, since it’s dynos after all, it doesn’t get worse as fast as performance-m does. but it started out so bad, that it remains far worse than our original t .medium even under load. adding more dynos to standard- x will help it remain steady under even higher load, but won’t help it’s underlying problem that it’s just slower than everyone else. discussion: thoughts and surprises i had been thinking of a t .medium (even with burst) as “typical” (it is after all much slower than my macbook), and has been assuming (in retrospect with no particular basis) that a heroku standard dyno would perform similarly. most discussion and heroku docs, as well as the naming itself, suggest that a ‘standard’ dyno is, well, standard, and performance dynos are for “super scale, high traffic apps”, which is not me. but in fact, heroku standard dynos are much slower and more variable in performance than a bursting t .medium. i suspect they are slower than other options you might consider non-heroku “typical” options. my conclusion is honestly that “standard” dynos are really “for very fast, well-optimized apps that can handle slow and variable cpu” and “performance” dynos are really “standard, matching the cpu speeds you’d get from a typical non-heroku option”. but this is not how they are documented or usually talked about. are other people having really different experiences/conclusions than me? if so, why, or where have i gone wrong? this of course has implications for estimating your heroku budget if considering switching over. :( if you have a well-optimized fast app, say even th percentile is ms (on bursting t .medium), then you can handle standard slowness — so what your th percentile is now ms (and during some time periods even much slower, s or worse, due to variability). that’s not so bad for a th percentile. one way to get a very fast is of course caching. there is lots of discussion of using caching in rails, sometimes the message (explicit or implicit) is “you have to use lots of caching to get reasonable performance cause rails is so slow.” what if many of these people are on heroku, and it’s really you have to use lots of caching to get reasonable performance on heroku standard dyno?? i personally don’t think caching is maintenance free; in my experience properly doing cache invalidation and dealing with significant processing spikes needed when you choose to invalidate your entire cache (cause cached html needs to change) lead to real maintenance/development cost. i have not needed caching to meet my performance goals on present architecture. everyone doesn’t necessarily have the same performance goals/requirements. mine of a low-traffic non-commercial site are are maybe more modest, i just need users not to be super annoyed. but whatever your performance goals, you’re going to have to spend more time on optimization on a heroku standard than something with much faster cpu — like a standard affordable mid-tier ec . am i wrong? one significant factor on heroku standard dyno performance is that they use shared/multi-tenant infrastructure. i wonder if they’ve actually gotten lower performance over time, as many customers (who you may be sharing with) have gotten better at maximizing their utilization, so the shared cpus are typically more busy? like a frog boiling, maybe nobody noticed that standard dynos have become lower performance? i dunno, brainstorming. or maybe there are so many apps that start on heroku instead of switcching from somewhere else, that people just don’t realize that standard dynos are much slower than other low/mid-tier options? i was expecting to pay a premium for heroku — but even standard- x’s are a significant premium over paying for t .medium ec yourself, one i found quite reasonable…. performance dynos are of course even more premium. i had a sort of baked-in premise that most rails apps are “io-bound”, they spend more time waiting on io than using cpu. i don’t know where i got that idea, i heard it once a long time ago and it became part of my mental model. i now do not believe this is true true of my app, and i do not in fact believe it is true of most rails apps in . i would hypothesize that most rails apps today are in fact cpu-bound. the performance-m dyno only has one cpu. i had somehow also been assuming that it would have two cpus — i’m not sure why, maybe just because at that price! it would be a much better deal with two cpus. instead we have a huge jump from $ performance-m to $ performance-l that has x the cpus and ~ x the ram. so it doesn’t make financial sense to have more than one performance-m dyno, you might as well go to performance-l. but this really complicates auto-scaling, whether using heroku’s feature , or the awesome rails autoscale add-on. i am not sure i can afford a performance-l all the time, and a performance-m might be sufficient most of the time. but if % of the time i’m going to need more (or even %, or even unexpectedly-mentioned-in-national-media), it would be nice to set things up to autoscale up…. i guess to financially irrational or more performance-m’s? :( the performance-l is a very big machine, that is significantly beefier than my current infrastructure. and has far more ram than i need/can use with only physical cores. if i consider standard dynos to be pretty effectively low tier (as i do), heroku to me is kind of missing mid-tier options. a cpu option at . g or g of ram would make a lot of sense to me, and actually be exactly what i need… really i think performance-m would make more sense with cpus at it’s existing already-premium price point, and to be called a “performance” dyno. . maybe heroku is intentionally trying set options to funnel people to the highest-priced performance-l. conclusion: what are we going to do? in my investigations of heroku, my opinion of the developer ux and general service quality only increases. it’s a great product, that would increase our operational capacity and reliability, and substitute for so many person-hours of sysadmin/operational time if we were self-managing (even on cloud architecture like ec ). but i had originally been figuring we’d use standard dynos (even more affordably, possibly auto-scaled with rails autoscale plugin), and am disappointed that they end up looking so much lower performance than our current infrastructure. could we use them anyway? response time going from ms to ms — hey, ms is still fine, even if i’m sad to lose those really nice numbers i got from a bit of optimization. but this app has a wide long-tail ; our th percentile going from ms to s, our th percentile going from ms to . s and our th going from . s to . s — a lot harder to swallow. especially when we know that due to standard dyno variability, a slow-ish page that on my present architecture is reliably . s, could really be anywhere from to (!) on heroku. i would anticipate having to spend a lot more developer time on optimization on heroku standard dynos — or, i this small over-burdened non-commercial shop, not prioritizing that (or not having the skills for it), and having our performance just get bad. so i’m really reluctant to suggest moving our app to heroku with standard dynos. a performance-l dyno is going to let us not have to think about performance any more than we do now, while scaling under high-traffic better than we do now — i suspect we’d never need to scale to more than one performance-l dyno. but it’s pricey for us. a performance-m dyno has a base-speed that’s fine, but scales very poorly and unaffordably. doesn’t handle an increase in load very well as one dyno, and to get more cpus you have to pay far too much (especially compared to standard dynos i had been assuming i’d use). so i don’t really like any of my options. if we do heroku, maybe we’ll try a performance-m, and “hope” our traffic is light enough that a single one will do? maybe with rails autoscale for traffic spikes, even though performance-m dynos isn’t financially efficient? if we are scaling to (or more!) performance-m’s more than very occasionally, switch to performance-l, which means we need to make sure we have the budget for it? share this: twitter facebook tagged ruby published by jrochkind view all posts by jrochkind published november , november , post navigation previous post deep dive: moving ruby projects from travis to github actions for ci next post are you talking to heroku redis in cleartext or ssl? leave a reply cancel reply enter your comment here... fill in your details below or click an icon to log in: email (required) (address never made public) name (required) website you are commenting using your wordpress.com account. ( log out / change ) you are commenting using your google account. ( log out / change ) you are commenting using your twitter account. ( log out / change ) you are commenting using your facebook account. ( log out / change ) cancel connecting to %s notify me of new comments via email. notify me of new posts via email. bibliographic wilderness is a blog by jonathan rochkind about digital library services, ruby, and web development. contact search for: email subscription enter your email address to subscribe to this blog and receive notifications of new posts by email. join other followers email address: subscribe recent posts product management february , rails auto-scaling on heroku january , managed solr saas options january , gem authors, check your release sizes january , every time you decide to solve a problem with code… january , archives archives select month february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) april ( ) march ( ) december ( ) october ( ) september ( ) august ( ) june ( ) april ( ) march ( ) february ( ) december ( ) november ( ) october ( ) september ( ) august ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) november ( ) october ( ) september ( ) august ( ) july ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) september ( ) august ( ) june ( ) may ( ) march ( ) february ( ) january ( ) november ( ) october ( ) september ( ) august ( ) july ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) september ( ) august ( ) june ( ) may ( ) april ( ) march ( ) feeds rss - posts rss - comments recent comments jrochkind on rails auto-scaling on heroku adam (rails autoscale) on rails auto-scaling on heroku on catalogers, programmers, and user tasks – gavia libraria on broad categories from class numbers replacing marc – gavia libraria on linked data caution jrochkind on deep dive: moving ruby projects from travis to github actions for ci jrochkind on deep dive: moving ruby projects from travis to github actions for ci jrochkind on deep dive: moving ruby projects from travis to github actions for ci eregontp on deep dive: moving ruby projects from travis to github actions for ci top posts yes, product owner and technical lead need to be different people bootstrap to : changes in how font size, line-height, and spacing is done. or "what happened to $line-height-computed." dealing with legacy and externally loaded code in webpack(er) activerecord: atomic check-and-update through optimistic locking are you talking to heroku redis in cleartext or ssl? top clicks bibwild.files.wordpress.c… apidock.com/rails/activer… github.com/mperham/sideki… bibwild.files.wordpress.c… opensolr.com a blog by jonathan rochkind. all original content licensed cc-by. create a website or blog at wordpress.com add your thoughts here... (optional) post to cancel privacy & cookies: this site uses cookies. by continuing to use this website, you agree to their use. to find out more, including how to control cookies, see here: cookie policy product management – bibliographic wilderness skip to content bibliographic wilderness menu about contact product management jrochkind general february , in my career working in the academic sector, i have realized that one thing that is often missing from in-house software development is “product management.” but what does that mean exactly? you don’t know it’s missing if you don’t even realize it’s a thing and people can use different terms to mean different roles/responsibilities. basically, deciding what the software should do. this is not about colors on screen or margins (what our stakeholderes often enjoy micro-managing) — i’d consider those still the how of doing it, rather than the what to do. the what is often at a much higher level, about what features or components to develop at all. when done right, it is going to be based on both knowledge of the end-user’s needs and preferences (user research); but also knowledge of internal stakeholder’s desires and preferences (overall organiational strategy, but also just practically what is going to make the right people happy to keep us resourced). also knowledge of the local capacity, what pieces do we need to put in place to get these things developed. when done seriously, it will necessarily involve prioritization — there are many things we could possibly done, some subset of them we very well may do eventually, but which ones should we do now? my experience tells me it is a very big mistake to try to have a developer doing this kind of product management. not because a developer can’t have the right skillset to do them. but because having the same person leading development and product management is a mistake. the developer is too close to the development lense, and there’s just a clarification that happens when these roles are separate. my experience also tells me that it’s a mistake to have a committee doing these things, much as that is popular in the academic sector. because, well, just of course it is. but okay this is all still pretty abstract. things might become more clear if we get more specific about the actual tasks and work of this kind of product management role. i found damilola ajiboye blog post on “product manager vs product marketing manager vs product owner” very clear and helpful here. while it is written so as to distinguish between three different product management related roles, but ajiboye also acknowledges that in a smaller organization “a product manager is often tasked with the duty of these roles. regardless of if the responsibilities are to be done by one or two or three person, ajiboye’s post serves as a concise listing of the work to be done in managing a product — deciding the what of the product, in an ongoing iterative and collaborative manner, so that developers and designers can get to the how and to implementation. i recommend reading the whole article, and i’ll excerpt much of it here, slightly rearranged. the product manager these individuals are often referred to as mini ceos of a product. they conduct customer surveys to figure out the customer’s pain and build solutions to address it. the pm also prioritizes what features are to be built next and prepares and manages a cohesive and digital product roadmap and strategy. the product manager will interface with the users through user interviews/feedback surveys or other means to hear directly from the users. they will come up with hypotheses alongside the team and validate them through prototyping and user testing. they will then create a strategy on the feature and align the team and stakeholders around it. the pm who is also the chief custodian of the entire product roadmap will, therefore, be tasked with the duty of prioritization. before going ahead to carry out research and strategy, they will have to convince the stakeholders if it is a good choice to build the feature in context at that particular time or wait a bit longer based on the content of the roadmap. the product marketing manager the pmm communicates vital product value — the “why”, “what” and “when” of a product to intending buyers. he manages the go-to-market strategy/roadmap and also oversees the pricing model of the product. the primary goal of a pmm is to create demand for the products through effective messaging and marketing programs so that the product has a shorter sales cycle and higher revenue. the product marketing manager is tasked with market feasibility and discovering if the features being built align with the company’s sales and revenue plan for the period. they also make research on how sought-after the feature is being anticipated and how it will impact the budget. they communicate the values of the feature; the why, what, and when to potential buyers — in this case users in countries with poor internet connection. [while expressed in terms of a for-profit enterprise selling something, i think it’s not hard to translate this to a non-profit or academic environment. you still have an audience whose uptake you need to be succesful, whether internal or external. — jrochkind ] the product owner a product owner (po) maximizes the value of a product through the creation and management of the product backlog, creation of user stories for the development team. the product owner is the customer’s representative to the development team. he addresses customer’s pain points by managing and prioritizing a visible product backlog. the po is the first point of call when the development team needs clarity about interpreting a product feature to be implemented. the product owner will first have to prioritize the backlog to see if there are no important tasks to be executed and if this new feature is worth leaving whatever is being built currently. they will also consider the development effort required to build the feature i.e the time, tools, and skill set that will be required. they will be the one to tell if the expertise of the current developers is enough or if more engineers or designers are needed to be able to deliver at the scheduled time. the product owner is also armed with the task of interpreting the product/feature requirements for the development team. they serve as the interface between the stakeholders and the development team. when you have someone(s) doing these roles well, it ensures that the development team is actually spending time on things that meet user and business needs. i have found that it makes things so much less stressful and more rewarding for everyone involved. when you have nobody doing these roles, or someone doing it in a cursory or un-intentional way not recognized as part of their core job responsibilities, or have a lead developer trying to do it on top of develvopment, i find it leads to feelings of: spinning wheels, everything-is-an-emergency, lack of appreciation, miscommunication and lack of shared understanding between stakeholders and developers, general burnout and dissatisfaction — and at the root, a product that is not meeting user or business needs well, leading to these inter-personal and personal problems. share this: twitter facebook published by jrochkind view all posts by jrochkind published february , post navigation previous post rails auto-scaling on heroku leave a reply cancel reply enter your comment here... fill in your details below or click an icon to log in: email (required) (address never made public) name (required) website you are commenting using your wordpress.com account. ( log out / change ) you are commenting using your google account. ( log out / change ) you are commenting using your twitter account. ( log out / change ) you are commenting using your facebook account. ( log out / change ) cancel connecting to %s notify me of new comments via email. notify me of new posts via email. bibliographic wilderness is a blog by jonathan rochkind about digital library services, ruby, and web development. contact search for: email subscription enter your email address to subscribe to this blog and receive notifications of new posts by email. join other followers email address: subscribe recent posts product management february , rails auto-scaling on heroku january , managed solr saas options january , gem authors, check your release sizes january , every time you decide to solve a problem with code… january , archives archives select month february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) april ( ) march ( ) december ( ) october ( ) september ( ) august ( ) june ( ) april ( ) march ( ) february ( ) december ( ) november ( ) october ( ) september ( ) august ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) november ( ) october ( ) september ( ) august ( ) july ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) september ( ) august ( ) june ( ) may ( ) march ( ) february ( ) january ( ) november ( ) october ( ) september ( ) august ( ) july ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) august ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) october ( ) september ( ) july ( ) june ( ) may ( ) april ( ) march ( ) february ( ) january ( ) december ( ) november ( ) september ( ) august ( ) june ( ) may ( ) april ( ) march ( ) feeds rss - posts rss - comments recent comments jrochkind on rails auto-scaling on heroku adam (rails autoscale) on rails auto-scaling on heroku on catalogers, programmers, and user tasks – gavia libraria on broad categories from class numbers replacing marc – gavia libraria on linked data caution jrochkind on deep dive: moving ruby projects from travis to github actions for ci jrochkind on deep dive: moving ruby projects from travis to github actions for ci jrochkind on deep dive: moving ruby projects from travis to github actions for ci eregontp on deep dive: moving ruby projects from travis to github actions for ci top posts yes, product owner and technical lead need to be different people bootstrap to : changes in how font size, line-height, and spacing is done. or "what happened to $line-height-computed." dealing with legacy and externally loaded code in webpack(er) activerecord: atomic check-and-update through optimistic locking are you talking to heroku redis in cleartext or ssl? top clicks bibwild.files.wordpress.c… apidock.com/rails/activer… github.com/mperham/sideki… bibwild.files.wordpress.c… opensolr.com a blog by jonathan rochkind. all original content licensed cc-by. create a website or blog at wordpress.com privacy & cookies: this site uses cookies. by continuing to use this website, you agree to their use. to find out more, including how to control cookies, see here: cookie policy ddosers are abusing microsoft rdp to make attacks more powerful | ars technica skip to main content biz & it tech science policy cars gaming & culture store forums subscribe close navigate store subscribe videos features reviews rss feeds mobile site about ars staff directory contact us advertise with ars reprints filter by topic biz & it tech science policy cars gaming & culture store forums settings front page layout grid list site theme black on white white on black sign in comment activity sign up or login to join the discussions! stay logged in | having trouble? sign up to comment and more sign up building a better ddos — ddosers are abusing microsoft rdp to make attacks more powerful ddos amplification attacks have abused all kinds of legit services. now, it's windows. dan goodin - jan , : pm utc enlarge / hacker attacking server or database. network security, database secure and personal data protection getty images reader comments with posters participating share this story share on facebook share on twitter share on reddit ddos-for-hire services are abusing the microsoft remote desktop protocol to increase the firepower of distributed denial-of-service attacks that paralyze websites and other online services, a security firm said this week. typically abbreviated as rdp, remote desktop protocol is the underpinning for a microsoft windows feature that allows one device to log into another device over the internet. rdp is mostly used by businesses to save employees the cost or hassle of having to be physically present when accessing a computer. as is typical with many authenticated systems, rdp responds to login requests with a much longer sequence of bits that establish a connection between the two parties. so-called booter/stresser services, which for a fee will bombard internet addresses with enough data to take them offline, have recently embraced rdp as a means to amplify their attacks, security firm netscout said. the amplification allows attackers with only modest resources to strengthen the size of the data they direct at targets. the technique works by bouncing a relatively small amount of data at the amplifying service, which in turn reflects a much larger amount of data at the final target. with an amplification factor of . to , gigabytes-per-second of requests directed at an rdp server will deliver roughly gbps to the target. “observed attack sizes range from ~ gbps – ~ gbps,” netscout researchers wrote. “as is routinely the case with newer ddos attack vectors, it appears that after an initial period of employment by advanced attackers with access to bespoke ddos attack infrastructure, rdp reflection/amplification has been weaponized and added to the arsenals of so-called booter/stresser ddos-for-hire services, placing it within the reach of the general attacker population.” advertisement further reading in-the-wild ddoses use new way to achieve unthinkable sizes ddos amplification attacks date back decades. as legitimate internet users collectively block one vector, attackers find new ones to take their place. ddos amplifiers have included open dns resolvers, the ws-discovery protocol used by iot devices, and the internet’s network time protocol. one of the most powerful amplification vectors in recent memory is the so-called memcached protocol which has a factor of , to . ddos amplification attacks work by using udp network packets, which are easily spoofable on many networks. an attacker sends the vector a request and spoofs the headers to give the appearance the request came from the target. the amplification vector then sends the response to the target whose address appears in the spoofed packets. there are about , rdp servers on the internet that can be abused in amplification attacks, netscout said. besides using udp packets, rdp can also rely on tcp packets. netscout recommended that rdp servers be accessible only over virtual private network services. in the event rdp servers offering remote access over udp can’t be immediately moved behind vpn concentrators, administrators should disable rdp over udp as an interim measure. besides harming the internet as a whole, unsecured rdp can be a hazard to the organizations that expose them to the internet. “the collateral impact of rdp reflection/amplification attacks is potentially quite high for organizations whose windows rdp servers are abused as reflectors/amplifiers,” netscout explained. “this may include partial or full interruption of mission-critical remote-access services, as well as additional service disruption due to transit capacity consumption, state-table exhaustion of stateful firewalls, load balancers, etc.” reader comments with posters participating share this story share on facebook share on twitter share on reddit dan goodin dan is the security editor at ars technica, which he joined in after working for the register, the associated press, bloomberg news, and other publications. email dan.goodin@arstechnica.com // twitter @dangoodin advertisement you must login or create an account to comment. channel ars technica ← previous story next story → related stories sponsored stories powered by today on ars store subscribe about us rss feeds view mobile site contact us staff advertise with us reprints newsletter signup join the ars orbital transmission mailing list to get weekly updates delivered to your inbox. sign me up → cnmn collection wired media group © condé nast. all rights reserved. use of and/or registration on any portion of this site constitutes acceptance of our user agreement (updated / / ) and privacy policy and cookie statement (updated / / ) and ars technica addendum (effective / / ). ars may earn compensation on sales from links on this site. read our affiliate link policy. your california privacy rights | do not sell my personal information the material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of condé nast. ad choices open knowledge foundation open knowledge justice programme newsletter * indicates required name: email: comment: email address * first name * last name * profession * barrister, solicitor (uk) or lawyer (non-uk) judge academic ngo officer government officer other you are... * in the uk outside of the uk in which area(s) of the law are you interested? public law criminal law environmental law ip law tort law family law insurance law banking law commercial law employment law other contact permissions * please select all the ways you would like to hear from open knowledge justice programme: email you can unsubscribe at any time by clicking the link in the footer of our emails. your information will not be used for any other purpose than to inform you about the open knowledge justice programme. we use mailchimp as our marketing platform. by clicking below to subscribe, you acknowledge that your information will be transferred to mailchimp for processing. learn more about mailchimp's privacy practices here. one-stop counterfeit certificate shops for all your malware-signing needs | ars technica skip to main content biz & it tech science policy cars gaming & culture store forums subscribe close navigate store subscribe videos features reviews rss feeds mobile site about ars staff directory contact us advertise with ars reprints filter by topic biz & it tech science policy cars gaming & culture store forums settings front page layout grid list site theme black on white white on black sign in comment activity sign up or login to join the discussions! stay logged in | having trouble? sign up to comment and more sign up for sale — one-stop counterfeit certificate shops for all your malware-signing needs certificates registered in names of real corporations are surprisingly easy to come by. dan goodin - feb , : pm utc a digital signature used by malware that infected the network of kaspersky lab in . counterfeit certificates that generate such fraudulent signatures are being sold online for use in other malware. kaspersky lab reader comments with posters participating share this story share on facebook share on twitter share on reddit the stuxnet worm that targeted iran's nuclear program almost a decade ago was a watershed piece of malware for a variety of reasons. chief among them, its use of cryptographic certificates belonging to legitimate companies to falsely vouch for the trustworthiness of the malware. last year, we learned that fraudulently signed malware was more widespread than previously believed. on thursday, researchers unveiled one possible reason: underground services that since have sold counterfeit signing credentials that are unique to each buyer. further reading stuxnet-style code signing is more widespread than anyone thought in many cases, the certificates are required to install software on windows and macos computers, while in others, they prevent the oses from displaying warnings that the software comes from an untrusted developer. the certificates also increase the chances that antivirus programs won't flag previously unseen files as malicious. a report published by threat intelligence provider recorded future said that starting last year, researchers saw a sudden increase in fraudulent certificates issued by browser- and operating system-trusted providers that were being used to sign malicious wares. the spike drove recorded future researchers to investigate the cause. what they found was surprising. "contrary to a common belief that the security certificates circulating in the criminal underground are stolen from legitimate owners prior to being used in nefarious campaigns, we confirmed with a high degree of certainty that the certificates are created for a specific buyer per request only and are registered using stolen corporate identities, making traditional network security appliances less effective," andrei barysevich, a researcher at recorded future, reported. advertisement barysevich identified four such sellers of counterfeit certificates since . two of them remain in business today. the sellers offered a variety of options. in , one provider calling himself c@t advertised certificates that used a microsoft technology known as authenticode for signing executable files and programming scripts that can install software. c@t offered code-signing certificates for macos apps as well. his fee: upwards of $ , per certificate. "in his advertisement, c@t explained that the certificates are registered under legitimate corporations and issued by comodo, thawte, and symantec—the largest and most respected issuers," thursday's report said. "the seller indicated that each certificate is unique and will only be assigned to a single buyer, which could be easily verified via herdprotect.com. according to c@t, the success rate of payload installations from signed files increases by to percent, and he even admitted to selling over certificates in less than six months." c@t's business dwindled in coming years as other providers undercut his prices. one competing service provided a bare-bones code-signing certificate for $ . for $ , , the service sold a signing certificate with extended validation—meaning it was issued to a corporate or business name that had been verified by the issuer. that premium price also ensured the certificate passed the smartscreen validation check various microsoft software perform to protect users against malicious apps. a package of fully authenticated internet domains with ev ssl encryption and code signing capabilities could also be purchased for $ , . the same service sold extended validation tls certificates for websites starting at $ . a different c@t competitor sold highly vetted class certificates for $ . enlarge recorded future "according to the information provided by both sellers during a private conversation, to guarantee the issuance and lifespan of the products, all certificates are registered using the information of real corporations," barysevich wrote. "with a high degree of confidence, we believe that the legitimate business owners are unaware that their data was used in the illicit activities. it is important to note that all certificates are created for each buyer individually with the average delivery time of two to four days." advertisement use of legitimate signing certificates to verify malicious apps and legitimate tls certificates to authenticate domain names that distribute those apps can make security protections less effective. recorded future researchers provided one seller with an unreported remote access trojan and convinced the seller to sign it with a certificate that had been recently issued by comodo. only eight of the top av providers detected an encrypted version of the trojan. only two av engines detected the same encrypted file when it was signed by the comodo certificate. "more disturbing results surfaced after the same test was conducted for a non-resident version of the payload," barysevich reported. "in that case, only six companies were capable of detecting an encrypted version, and only endgame protection recognized the file as malicious." while thursday's report shows how simple it is to bypass many of the protections provided by code-signing requirements, barysevich said that counterfeit certificates are likely to be used only in niche campaigns that target a small number of people or organizations. "although code signing certificates can be effectively used in widespread malware campaigns such as the distribution of banking trojan or ransomware, the validity of the certificate used to sign a payload would be invalidated fairly quickly," he explained. "therefore, we believe that the limited number of power-users specializing in more sophisticated and targeted campaigns, such as corporate espionage, is the main driving force behind the new service." reader comments with posters participating share this story share on facebook share on twitter share on reddit dan goodin dan is the security editor at ars technica, which he joined in after working for the register, the associated press, bloomberg news, and other publications. email dan.goodin@arstechnica.com // twitter @dangoodin advertisement you must login or create an account to comment. channel ars technica ← previous story next story → related stories sponsored stories powered by today on ars store subscribe about us rss feeds view mobile site contact us staff advertise with us reprints newsletter signup join the ars orbital transmission mailing list to get weekly updates delivered to your inbox. sign me up → cnmn collection wired media group © condé nast. all rights reserved. use of and/or registration on any portion of this site constitutes acceptance of our user agreement (updated / / ) and privacy policy and cookie statement (updated / / ) and ars technica addendum (effective / / ). ars may earn compensation on sales from links on this site. read our affiliate link policy. your california privacy rights | do not sell my personal information the material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of condé nast. ad choices trump-tweets-wayback : ed summers : free download, borrow, and streaming : internet archive skip to main content see what's new with book lending at the internet archive a line drawing of the internet archive headquarters building façade. an illustration of a magnifying glass. an illustration of a magnifying glass. an illustration of a horizontal line over an up pointing arrow. upload an illustration of a person's head and chest. sign up | log in an illustration of a computer application window wayback machine an illustration of an open book. books an illustration of two cells of a film strip. video an illustration of an audio speaker. audio an illustration of a . " floppy disk. software an illustration of two photographs. images an illustration of a heart shape donate an illustration of text ellipses. more an icon used to represent a menu that can be toggled by interacting with this icon. about blog projects help donate an illustration of a heart shape contact jobs volunteer people search metadata search text contents search tv news captions search archived websites advanced search sign up for free log in trump-tweets-wayback item preview trump-tweets-wayback.png remove-circle share or embed this item embed embed (for wordpress.com hosted blogs and archive.org item tags) [archiveorg trump-tweets-wayback width= height= frameborder= webkitallowfullscreen=true mozallowfullscreen=true] want more? advanced embedding details, examples, and help! no_favorite share flag flag this item for graphic violence graphic sexual content texts trump-tweets-wayback by ed summers publication date - - usage attribution-noderivatives . international topics twitter, trump, internet archive, python collection opensource_media this contains a snapshot of data about trump's tweets that are in the internet archive. it consists of a jupyter notebook that was used to query the internet archive's cdx api to see what snapshots are available for the url prefix https://twitter.com/realdonaldtrump/status/{tweet-id}. the notebook serializes the results as a sqlite database that has information about million snapshots that includes the url, timestamp, size in bytes, http status code, and http media type. the sqlite database needs to be decompressed before the noteboo will work properly. addeddate - - : : identifier trump-tweets-wayback identifier-ark ark:/ /t km b scanner internet archive html uploader . . plus-circle add review comment reviews there are no reviews yet. be the first one to write a review. views download options download file gzip download download file item tile download download file png download download file torrent download download files download original show all in collections community data uploaded by edsu on january , similar items (based on metadata) terms of service (last updated / / ) zbw labs jump to navigation english deutsch main menu news about us news building the swib participants map - - by joachim neubert here we describe the process of building the interactive swib participants map, created by a query to wikidata. the map was intended to support participants of swib to make contacts in the virtual conference space. however, in compliance with gdpr we want to avoid publishing personal details. so we choose to publish a map of institutions, to which the participants are affiliated. (obvious downside: the un-affiliated participants could not be represented on the map). we suppose that the method can be applied to other conferences and other use cases - e.g., the downloaders of scientific software or the institutions subscribed to an academic journal. therefore, we describe the process in some detail. wikidata for authorities linked data read more about building the swib participants map log in or register to post comments journal map: developing an open environment for accessing and analyzing performance indicators from journals in economics - - by timo borst by franz osorio, timo borst introduction bibliometrics, scientometrics, informetrics and webometrics have been both research topics and practical guidelines for publishing, reading, citing, measuring and acquiring published research for a while (hood ). citation databases and measures had been introduced in the s, becoming benchmarks both for the publishing industry and academic libraries managing their holdings and journal acquisitions that tend to be more selective with a growing number of journals on the one side, budget cuts on the other. due to the open access movement triggering a transformation of traditional publishing models (schimmer ), and in the light of both global and distributed information infrastructures for publishing and communicating on the web that have yielded more diverse practices and communities, this situation has dramatically changed: while bibliometrics of research output in its core understanding still is highly relevant to stakeholders and the scientific community, visibility, influence and impact of scientific results has shifted to locations in the world wide web that are commonly shared and quickly accessible not only by peers, but by the general public (thelwall ). this has several implications for different stakeholders who are referring to metrics in dealing with scientific results: with the rise of social networks, platforms and their use also by academics and research communities, the term 'metrics' itself has gained a broader meaning: while traditional citation indexes only track citations of literature published in (other) journals, 'mentions', 'reads' and 'tweets', albeit less formal, have become indicators and measures for (scientific) impact. altmetrics has influenced research performance, evaluation and measurement, which formerly had been exclusively associated with traditional bibliometrics. scientists are becoming aware of alternative publishing channels and both the option and need of 'self-advertising' their output. in particular academic libraries are forced to manage their journal subscriptions and holdings in the light of increasing scientific output on the one hand, and stagnating budgets on the other. while editorial products from the publishing industry are exposed to a global competing market requiring a 'brand' strategy, altmetrics may serve as additional scattered indicators for scientific awareness and value. against this background, we took the opportunity to collect, process and display some impact or signal data with respect to literature in economics from different sources, such as 'traditional' citation databases, journal rankings and community platforms resp. altmetrics indicators: citec. the long-standing citation service maintainted by the repec community provided a dump of both working papers (as part of series) and journal articles, the latter with significant information on classic impact factors such as impact factor ( and years) and h-index. rankings of journals in economics including scimago journal rank (sjr) and two german journal rankings, that are regularly released and updated (vhb jourqual, handelsblatt ranking). usage data from altmetric.com that we collected for those articles that could be identified via their digital object identifier. usage data from the scientific community platform and reference manager mendeley.com, in particular the number of saves or bookmarks on an individual paper. requirements a major consideration for this project was finding an open environment in which to implement it. finding an open platform to use served a few purposes. as a member of the "leibniz research association," zbw has a commitment to open science and in part that means making use of open technologies to as great extent as possible (the zbw - open scienc...). this open system should allow direct access to the underlying data so that users are able to use it for their own investigations and purposes. additionally, if possible the user should be able to manipulate the data within the system. the first instance of the project was created in tableau, which offers a variety of means to express data and create interfaces for the user to filter and manipulate data. it also can provide a way to work with the data and create visualizations without programming skills or knowledge. tableau is one of the most popular tools to create and deliver data visualization in particular within academic libraries (murphy ). however, the software is proprietary and has a monthly fee to use and maintain, as well as closing off the data and making only the final visualization available to users. it was able to provide a starting point for how we wanted to the data to appear to the user, but it is in no way open. challenges the first technical challenge was to consolidate the data from the different sources which had varying formats and organizations. broadly speaking, the bibliometric data (citec and journal rankings) existed as a spread sheet with multiple pages, while the altmetrics and mendeley data came from a database dumps with multiple tables that were presented as several csv files. in addition to these different formats, the data needed to be cleaned and gaps filled in. the sources also had very different scopes. the altmetrics and mendeley data covered only journals, the bibliometric data, on the other hand, had more than , journals. transitioning from tableau to an open platform was big challenge. while there are many ways to create data visualizations and present them to users, the decision was made to use r to work with the data and shiny to present it. r is used widely to work with data and to present it (kläre ). the language has lots of support for these kinds of task over many libraries. the primary libraries used were r plotly and r shiny. plotly is a popular library for creating interactive visualizations. without too much work plotly can provide features including information popups while hovering over a chart and on the fly filtering. shiny provides a framework to create a web application to present the data without requiring a lot of work to create html and css. the transition required time spent getting to know r and its libraries, to learn how to create the kinds of charts and filters that would be useful for users. while shiny alleviates the need to create html and css, it does have a specific set of requirements and structures in order to function. the final challenge was in making this project accessible to users such that they would be able to see what we had done, have access to the data, and have an environment in which they could explore the data without needing anything other than what we were providing. in order to achieve this we used binder as the platform. at it's most basic binder makes it possible to share a jupyter notebook stored in a github repository with a url by running the jupyter notebook remotely and providing access through a browser with no requirements placed on the user. additionally, binder is able to run a web application using r and shiny. to move from a locally running instance of r shiny to one that can run in binder, instructions for the runtime environment need to be created and added to the repository. these include information on what version of the language to use, which packages and libraries to install for the language, and any additional requirements there might be to run everything. solutions given the disparate sources and formats for the data, there was work that needed to be done to prepare it for visualization. the largest dataset, the bibliographic data, had several identifiers for each journal but without journal names. having the journals names is important because in general the names are how users will know the journals. adding the names to the data would allow users to filter on specific journals or pull up two journals for a comparison. providing the names of the journals is also a benefit for anyone who may repurpose the data and saves them from having to look them up. in order to fill this gap, we used metadata available through research papers in economics (repec). repec is an organization that seeks to "enhance the dissemination of research in economics and related sciences". it contains metadata for more than million papers available in different formats. the bibliographic data contained repec handles which we used to look up the journal information as xml and then parse the xml to find the title of the journal. after writing a small python script to go through the repec data and find the missing names there were only journals whose names were still missing. for the data that originated in an mysql database, the major work that needed to be done was to correct the formatting. the data was provided as csv files but it was not formatted such that it could be used right away. some of the fields had double quotation marks and when the csv file was created those quotes were put into other quotation marks resulting doubled quotation marks which made machine parsing difficult without intervention directly on the files. the work was to go through the files and quickly remove the doubled quotation marks. in addition to that, it was useful for some visualizations to provide a condensed version of the data. the data from the database was at the article level which is useful for some things, but could be time consuming for other actions. for example, the altmetrics data covered only journals but had almost , rows. we could use the python library pandas to go through the all those rows and condense the data down so that there are only rows with the data for each column being the sum of all rows. in this way, there is a dataset that can be used to easily and quickly generate summaries on the journal level. shiny applications require a specific structure and files in order to do the work of creating html without needing to write the full html and css. at it's most basic there are two main parts to the shiny application. the first defines the user interface (ui) of the page. it says what goes where, what kind of elements to include, and how things are labeled. this section defines what the user interacts with by creating inputs and also defining the layout of the output. the second part acts as a server that handles the computations and processing of the data that will be passed on to the ui for display. the two pieces work in tandem, passing information back and forth to create a visualization based on user input. using shiny allowed almost all of the time spent on creating the project to be concentrated on processing the data and creating the visualizations. the only difficulty in creating the frontend was making sure all the pieces of the ui and server were connected correctly. binder provided a solution for hosting the application, making the data available to users, and making it shareable all in an open environment. notebooks and applications hosted with binder are shareable in part because the source is often a repository like github. by passing a github repository to binder, say one that has a jupyter notebook in it, binder will build a docker image to run the notebook and then serve the result to the user without them needing to do anything. out of the box the docker image will contain only the most basic functions. the result is that if a notebook requires a library that isn't standard, it won't be possible to run all of the code in the notebook. in order to address this, binder allows for the inclusion in a repository of certain files that can define what extra elements should be included when building the docker image. this can be very specific such as what version of the language to use and listing various libraries that should be included to ensure that the notebook can be run smoothly. binder also has support for more advanced functionality in the docker images such as creating a postgres database and loading it with data. these kinds of activities require using different hooks that binder looks for during the creation of the docker image to run scripts. results and evaluation the final product has three main sections that divide the data categorically into altmetrics, bibliometrics, and data from mendeley. there are additionally some sections that exist as areas where something new could be tried out and refined without potentially causing issues with the three previously mentioned areas. each section has visualizations that are based on the data available. considering the requirements for the project, the result goes a long way to meeting the requirements. the most apparent area that the journal map succeeds in is its goals is of presenting data that we have collected. the application serves as a dashboard for the data that can be explored by changing filters and journal selections. by presenting the data as a dashboard, the barrier to entry for users to explore the data is low. however, there exists a way to access the data directly and perform new calculations, or create new visualizations. this can be done through the application's access to an r-studio environment. access to r-studio provides two major features. first, it gives direct access to the all the underlying code that creates the dashboard and the data used by it. second, it provides an r terminal so that users can work with the data directly. in r-studio, the user can also modify the existing files and then run them from r-studio to see the results. using binder and r as the backend of the applications allows us to provide users with different ways to access and work with data without any extra requirements on the part of the user. however, anything changed in r-studio won't affect the dashboard view and won't persist between sessions. changes exist only in the current session. all the major pieces of this project were able to be done using open technologies: binder to serve the application, r to write the code, and github to host all the code. using these technologies and leveraging their capabilities allows the project to support the open science paradigm that was part of the impetus for the project. the biggest drawback to the current implementation is that binder is a third party host and so there are certain things that are out of our control. for example, binder can be slow to load. it takes on average + minutes for the docker image to load. there's not much, if anything, we can do to speed that up. the other issue is that if there is an update to the binder source code that breaks something, then the application will be inaccessible until the issue is resolved. outlook and future work the application, in its current state, has parts that are not finalized. as we receive feedback, we will make changes to the application to add or change visualizations. as mentioned previously, there a few sections that were created to test different visualizations independently of the more complete sections, those can be finalized. in the future it may be possible to move from binderhub to a locally created and administered version of binder. there is support and documentation for creating local, self hosted instances of binder. going that direction would give more control, and may make it possible to get the docker image to load more quickly. while the application runs stand-alone, the data that is visualized may also be integrated in other contexts. one option we are already prototyping is integrating the data into our subject portal econbiz, so users would be able to judge the scientific impact of an article in terms of both bibliometric and altmetric indicators. references william w. hood, concepcion s. wilson. the literature of bibliometrics, scientometrics, and informetrics. scientometrics , – springer science and business media llc, . link r. schimmer. disrupting the subscription journals’ business model for the necessary large-scale transformation to open access. ( ). link mike thelwall, stefanie haustein, vincent larivière, cassidy r. sugimoto. do altmetrics work? twitter and ten other social web services. plos one , e public library of science (plos), . link the zbw - open science future. link sarah anne murphy. data visualization and rapid analytics: applying tableau desktop to support library decision-making. journal of web librarianship , – informa uk limited, . link christina kläre, timo borst. statistic packages and their use in research in economics | edawax - blog of the project ’european data watch extended’. edawax - european data watch extended ( ). link journal map - binder application for displaying and analyzing metrics data about scientific journals read more about journal map: developing an open environment for accessing and analyzing performance indicators from journals in economics log in or register to post comments integrating altmetrics into a subject repository - econstor as a use case - - by wolfgang riese back in the zbw leibniz information center for economics (zbw) teamed up with the göttingen state and university library (sub), the service center of götting library federation (vzg) and gesis leibniz institute for the social sciences in the *metrics project funded by the german research foundation (dfg). the aim of the project was: “… to develop a deeper understanding of *metrics, especially in terms of their general significance and their perception amongst stakeholders.” (*metrics project about). in the practical part of the project the following dspace based repositories of the project partners participated as data sources for online publications and – in the case of econstor – also as implementer for the presentation of the social media signals: econstor - a subject repository for economics and business studies run by the zbw, currently (aug. ) containing round about , downloadable files, goescholar - the publication server of the georg-august-universität göttingen run by the sub göttingen, offering approximately , publicly browsable items so far, ssoar - the “social science open access repository” maintained by gesis, currently containing about , publicly available items. in the work package “technology analysis for the collection and provision of *metrics” of the project an analysis of currently available *metrics technologies and services had been performed. as stated by [wilsdon ], currently suppliers of altmetrics “remain too narrow (mainly considering research products with dois)”, which leads to problems to acquire *metrics data for repositories like econstor with working papers as the main content. as up to now it is unusual – at least in the social sciences and economics – to create dois for this kind of documents. only the resulting final article published in a journal will receive a doi. based on the findings in this work package, a test implementation of the *metrics crawler had been built. the crawler had been actively deployed from early to spring at the vzg. for the aggregation of the *metrics data the crawler had been fed with persistent identifiers and metadata from the aforementioned repositories. at this stage of the project, the project partners still had the expectation, that the persistent identifiers (e.g. handle, urns, …), or their local url counterparts, as used by the repositories could be harnessed to easily identify social media mentions of their documents, e.g. for econstor: handle: “hdl: /…” handle.net resolver url: “http(s)://hdl.handle.net/ /…” econstor landing page url with handle: “http(s)://www.econstor.eu/handle/ /…” econstor bitstream (pdf) url with handle: “http(s)://www.econstor.eu/bitstream/ /…” integrating altmetrics data into econstor read more about integrating altmetrics into a subject repository - econstor as a use case log in or register to post comments th century press archives: data donation to wikidata - - by joachim neubert zbw is donating a large open dataset from the th century press archives to wikidata, in order to make it better accessible to various scientific disciplines such as contemporary, economic and business history, media and information science, to journalists, teachers, students, and the general public. the th century press archives (pm ) is a large public newspaper clippings archive, extracted from more than different sources published in germany and all over the world, covering roughly a full century ( - ). the clippings are organized in thematic folders about persons, companies and institutions, general subjects, and wares. during a project originally funded by the german research foundation (dfg), the material up to has been digitized. , folders with more than two million pages up to are freely accessible online. the fine-grained thematic access and the public nature of the archives makes it to our best knowledge unique across the world (more information on wikipedia) and an essential research data fund for some of the disciplines mentioned above. the data donation does not only mean that zbw has assigned a cc license to all pm metadata, which makes it compatible with wikidata. (due to intellectual property rights, only the metadata can be licensed by zbw - all legal rights on the press articles themselves remain with their original creators.) the donation also includes investing a substantial amount of working time (during, as planned, two years) devoted to the integration of this data into wikidata. here we want to share our experiences regarding the integration of the persons archive metadata. linked data open data read more about th century press archives: data donation to wikidata log in or register to post comments zbw's contribution to "coding da vinci": dossiers about persons and companies from th century press archives - - by joachim neubert at th and th of october, the kick-off for the "kultur-hackathon" coding da vinci is held in mainz, germany, organized this time by glam institutions from the rhein-main area: "for five weeks, devoted fans of culture and hacking alike will prototype, code and design to make open cultural data come alive." new software applications are enabled by free and open data. for the first time, zbw is among the data providers. it contributes the person and company dossiers of the th century press archive. for about a hundred years, the predecessor organizations of zbw in kiel and hamburg had collected press clippings, business reports and other material about a wide range of political, economic and social topics, about persons, organizations, wares, events and general subjects. during a project funded by the german research organization (dfg), the documents published up to (about , million pages) had been digitized and are made publicly accessible with according metadata, until recently solely in the "pressemappe . jahrhundert" (pm ) web application. additionally, the dossiers - for example about mahatma gandhi or the hamburg-bremer afrika linie - can be loaded into a web viewer. as a first step to open up this unique source of data for various communities, zbw has decided to put the complete pm metadata* under a cc-zero license, which allows free reuse in all contexts. for our coding da vinci contribution, we have prepared all person and company dossiers which already contain documents. the dossiers are interlinked among each other. controlled vocabularies (for, e.g., "country", or "field of activity") provide multi-dimensional access to the data. most of the persons and a good share of organizations were linked to gnd identifiers. as a starter, we had mapped dossiers to wikidata according to existing gnd ids. that allows to run queries for pm dossiers completely on wikidata, making use of all the good stuff there. an example query shows the birth places of pm economists on a map, enriched with images from wikimedia commons. the initial mapping was much extended by fantastic semi-automatic and manual mapping efforts by the wikidata community. so currently more than % of the dossiers about - often rather prominent - pm persons are linked not only to wikidata, but also connected to wikipedia pages. that offers great opportunities for mash-ups to further data sources, and we are looking forward to what the "coding da vinci" crowd may make out of these opportunities. technically, the data has been converted from an internal intermediate format to still quite experimental rdf and loaded into a sparql endpoint. there it was enriched with data from wikidata and extracted with a construct query. we have decided to transform it to json-ld for publication (following practices recommended by our hbz colleagues). so developers can use the data as "plain old json", with the plethora of web tools available for this, while linked data enthusiasts can utilize sophisticated semantic web tools by applying the provided json-ld context. in order to make the dataset discoverable and reusable for future research, we published it persistently at zenodo.org. with it, we provide examples and data documentation. a github repository gives you additional code examples and a way to address issues and suggestions. * for the scanned documents, the legal regulations apply - zbw cannot assign licenses here. pressemappe . jahrhundert linked data read more about zbw's contribution to "coding da vinci": dossiers about persons and companies from th century press archives log in or register to post comments wikidata as authority linking hub: connecting repec and gnd researcher identifiers - - by joachim neubert in the econbiz portal for publications in economics, we have data from different sources. in some of these sources, most notably zbw's "econis" bibliographical database, authors are disambiguated by identifiers of the integrated authority file (gnd) - in total more than , . data stemming from "research papers in economics" (repec) contains another identifier: repec authors can register themselves in the repec author service (ras), and claim their papers. this data is used for various rankings of authors and, indirectly, of institutions in economics, which provides a big incentive for authors - about , have signed into ras - to keep both their article claims and personal data up-to-date. while gnd is well known and linked to many other authorities, ras had no links to any other researcher identifier system. thus, until recently, the author identifiers were disconnected, which precludes the possibility to display all publications of an author on a portal page. to overcome that limitation, colleagues at zbw have matched a good , authors with ras and gnd ids by their publications (see details here). making that pre-existing mapping maintainable and extensible however would have meant to set up some custom editing interface, would have required storage and operating resources and wouldn't easily have been made publicly accessible. in a previous article, we described the opportunities offered by wikidata. now we made use of it. wikidata for authorities authority control wikidata read more about wikidata as authority linking hub: connecting repec and gnd researcher identifiers log in or register to post comments new version of multi-lingual jel classification published in lod - - by joachim neubert the journal of economic literature classification scheme (jel) was created and is maintained by the american economic association. the aea provides this widely used resource freely for scholarly purposes. thanks to andré davids (ku leuven), who has translated the originally english-only labels of the classification to french, spanish and german, we provide a multi-lingual version of jel. it's lastest version (as of - ) is published in the formats rdfa and rdf download files. these formats and translations are provided "as is" and are not authorized by aea. in order to make changes in jel tracable more easily, we have created lists of inserted and removed jel classes in the context of the skos-history project. jel klassifikation für linked open data skos-history linked data read more about new version of multi-lingual jel classification published in lod log in or register to post comments economists in wikidata: opportunities of authority linking - - by joachim neubert wikidata is a large database, which connects all of the roughly wikipedia projects. besides interlinking all wikipedia pages in different languages about a specific item – e.g., a person -, it also connects to more than different sources of authority information. the linking is achieved by a „authority control“ class of wikidata properties. the values of these properties are identifiers, which unambiguously identify the wikidata item in external, web-accessible databases. the property definitions includes an uri pattern (called „formatter url“). when the identifier value is inserted into the uri pattern, the resulting uri can be used to look up the authoritiy entry. the resulting uri may point to a linked data resource - as it is the case with the gnd id property. this, on the one hand, provides a light-weight and robust mechanism to create links in the web of data. on the other hand, these links can be exploited by every application which is driven by one of the authorities to provide additional data: links to wikipedia pages in multiple languages, images, life data, nationality and affiliations of the according persons, and much more. wikidata item for the indian economist bina agarwal, visualized via the sqid browser wikidata for authorities wikidata authority control linked data read more about economists in wikidata: opportunities of authority linking log in or register to post comments integrating a research data repository with established research practices - - by timo borst authors: timo borst, konstantin ott in recent years, repositories for managing research data have emerged, which are supposed to help researchers to upload, describe, distribute and share their data. to promote and foster the distribution of research data in the light of paradigms like open science and open access, these repositories are normally implemented and hosted as stand-alone applications, meaning that they offer a web interface for manually uploading the data, and a presentation interface for browsing, searching and accessing the data. sometimes, the first component (interface for uploading the data) is substituted or complemented by a submission interface from another application. e.g., in dataverse or in ckan data is submitted from remote third-party applications by means of data deposit apis [ ]. however the upload of data is organized and eventually embedded into a publishing framework (data either as a supplement of a journal article, or as a stand-alone research output subject to review and release as part of a ‘data journal’), it definitely means that this data is supposed to be made publicly available, which is often reflected by policies and guidelines for data deposit. institutional view on research data read more about integrating a research data repository with established research practices log in or register to post comments content recommendation by means of eexcess - - by timo borst authors: timo borst, nils witt since their beginnings, libraries and related cultural institutions were confident in the fact that users had to visit them in order to search, find and access their content. with the emergence and massive use of the world wide web and associated tools and technologies, this situation has drastically changed: if those institutions still want their content to be found and used, they must adapt themselves to those environments in which users expect digital content to be available. against this background, the general approach of the eexcess project is to ‘inject’ digital content (both metadata and object files) into users' daily environments like browsers, authoring environments like content management systems or google docs, or e-learning environments. content is not just provided, but recommended by means of an organizational and technical framework of distributed partner recommenders and user profiles. once a content partner has connected to this framework by establishing an application program interface (api) for constantly responding to the eexcess queries, the results will be listed and merged with the results of the other partners. depending on the software component installed either on a user’s local machine or on an application server, the list of recommendations is displayed in different ways: from a classical, text-oriented list, to a visualization of metadata records. eexcess recommender recommender system metadata economics read more about content recommendation by means of eexcess log in or register to post comments pages next › last » tags in dbpedia - web taxonomy your browser does not support canvas. application programming interface authority control drupal economics electronic publishing impact factor linked data organizer recommender system repository (publishing) thesaurus wikidata search form search (rdf/xml, turtle, nt) imprint privacy powered by drupal andromeda yelton skip to content andromeda yelton menu home about contact resume hamlet lita talks machine learning (ala midwinter ) boston python meetup (august , ) swib libtechconf code lib keynote texas library association online northwest : five conversations about code new jersey esummit (may , ) westchester library association (january , ) bridging the digital divide with mobile services (webjunction, july ) archival face recognition for fun and nonprofit in , dominique luster gave a super good code lib talk about applying ai to metadata for the charles “teenie” harris collection at the carnegie museum of art — more than , photographs of black life in pittsburgh. they experimented with solutions to various metadata problems, but the one that’s stuck in my head since is the face recognition one. it sure would be cool if you could throw ai at your digitized archival photos to find all the instances of the same person, right? or automatically label them, given that any of them are labeled correctly? sadly, because we cannot have nice things, the data sets used for pretrained face recognition embeddings are things like lots of modern photos of celebrities, a corpus which wildly underrepresents ) archival photos and ) black people. so the results of the face recognition process are not all that great. i have some extremely technical ideas for how to improve this — ideas which, weirdly, some computer science phds i’ve spoken with haven’t seen in the field. so i would like to experiment with them. but i must first invent the universe set up a data processing pipeline. three steps here: fetch archival photographs; do face detection (draw bounding boxes around faces and crop them out for use in the next step); do face recognition. for step , i’m using dpla, which has a super straightforward and well-documented api and an easy-to-use python wrapper (which, despite not having been updated in a while, works just fine with python . , the latest version compatible with some of my dependencies). for step , i’m using mtcnn, because i’ve been following this tutorial. for step , face recognition, i’m using the steps in the same tutorial, but purely for proof-of-concept — the results are garbage because archival photos from mid-century don’t actually look anything like modern-day celebrities. (neural net: “i have % confidence this is stevie wonder!” how nice for you.) clearly i’m going to need to build my own corpus of people, which i have a plan for (i.e. i spent some quality time thinking about numpy) but haven’t yet implemented. so far the gotchas have been: gotcha : if you fetch a page from the api and assume you can treat its contents as an image, you will be sad. you have to treat them as a raw data stream and interpret that as an image, thusly: from pil import image import requests response = requests.get(url, stream=true) response.raw.decode_content = true data = requests.get(url).content image.open(io.bytesio(data)) this code is, of course, hilariously lacking in error handling, despite fetching content from a cesspool of untrustworthiness, aka the internet. it’s a first draft. gotcha : you see code snippets to convert images to pixel arrays (suitable for ai ingestion) that look kinda like this: np.array(image).astype('uint '). except they say astype('float ') instead of astype('uint '). i got a creepy photonegative effect when i used floats. gotcha : although pil was happy to manipulate the .pngs fetched from the api, it was not happy to write them to disk; i needed to convert formats first (image.convert('rgb')). gotcha : the suggested keras_vggface library doesn’t have a pipfile or requirements.txt, so i had to manually install keras and tensorflow. luckily the setup.py documented the correct versions. sadly the tensorflow version is only compatible with python up to . (hence the comment about dpyla compatibility above). i don’t love this, but it got me up and running, and it seems like an easy enough part of the pipeline to rip out and replace if it’s bugging me too much. the plan from here, not entirely in order, subject to change as i don’t entirely know what i’m doing until after i’ve done it: build my own corpus of identified people this means the numpy thoughts, above it also means spending more quality time with the api to see if i can automatically apply names from photo metadata rather than having to spend too much of my own time manually labeling the corpus decide how much metadata i need to pull down in my data pipeline and how to store it figure out some kind of benchmark and measure it try out my idea for improving recognition accuracy benchmark again hopefully celebrate awesomeness andromeda uncategorized leave a comment february , sequence models of language: slightly irksome not much ai blogging this week because i have been buried in adulting all week, which hasn’t left much time for machine learning. sadface. however, i’m in the last week of the last deeplearning.ai course! (well. of the deeplearning.ai sequence that existed when i started, anyway. they’ve since added an nlp course and a gans course, so i’ll have to think about whether i want to take those too, but at the moment i’m leaning toward a break from the formal structure in order to give myself more time for project-based learning.) this one is on sequence models (i.e. “the data comes in as a stream, like music or language”) and machine translation (“what if we also want our output to be a stream, because we are going from a sentence to a sentence, and not from a sentence to a single output as in, say, sentiment analysis”). and i have to say, as a former language teacher, i’m slightly irked. because the way the models work is — ok, consume your input sentence one token at a time, with some sort of memory that allows you to keep track of prior tokens in processing current ones (so far, so okay). and then for your output — spit out a few most-likely candidate tokens for the first output term, and then consider your options for the second term and pick your most-likely two-token pairs, and then consider all the ways your third term could combine with those pairs and pick your most likely three-token sequences, et cetera, continue until done. and that is…not how language works? look at cicero, presuming upon your patience as he cascades through clause after clause which hang together in parallel but are not resolved until finally, at the end, a verb. the sentence’s full range of meanings doesn’t collapse until that verb at the end, which means you cannot be certain if you move one token at a time; you need to reconsider the end in light of the beginning. but, at the same time, that ending token is not equally presaged by all former tokens. it is a verb, it has a subject, and when we reached that subject, likely near the beginning of the sentence, helpfully (in latin) identified by the nominative case, we already knew something about the verb — a fact we retained all the way until the end. and on our way there, perhaps we tied off clause after clause, chunking them into neat little packages, but none of them nearly so relevant to the verb — perhaps in fact none of them really tied to the verb at all, because they’re illuminating some noun we met along the way. pronouns, pointing at nouns. adjectives, pointing at nouns. nouns, suspended with verbs like a mobile, hanging above and below, subject and object. adverbs, keeping company only with verbs and each other. there’s so much data in the sentence about which word informs which that the beam model casually discards. wasteful. and forcing the model to reinvent all these things we already knew — to allocate some of its neural space to re-engineering things we could have told it from the beginning. clearly i need to get my hands on more modern language models (a bizarre sentence since this class is all of years old, but the field moves that fast). andromeda uncategorized comment january , adapting coursera’s neural style transfer code to localhost last time, when making cats from the void, i promised that i’d discuss how i adapted the neural style transfer code from coursera’s convolutional neural networks course to run on localhost. here you go! step : first, of course, download (as python) the script. you’ll also need the nst_utils.py file, which you can access via file > open. step : while the coursera file is in .py format, it’s ipython in its heart of hearts. so i opened a new file and started copying over the bits i actually needed, reading them as i went to be sure i understood how they all fit together. along the way i also organized them into functions, to clarify where each responsibility happened and give it a name. the goal here was ultimately to get something i could run at the command line via python dpla_cats.py, so that i could find out where it blew up in step . step : time to install dependencies. i promptly made a pipenv and, in running the code and finding what importerrors showed up, discovered what i needed to have installed: scipy, pillow, imageio, tensorflow. whatever available versions of the former three worked, but for tensorflow i pinned to the version used in coursera — . . — because there are major breaking api changes with the current ( .x) versions. this turned out to be a bummer, because tensorflow promptly threw warnings that it could be much faster on my system if i compiled it with various flags my computer supports. ok, so i looked up the docs for doing that, which said i needed bazel/bazelisk — but of course i needed a paleolithic version of that for tensorflow . . compat, so it was irritating to install — and then running that failed because it needed a version of java old enough that i didn’t have it, and at that point i gave up because i have better things to do than installing quasi-eoled java versions. updating the code to be compatible with the latest tensorflow version and compiling an optimized version of that would clearly be the right answer, but also it would have been work and i wanted messed-up cat pictures now. (as for the rest of my dependencies, i ended up with scipy== . . , pillow== . . , and imageio== . . , and then whatever sub-dependencies pipenv installed. just in case the latest versions don’t work by the time you read this. 🙂 at this point i had achieved goal , aka “getting anything to run at all”. step : i realized that, honestly, almost everything in nst_utils wanted to be an imageutility, which was initialized with metadata about the content and style files (height, width, channels, paths), and carried the globals (shudder) originally in nst_utils as class data. this meant that my new dpla_cats script only had to import imageutility rather than * (from x import * is, of course, deeply unnerving), and that utility could pingpong around knowing how to do the things it knew how to do, whenever i needed to interact with image-y functions (like creating a generated image or saving outputs) rather than neural-net-ish stuff. everything in nst_utils that properly belonged in an imageutility got moved, step by step, into that class; i think one or two functions remained, and they got moved into the main script. step : ughhh, scope. the notebook plays fast and loose with scope; the raw python script is, rightly, not so forgiving. but that meant i had to think about what got defined at what level, what got passed around in an argument, what order things happened in, et cetera. i’m not happy with the result — there’s a lot of stuff that will fail with minor edits — but it works. scope errors will announce themselves pretty loudly with exceptions; it’s just nice to know you’re going to run into them. step a: you have to initialize the adam optimizer before you run sess.run(tf.global_variables_initializer()). (thanks, stackoverflow!) the error message if you don’t is maddeningly unhelpful. (failedpreconditionerror, i mean, what.) step : argparse! i spent some quality time reading this neural style implementation early on and thought, gosh, that’s argparse-heavy. then i found myself wanting to kick off a whole bunch of different script runs to do their thing overnight investigating multiple hypotheses and discovered how very much i wanted there to be command-line arguments, so i could configure all the different things i wanted to try right there and leave it alone. aw yeah. i’ve ended up with the following: parser.add_argument('--content', required=true) parser.add_argument('--style', required=true) parser.add_argument('--iterations', default= ) # was parser.add_argument('--learning_rate', default= . ) # was . parser.add_argument('--layer_weights', nargs= , default=[ . , . , . , . , . ]) parser.add_argument('--run_until_steady', default=false) parser.add_argument('--noisy_start', default=true) content is the path to the content image; style is the path to the style image; iterations and learning_rate are the usual; layer_weights is the value of style_layers in the original code, i.e. how much to weight each layer; run_until_steady is a bad api because it means to ignore the value of the iterations parameter and instead run until there is no longer significant change in cost; and noisy_start is whether to use the content image plus static as the first input or just the plain content image. i can definitely see adding more command line flags if i were going to be spending a lot of time with this code. (for instance, a layer_names parameter that adjusted what style_layers considered could be fun! or making “significant change in cost” be a user-supplied rather than hardcoded parameter!) step a: correspondingly, i configured the output filenames to record some of the metadata used to create the image (content, style, layer_weights), to make it easier to keep track of which images came from which script runs. stuff i haven’t done but it might be great: updating tensorflow, per above, and recompiling it. the slowness is acceptable — i can run quite a few trials on my macbook overnight — but it would get frustrating if i were doing a lot of this. supporting both num_iterations and run_until_steady means my iterator inside the model_nn function is kind of a mess right now. i think they’re itching to be two very thin subclasses of a superclass that knows all the things about neural net training, with the subclass just handling the iterator, but i didn’t spend a lot of time thinking about this. reshaping input files. right now it needs both input files to be the same dimensions. maybe it would be cool if it didn’t need that. trying different pretrained models! it would be easy to pass a different arg to load_vgg_model. it would subsequently be annoying to make sure that style_layers worked — the available layer names would be different, and load_vgg_model makes a lot of assumptions about how that model is shaped. as your reward for reading this post, you get another cat image! a friend commented that a thing he dislikes about neural style transfer is that it’s allergic to whitespace; it wants to paint everything with a texture. this makes sense — it sees subtle variations within that whitespace and it tries to make them conform to patterns of variation it knows. this is why i ended up with the noisy_start flag; i wondered what would happen if i didn’t add the static to the initial image, so that the original negative space stayed more negative-spacey. this, as you can probably tell, uses the harlem renaissance style image. it’s still allergic to negative space — even without the generated static there are variations in pixel color in the original — but they are much subtler, so instead of saying “maybe what i see is coiled hair?” it says “big open blue patches; we like those”. but the semantics of the original image are more in place — the kittens more kitteny, the card more readable — even though the whole image has been pushed more to colorblocks and bold lines. i find i like the results better without the static — even though the cost function is larger, and thus in a sense the algorithm is less successful. look, one more. superhero! andromeda uncategorized leave a comment january , dear internet, merry christmas; my robot made you cats from the void recently i learned how neural style transfer works. i wanted to be able to play with it more and gain some insights, so i adapted the coursera notebook code to something that works on localhost (more on that in a later post), found myself a nice historical cat image via dpla, and started mashing it up with all manner of images of varying styles culled from dpla’s list of primary source sets. (it really helped me that these display images were already curated for looking cool, and cropped to uniform size!) these sweet babies do not know what is about to happen to them. let’s get started, shall we? style image from the fake news in the s: yellow journalism primary source set. i really love how this one turned out. it’s pulled the blue and yellow colors, and the concerned face of the lower kitten was a perfect match for the expression on the right-hand muckraker. the lines of the card have taken on the precise quality of those in the cartoon — strong outlines and textured interiors. “merry christmas” the bird waves, like an eager newsboy. style image from the food and social justice exhibit. this is one of the first ones i made, and i was delighted by how it learned the square-iness of its style image. everything is more snapped to a grid. the colors are bolder, too, cueing off of that dominant yellow. the christmas banner remains almost readable and somehow heraldic. style image from the truth, justice, and the american way primary source set. how about christmas of steel? these kittens have broadly retained their shape (perhaps as the figures in the comic book foreground have organic detail?), but the background holly is more polygon-esque. the colors have been nudged toward primary, and the static of the background has taken on a swirl of dynamic motion lines. style image from the visual art during the harlem renaissance primary source set. how about starting with something boldly colored and almost abstract? why look: the kittens have learned a world of black and white and blue, with the background transformed into that stippled texture it picked up from the hair. the holly has gone more colorblocky and the lines bolder. style image from the treaty of versailles and the end of world war i primary source set. this one learned its style so aptly that i couldn’t actually tell where the boundary between the second and third images was when i was placing that equals sign. the soft pencil lines, the vertical textures of shadows and jail bars, the fact that all the colors in the world are black and white and orange (the latter mostly in the middle) — these kittens are positively melting before the force of wilsonian propaganda. imagine them in the hall of mirrors, drowning in gold and reflecting back at you dozens of times, for full nightmare effect. style image from the victorian era primary source set. shall we step back a few decades to something slightly more calming? these kittens have learned to take on soft lines and swathes of pale pink. the holly is perfectly happy to conform itself to the texture of these new england trees. the dark space behind the kittens wonders if, perhaps, it is meant to be lapels. i totally can’t remember how i found this cropped version of us food propaganda. and now for kittens from the void. brown, it has learned. the world is brown. the space behind the kittens is brown. those dark stripes were helpfully already brown. the eyes were brown. perhaps they can be the same brown, a hole dropped through kitten-space. i thought this was honestly pretty creepy, and i wondered if rerunning the process with different layer weights might help. each layer of the neural net notices different sorts of things about its image; it starts with simpler things (colors, straight lines), moves through compositions of those (textures, basic shapes), and builds its way up to entire features (faces). the style transfer algorithm looks at each of those layers and applies some of its knowledge to the generated image. so i thought, what if i change the weights? the initial algorithm weights each of five layers equally; i reran it weighted toward the middle layers and entirely ignoring the first layer, in hopes that it would learn a little less about gaping voids of brown. same thing, less void. this worked! there’s still a lot of brown, but the kitten’s eye is at least separate from its facial markings. my daughter was also delighted by how both of these images want to be letters; there are lots of letter-ish shapes strewn throughout, particularly on the horizontal line that used to be the edge of a planter, between the lower cat and the demon holly. so there you go, internet; some christmas cards from the nightmare realm. may bring fewer nightmares to us all. andromeda uncategorized comment december , december , this week in my ai after visualizing a whole bunch of theses and learning about neural style transfer and flinging myself at t-sne i feel like i should have something meaty this week but they can’t all be those weeks, i guess. still, i’m trying to hold myself to friday ai blogging, so here are some work notes: finished course of the deeplearning.ai sequence. yay! the facial recognition assignment is kind of buggy and poorly documented and i felt creepy for learning it in the first place, but i’m glad to have finished. only one more course to go! it’s a -week course, so if i’m particularly aggressive i might be able to get it all done by year’s end. tried making a d version of last week’s visualization — several people had asked — but it turned out to not really add anything. oh well. been thinking about charlie harper’s talk at swib this year, generating metadata subject labels with doc vec and dbpedia. this talk really grabbed me because he started with the exact same questions and challenges as hamlet — seriously, the first seven and a half minutes of this talk could be the first seven and a half minutes of a talk on hamlet, essentially verbatim — but took it off in a totally different direction (assigning subject labels). i have lots of ideas about where one might go with this but right now they are all sparkling voronoi diagrams in my head and that’s not a language i can readily communicate. all done with the second iteration of my ai for librarians course. there were some really good final projects this term. yay, students! andromeda uncategorized comment december , december , though these be matrices, yet there is method in them. when i first trained a neural net on , theses to make hamlet, one of the things i most wanted to do is be able to visualize them. if word vec places documents ‘near’ each other in some kind of inferred conceptual space, we should be able to see some kind of map of them, yes? even if i don’t actually know what i’m doing? turns out: yes. and it’s even better than i’d imagined. , graduate theses, arranged by their conceptual similarity. let me take you on a tour! region is biochemistry. the red dots are biology; the orange ones, chemistry. theses here include positional cloning and characterization of the mouse pudgy locus and biosynthetic engineering for the assembly of better drugs. if you look closely, you will see a handful of dots in different colors, like a buttery yellow. this color is electrical engineering & computer science, and its dots in this region include computational regulatory genomics : motifs, networks, and dynamics — that is to say, a computational biology thesis that happens to have been housed in computation rather than biology. the green south of region is physics. but you will note a bit of orange here. yes, that’s chemistry again; for example, dynamic nuclear polarization of amorphous and crystalline small molecules. if (like me), you almost majored in chemistry and realized only your senior year that the only chemistry classes that interested you were the ones that were secretly physics…this is your happy place. in fact, most of the theses here concern nuclear magnetic resonance applications. region has a striking vertical green stripe which turns out to be the nuclear engineering department. but you’ll see some orange streaks curling around it like fingers, almost suggesting three-dimensional depth. i point this out as a reminder that the original neural net embeds these , documents in a -dimensional space; i have projected that down to dimensions because i don’t know about you but i find dimensions somewhat challenging to visualize. however — just as objects may overlap in a -dimensional photo even when they are quite distant in -dimensional space — dots that are close together in this projection may be quite far apart in reality. trust the overall structure more than each individual element. the map is not the territory. that little yellow thumb by region is mathematics, now a tiny appendage off of the giant discipline it spawned — our old friend buttery yellow, aka electrical engineering & computer science. if you zoom in enough you find eecs absolutely everywhere, applied to all manner of disciplines (as above with biology), but the bulk of it — including the quintessential parts, like compilers — is right here. dramatically red region , clustered together tightly and at the far end, is architecture. this is a renowned department (it graduated i.m. pei!), but definitely a different sort of creature than most of mit, so it makes sense that it’s at one extreme of the map. that said, the other two programs in its school — urban studies & planning and media arts & sciences — are just to its north. region — tiny, yellow, and pale; you may have missed it at first glance — is linguistics island, housing theses such as topics in the stress and syntax of words. you see how there are also a handful of red dots on this island? they are brain & cognitive science theses — and in particular, ones that are secretly linguistics, like intonational phrasing in language production and comprehension. similarly — although at mit it is not the department of linguistics, but the department of linguistics & philosophy — the philosophy papers are elsewhere. (a few of the very most abstract ones are hanging out near math.) and what about region , the stingray swimming vigorously away from everything else? i spent a long time looking at this and not seeing a pattern. you can tell there’s a lot of colors (departments) there, randomly assorted; even looking at individual titles i couldn’t see anything. only when i looked at the original documents did i realize that this is the island of terrible ocr. almost everything here is an older thesis, with low-quality printing or even typewriting, often in a regrettable font, maybe with the reverse side of the page showing through. (a randomly chosen example; pdf download.) a good reminder of the importance of high-quality digitization labor. a heartbreaking example of the things we throw away when we make paper the archival format for born-digital items. and also a technical inspiration — look how much vector space we’ve had to carve out to make room for these! the poor neural net, trying desperately to find signal in the noise, needing all this space to do it. i’m tempted to throw out the entire leftmost quarter of this graph, rerun the d projection, and see what i get — would we be better able to see the structures in the high-quality data if they had room to breathe? and were i to rerun the entire neural net training process again, i’d want to include some sort of threshhold score for ocr quality. it would be a shame to throw things away — especially since they will be a nonrandom sample, mostly older theses — but i have already had to throw away things i could not ocr at all in an earlier pass, and, again, i suspect the neural net would do a better job organizing the high-quality documents if it could use the whole vector space to spread them out, rather than needing some of it to encode the information “this is terrible ocr and must be kept away from its fellows”. clearly i need to share the technical details of how i did this, but this post is already too long, so maybe next week. tl;dr i reached out to matt miller after reading his cool post on vectorizing the dpla and he tipped me off to umap and here we are — thanks, matt! and just as clearly you want to play with this too, right? well, it’s super not ready to be integrated into hamlet due to any number of usability issues but if you promise to forgive me those — have fun. you see how when you hover over a dot you get a label with the format . -x.txt? it corresponds to a url of the format https://hamlet.andromedayelton.com/similar_to/x. go play :). andromeda uncategorized comment december , december , of such stuff are (deep)dreams made: convolutional networks and neural style transfer skipped fridai blogging last week because of thanksgiving, but let’s get back on it! top-of-mind today are the firing of ai queen timnit gebru (letter of support here) and a couple of grant applications that i’m actually eligible for (this is rare for me! i typically need things for which i can apply in my individual capacity, so it’s always heartening when they exist — wish me luck). but for blogging today, i’m gonna talk about neural style transfer, because it’s cool as hell. i started my ml-learning journey on coursera’s intro ml class and have been continuing with their deeplearning.ai sequence; i’m on course of there, so i’ve just gotten to neural style transfer. this is the thing where a neural net outputs the content of one picture in the style of another: via https://medium.com/@build_it_for_fun/neural-style-transfer-with-swift-for-tensorflow-b b . ok, so! let me explain while it’s still fresh. if you have a neural net trained on images, it turns out that each layer is responsible for recognizing different, and progressively more complicated, things. the specifics vary by neural net and data set, but you might find that the first layer gets excited about straight lines and colors; the second about curves and simple textures (like stripes) that can be readily composed from straight lines; the third about complex textures and simple objects (e.g. wheels, which are honestly just fancy circles); and so on, until the final layers recognize complex whole objects. you can interrogate this by feeding different images into the neural net and seeing which ones trigger the highest activation in different neurons. below, each × grid represents the most exciting images for a particular neuron. you can see that in this network, there are layer neurons excited about colors (green, orange), and about lines of particular angles that form boundaries between dark and colored space. in layer , these get built together like tiny image legos; now we have neurons excited about simple textures such as vertical stripes, concentric circles, and right angles. via https://adeshpande .github.io/the- -deep-learning-papers-you-need-to-know-about.html, originally from zeller & fergus, visualizing and understanding convolutional networks so how do we get from here to neural style transfer? we need to extract information about the content of one image, and the style of another, in order to make a third image that approximates both of them. as you already expect if you have done a little machine learning, that means that we need to write cost functions that mean “how close is this image to the desired content?” and “how close is this image to the desired style?” and then there’s a wrinkle that i haven’t fully understood, which is that we don’t actually evaluate these cost functions (necessarily) against the outputs of the neural net; we actually compare the activations of the neurons, as they react to different images — and not necessarily from the final layer! in fact, choice of layer is a hyperparameter we can vary (i super look forward to playing with this on the coursera assignment and thereby getting some intuition). so how do we write those cost functions? the content one is straightforward: if two images have the same content, they should yield the same activations. the greater the differences, the greater the cost (specifically via a squared error function that, again, you may have guessed if you’ve done some machine learning). the style one is beautifully sneaky; it’s a measure of the difference in correlation between activations across channels. what does that mean in english? well, let’s look at the van gogh painting, above. if an edge detector is firing (a boundary between colors), then a swirliness detector is probably also firing, because all the lines are curves — that’s characteristic of van gogh’s style in this painting. on the other hand, if a yellowness detector is firing, a blueness detector may or may not be (sometimes we have tight parallel yellow and blue lines, but sometimes yellow is in the middle of a large yellow region). style transfer posits that artistic style lies in the correlations between different features. see? sneaky. and elegant. finally, for the style-transferred output, you need to generate an image that does as well as possible on both cost functions simultaneously — getting as close to the content as it can without unduly sacrificing the style, and vice versa. as a side note, i think i now understand why deepdream is fixated on a really rather alarming number of eyes. since the layer choice is a hyperparameter, i hypothesize that choosing too deep a layer — one that’s started to find complex features rather than mere textures and shapes — will communicate to the system, yes, what i truly want is for you to paint this image as if those complex features are matters of genuine stylistic significance. and, of course, eyes are simple enough shapes to be recognized relatively early (not very different from concentric circles), yet ubiquitous in image data sets. so…this is what you wanted, right? the eager robot helpfully offers. https://www.ucreative.com/inspiration/google-deep-dream-is-the-trippiest-thing-in-the-internet/ i’m going to have fun figuring out what the right layer hyperparameter is for the coursera assignment, but i’m going to have so much more fun figuring out the wrong ones. andromeda uncategorized comments december , december , let’s visualize some hamlet data! or, d and t-sne for the lols. in , i trained a neural net on ~ k graduate theses using the doc vec algorithm, in hopes that doing so would provide a backend that could support novel and delightful discovery mechanisms for unique library content. the result, hamlet, worked better than i hoped; it not only pulls together related works from different departments (thus enabling discovery that can’t be supported with existing metadata), but it does a spirited job on documents whose topics are poorly represented in my initial data set (e.g. when given a fiction sample it finds theses from programs like media studies, even though there are few humanities theses in the data set). that said, there are a bunch of exploratory tools i’ve had in my head ever since that i’ve not gotten around to implementing. but here, in the spirit of tossing out things that don’t bring me joy (like ) and keeping those that do, i’m gonna make some data viz! there are only two challenges with this: by default doc vec embeds content in a -dimensional space, which is kind of hard to visualize. i need to project that down to or dimensions. i don’t actually know anything about dimensionality reduction techniques, other than that they exist. i also don’t know know javascript much beyond a copy-paste level. i definitely don’t know d , or indeed the pros and cons of various visualization libraries. also art. or, like, all that stuff in tufte’s book, which i bounced off of. (but aside from that, mr. lincoln, how was the play?) i decided i should start with the pages that display the theses most similar to a given thesis (shout-out to jeremy brown, startup founder par excellence) rather than with my ideas for visualizing the whole collection, because i’ll only need to plot ten or so points instead of k. this will make it easier for me to tell visually if i’m on the right track and should let me skip dealing with performance issues for now. on the down side, it means i may need to throw out any code i write at this stage when i’m working on the next one. 🤷‍♀️ and i now have a visualization on localhost! which you can’t see because i don’t trust it yet. but here are the problems i’ve solved thus far: it’s hard to copy-paste d examples on the internet. d ’s been around for long enough there’s substantial content about different versions, so you have to double-check. but also most of the examples are live code notebooks on observable, which is a wicked cool service but not the same environment as a web page! if you just copy-paste from there you will have things that don’t work due to invisible environment differences and then you will be sad. 😢 i got tipped off to this by mollie marie pettit’s great your first d scatterplot notebook, which both names the phenomenon and provides two versions of the code (the live-editable version and the one you can actually copy/paste into your editor). if you start googling for dimensionality reduction techniques you will mostly find people saying “use t-sne”, but t-sne is a lying liar who lies. mind you, it’s what i’m using right now because it’s so well-documented it was the easiest thing to set up. (this is why i said above that i don’t trust my viz.) but it produces different results for the same data on different pageloads (obviously different, so no one looking at the page will trust it either), and it’s not doing a good job preserving the distances i care about. (i accept that anything projecting from d down to d will need to distort distances, but i want to adequately preserve meaning — i want the visualization to not just look pretty but to give people an intellectually honest insight into the data — and i’m not there yet.) conveniently this is not my first time at the software engineering rodeo, so i encapsulated my dimensionality reduction strategy inside a function, and i can swap it out for whatever i like without needing to rewrite the d as long as i return the same data structure. so that’s my next goal — try out umap (hat tip to matt miller for suggesting that to me), try out pca, fiddle some parameters, try feeding it just the data i want to visualize vs larger neighborhoods, see if i’m happier with what i get. umap in particular alleges itself to be fast with large data sets, so if i can get it working here i should be able to leverage that knowledge for my ideas for visualizing the whole thing. onward, upward, et cetera. 🎉 andromeda uncategorized comments november , ai in the library, round one the san josé state university school of information wanted to have a half-course on artificial intelligence in their portfolio, and asked me to develop and teach it. (thanks!) so i got a blank canvas on which to paint eight weeks of…whatever you might want graduate students in library & information science students to know about ai. for those of you who just want the reading list, here you go. for those of you who thought about the second-to-last sentence: ahahaha. this is fine. this is of course the problem of all teachers — too much material, too little time — and in an ischool it’s further complicated because, while many students have technological interests and expertise, few have programming skills and even fewer have mathematical backgrounds, so this course can’t be “intro to programming neural nets”. i can gesture in the direction of linear algebra and high-dimensional spaces, but i have to translate it all into human english first. but further, even if i were to do that, it wouldn’t be the right course! as future librarians, very few of my students will be programming neural nets. they are much more likely to be helping students find sources for papers, or helping researchers find or manage data sets, or supporting professors who are developing classes, helping patrons make sense of issues in the news, and evaluating vendor pitches about ai products. which means i don’t need people who can write neural net code; i need people who understand the basics of how machine learning operates, who can do some critical analysis, situate it in its social context. people who know some things about what data is good for, how it’s hard, where to find it. people who know at least the general direction in which they might find news articles and papers and conferences that their patrons will care about. people who won’t be too dazzled by product hype and can ask pointed questions about how products really work, and whether they respect library values. and, while we’re at it, people who have some sense of what ai can do, not just theoretically, but concretely in real-world library settings. eight weeks: go! what i ended up doing was -week modules, with a rough alternation of theory and library case studies, and a pretty wild mix of readings: conference presentations, scholarly papers from a variety of disciplines, hilarious computational misadventures, news articles, data visualizations. i mostly kept a lid on the really technical stuff in the required readings, but tossed a lot of it into optional readings, so that students with that background or interest could pull on those threads. (and heavily annotated the optional readings, to give people a sense of what might interest them; i’d like to say this is why surprisingly many of my students did some optional reading, but actually they’re just awesome.) for case studies, we looked at the northern illinois university dime novels collection experiments; metadata enrichment in the charles teenie harris archive; my own work with hamlet; and the university of rhode island ai lab. this let us hit a gratifyingly wide variety of machine learning techniques, use cases (metadata, discovery, public services), and settings (libraries, archives). do i have a couple of pages of things to change up next time i teach the class (this fall)? of course i do. but i think it went well for a first-time class (particularly for a first-time class in the middle of a global catastrophe…) big ups to the following: matthew short of niu and bohyun kim of uri, for guest speaking; everyone at sjsu who worked on their “how to teach online” materials, especially debbie faires — their onboarding did a good job of conveying sjsu-specific expectations and building a toolkit for teaching specifically online in a way that was useful to me as someone with a lot of offline teaching experience; zeynep tufecki, momin malik, catherine d’ignazio, who suggested readings that i ended up assigning; and my students, who are about to get a paragraph. my students. look. you signed up to take a class online — it’s an all-online program — but none of you signed up to do it while being furloughed, while homeschooling, while being sick with a scary new virus. and you knocked it out of the park. week after week, asking for the smallest of extensions to hold it all together, breaking my heart in private messages, while publicly writing thoughtful, well-researched, footnoted discussion posts. while not only doing even the optional readings, but finding astonishment and joy in them. while piecing together the big ideas about data and bias and fairness and the genuine alienness of machine intelligence. i know for certain, not as an article of faith but as a statement of fact, that i will keep seeing your names out there, that your careers will go places, and i hope i am lucky enough to meet you in person someday. andromeda uncategorized comments may , may , adventures with parsing django uploaded csv files in python let’s say you’re having problems parsing a csv file, represented as an inmemoryuploadedfile, that you’ve just uploaded through a django form. there are a bunch of answers on stackoverflow! they all totally work with python ! …and lead to hours of frustration if, say, hypothetically, like me, you’re using python . if you are getting errors like _csv.error: iterator should return strings, not bytes (did you open the file in text mode?) — and then getting different errors about dictreader not getting an expected iterator after you use .decode('utf- ') to coerce your file to str — this is the post for you. it turns out all you need to do (e.g. in your form_valid) is: csv_file.seek( ) csv.dictreader(io.stringio(csv_file.read().decode('utf- '))) what’s going on here? the seek statement ensures the pointer is at the beginning of the file. this may or may not be required in your case. in my case, i’d already read the file in my forms.py in order to validate it, so my file pointer was at the end. you’ll be able to tell that you need to seek() if your csv.dictreader() doesn’t throw any errors, but when you try to loop over the lines of the file you don’t even enter the for loop (e.g. print() statements you put in it never print) — there’s nothing left to loop over if you’re at the end of the file. read() gives you the file contents as a bytes object, on which you can call decode(). decode('utf- ') turns your bytes into a string, with known encoding. (make sure that you know how your csv is encoded to start with, though! that’s why i was doing validation on it myself. unicode, dammit is going to be my friend here. even if i didn’t want an excuse to use it because of its title alone. which i do.) io.stringio() gives you the iterator that dictreader needs, while ensuring that your content remains stringy. tl;dr i wrote two lines of code (but eight lines of comments) for a problem that took me hours to solve. hopefully now you can copy these lines, and spend only a few minutes solving this problem! andromeda uncategorized comments april , posts navigation older posts blog at wordpress.com. add your thoughts here... (optional) post to cancel privacy & cookies: this site uses cookies. by continuing to use this website, you agree to their use. to find out more, including how to control cookies, see here: cookie policy gfdrr labs | gfdrr skip to main content header main navigation who we are how we work where we work knowledge hub what's new dropdown main navigation who we are + funding structure & partnerships meet our team how we work + gfdrr labs challenge fund understanding risk open data for resilience initiative disruptive technologies resilient infrastructure building regulation for resilience global program for safer schools nature-based solutions resilient cities city resilience program hydromet services africa hydromet program climate risk and early warning systems (crews) crews caribbean financial protection sfrarr central asia caribbean regional resilience building facility drf analytics drfi in caribbean octs global risk financing facility social resilience drm-fcv nexus gender resilience to climate change small island states resilience initiative canada-caribbean resilience facility resilient recovery where we work + africa europe and central asia east asia and pacific latin america and caribbean middle east and north africa south asia search grants knowledge hub + publications online tools e-learning what's new + feature stories events blogs news videos search gfdrr labs unlocking access to risk information gfdrr labs resilient infrastructure resilient cities hydromet services and early warning systems financial protection social resilience resilience to climate change resilient recovery gfdrr labs uses cutting-edge science and technology to make high-quality risk information available faster and at lower costs, and develop new tools that allow decision-makers and communities to collect, share, and understand risk information. we tailor our approach to support the full range of disaster risk management interventions — from preparedness to risk reduction to consideration of financial solutions. read more view all grants highlights from gfdrr labs community understanding risk (ur) bringing together practitioners in disaster risk identification, assessment & communication. learn more program disruptive technologies for disaster resilience exploring the use of disruptive technologies for disaster risk management and resilience. learn more online tool opendri mobilizing the global open data movement to tackle disaster and climate risk. learn more pillars pillars of our gfdrr labs work open source tool development technical advisory service research and knowledge sharing partnerships for innovation open source tool development gfdrr creates tools to promote wide access to vital data and information. the open data for resilience initiative (opendri) encourages governments to make official data available to the public — and supports local communities to play a greater role in risk management through projects like community mapping — all through the use of open source tools. thinkhazard! is a new web-based tool that enables non-specialists to consider the impact of disasters on new development projects. code for resilience works with country partners to co-invest in developing capacity, community, and tools that leverage technological innovations to strengthen resilience to natural disasters. publication open data for resilience initiative: field guide learn more online tool code for resilience learn more online tool thinkhazard! learn more emma phillips head of gfdrr labs ephillips@worldbank.org technical advisory service gfdrr helps to ensure decision-makers, from the local to the international level, are taking advantage of new approaches and technology in order to deliver risk information to more people faster, cheaper, and with greater functionality. collaborating across areas of engagement, such as the financial protection and hydromet teams, gfdrr works to advance the use of risk information. national-level risk profiles have been developed and rolled out in africa, central asia, and europe. in sub-saharan africa, the first phase focused on five countries: ethiopia, kenya, niger, senegal, and uganda. these risk assessments provide valuable information on the risks from earthquake, flood, drought, volcano, and landslides to aid decision-makers in better understanding their risk to support sustainable development. publication disaster risk country profiles learn more emma phillips head of gfdrr labs ephillips@worldbank.org research and knowledge sharing the pace of innovation moves rapidly, and fresh knowledge needs to be shared. gfdrr prioritizes making new methods of disaster risk assessment available and increasing understanding for other stakeholders – including best practice publications and analysis. the "making of a riskier future" publication, released in may , makes the case for all risk assessments to consider how climate and socio-economic changes will shape and grow risk in the future. this research informs future interventions. emma phillips head of gfdrr labs ephillips@worldbank.org partnerships for innovation gfdrr works with a wide variety of partners at both the local and multinational level to bring the most innovative and custom-made solutions to countries around the world, each with different needs and priorities. key partners include producers and users of risk data, such as national governments and local communities. through these ecosystems, gfdrr shares knowledge, fosters connection, and encourages collaboration. it also works closely with other development partners, including un agencies, ngos, technology companies, insurers, and academic institutions. the understanding risk community is an important focal point for partnerships. the challenge fund, a joint project with the uk government, awards grants for positively disruptive ideas that can overcome entrenched barriers to understanding risk. through the fund, new methods such as open-source mobile weather stations, drones, crowdsourcing, social media, and satellite imagery are being applied. program challenge fund learn more collection understanding risk knowledge products and proceedings learn more online exhibit the art of resilience learn more emma phillips head of gfdrr labs ephillips@worldbank.org tools & resources view all tools online tool unbreakable: resilience indicator toolbox learn more > dataset global landslide hazard map learn more > online tool thinkhazard! learn more > online tool return period calculator for likelihood of hazard events learn more > online course e-learning: understanding risk learn more > online tool geonode learn more > explore more of our work. view all resources event open data day learn more news supporting data for resilience at open data day learn more blog everybody has a role: how stakeholders are contributing to increase urban resilience in cameroon learn more guidance note the open cities africa initiative develops critical disaster risk data a guidance note was published to document the best uses of drones in collecting geospatial data... learn more blog how a hackathon sparked resilience and innovation throughout one of the largest western african cities learn more results in resilience results in resilience: disruptive technologies for disaster risk management in africa learn more blog making cities resilient helps youth thrive professionally: a tale of resilience from congo learn more blog african communities are closing the digital map gap for cities learn more event understanding risk (ur) global forum bringing together the , -strong ur community of disaster risk experts & practitioners. learn more global facility for disaster reduction and recovery (gfdrr) world bank headquarters h street nw washington dc (u.s.) footer main navigation who we are funding structure & partnerships meet our team how we work gfdrr labs resilient infrastructure resilient cities hydromet services financial protection social resilience resilience to climate change resilient recovery where we work africa europe and central asia east asia and pacific latin america and caribbean middle east and north africa south asia search grants knowledge hub publications online tools e-learning what's new feature stories events blogs news videos search follow us © global facility for disaster reduction and recovery terms of use privacy policy contact us new supply chain attack uses poisoned updates to infect gamers’ computers | ars technica skip to main content biz & it tech science policy cars gaming & culture store forums subscribe close navigate store subscribe videos features reviews rss feeds mobile site about ars staff directory contact us advertise with ars reprints filter by topic biz & it tech science policy cars gaming & culture store forums settings front page layout grid list site theme black on white white on black sign in comment activity sign up or login to join the discussions! stay logged in | having trouble? sign up to comment and more sign up beware of malicious updates — new supply chain attack uses poisoned updates to infect gamers’ computers if you've used noxplayer in the past months, it's time to check for malware. dan goodin - feb , : pm utc enlarge / circuit board with speed motion and light. getty images reader comments with posters participating share this story share on facebook share on twitter share on reddit update / / : eset has updated its post to report that bignox's initial denial of the compromise was a misunderstanding on its part and that it has since taken these steps to improve security for their users. eset said it assumes no responsibility for the accuracy of the information provided by bignox. researchers have uncovered a software supply chain attack that is being used to install surveillance malware on the computers of online gamers. the unknown attackers are targeting select users of noxplayer, a software package that emulates the android operating system on pcs and macs. people use it primarily for playing mobile android games on these platforms. noxplayer-maker bignox says the software has million users in countries. poisoning the well security firm eset said on monday that the bignox software distribution system was hacked and used to deliver malicious updates to select users. the initial updates were delivered last september through the manipulation of two files: the main bignox binary nox.exe and noxpack.exe, which downloads the update itself. “we have sufficient evidence to state that the bignox infrastructure (res .bignox.com) was compromised to host malware, and also to suggest that their http api infrastructure (api.bignox.com) could have been compromised,” eset malware researcher ignacio sanmillan wrote. “in some cases, additional payloads were downloaded by the bignox updater from attacker-controlled servers. this suggests that the url field, provided in the reply from the bignox api, was tampered with by the attackers.” in a nutshell, the attack works this way: on launch, nox.exe sends a request to a programming interface to query update information. the bignox api server responds with update information that includes a url where the legitimate update is supposed to be available. eset speculates that the legitimate update may have been replaced with malware or, alternatively, a new filename or url was introduced. malware is then installed on the target’s machine. the malicious files aren’t digitally signed the way legitimate updates are. that suggests the bignox software build system isn’t compromised; only the systems for delivering updates are. the malware performs limited reconnaissance on the targeted computer. the attackers further tailor the malicious updates to specific targets of interest. advertisement the bignox api server responds to a specific target with update information that points to the location of the malicious update on an attacker-controlled server. the intrusion flow observed is depicted below. enlarge eset eset malware researcher sanmillan added: legitimate bignox infrastructure was delivering malware for specific updates. we observed that these malicious updates were only taking place in september . furthermore, we observed that for specific victims, malicious updates were downloaded from attacker-controlled infrastructure subsequently and throughout the end of and early . we are highly confident that these additional updates were performed by nox.exe supplying specific parameters to noxpack.exe, suggesting that the bignox api mechanism may have also been compromised to deliver tailored malicious updates. it could also suggest the possibility that victims were subjected to a mitm attack, although we believe this hypothesis is unlikely since the victims we discovered are in different countries, and attackers already had a foothold on the bignox infrastructure. furthermore, we were able to reproduce the download of the malware samples hosted on res .bignox.com from a test machine and using https. this discards the possibility that a mitm attack was used to tamper the update binary. eset has observed three different malware variants being installed. there’s no sign of any of the malware trying to make financial gains on behalf of the attackers. that led the security company to believe the malware is being used to surveil targets. sanmillan said that of more than , eset users who have noxplayer installed, only five of them received a malicious update. the numbers underscore just how targeted the attacks are. targets are located in taiwan, hong kong, and sri lanka. sanmillan said that eset contacted bignox with the findings and the software maker denied being affected. bignox representatives didn’t respond to email seeking comment for this post. anyone who has used noxplayer over the past five months should take time to carefully inspect their systems for signs of compromise. monday’s post provides a list of files and settings that will indicate when a computer has received a malicious update. while the eset post refers only to the windows version of the software, there’s currently no way to rule out the possibility that macos users were targeted, too. reader comments with posters participating share this story share on facebook share on twitter share on reddit dan goodin dan is the security editor at ars technica, which he joined in after working for the register, the associated press, bloomberg news, and other publications. email dan.goodin@arstechnica.com // twitter @dangoodin advertisement you must login or create an account to comment. channel ars technica ← previous story next story → related stories sponsored stories powered by today on ars store subscribe about us rss feeds view mobile site contact us staff advertise with us reprints newsletter signup join the ars orbital transmission mailing list to get weekly updates delivered to your inbox. sign me up → cnmn collection wired media group © condé nast. all rights reserved. use of and/or registration on any portion of this site constitutes acceptance of our user agreement (updated / / ) and privacy policy and cookie statement (updated / / ) and ars technica addendum (effective / / ). ars may earn compensation on sales from links on this site. read our affiliate link policy. your california privacy rights | do not sell my personal information the material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of condé nast. ad choices amazon elasticsearch service pricing - amazon web services click here to return to amazon web services homepage contact sales support english my account create an aws account products solutions pricing documentation learn partner network aws marketplace customer enablement events explore more عربي bahasa indonesia deutsch english español français italiano português tiếng việt türkçe Ρусский ไทย 日本語 한국어 中文 (简体) 中文 (繁體) aws management console account settings billing & cost management security credentials aws personal health dashboard support center knowledge center aws support overview click here to return to amazon web services homepage products solutions pricing introduction to aws getting started documentation training and certification developer center customer success partner network aws marketplace support log into console download the mobile app amazon elasticsearch service overview features pricing getting started resources faqs the elk stack resources customers articles the elk stack what is elasticsearch? what is kibana? what is logstash? data ingestion products analytics amazon elasticsearch service amazon elasticsearch service pricing amazon elasticsearch service pricing with amazon elasticsearch service, you pay only for what you use. there is no minimum fee or usage requirement. you are charged only for amazon elasticsearch service instance hours, amazon ebs storage (if you choose this option), and data transfer. free tier you can get started for free on amazon elasticsearch service with the aws free tier. for customers in the aws free tier, amazon elasticsearch service provides free usage of up to hours per month of a t .small.elasticsearch or t .small.elasticsearch instance and gb per month of optional ebs storage (magnetic or general purpose). if you exceed the free tier usage limits, you will be charged the amazon elasticsearch service rates for the additional resources you use. see offer terms for more details. on-demand instance pricing except as otherwise noted, our prices are exclusive of applicable taxes and duties, including vat and applicable sales tax. for customers with a japanese billing address, use of aws is subject to japanese consumption tax. learn more. reserved instance pricing with amazon elasticsearch service reserved instances, you can reserve instances for one- or three-year term and get significant savings on usage costs compared to on-demand instances. functionally, on-demand and reserved instances are identical. from a billing perspective, however, reserved instances can provide significant cost savings. reserved instances have three payment options: no upfront reserved instances (nuri) – nuris offer significant savings compared to on-demand instance pricing. you pay nothing upfront, but you commit to pay for the reserved instances over the course of a one- or three-year term. one-year nuris offer a % discount and three-year nuris offer a % discount. for t .medium, one-year nuris offer a % discount and three-year nuris offer a % discount partial upfront reserved instances (puri) – puris offer higher savings than nuris. this option requires you to pay a portion of the total cost upfront and pay the remainder of the cost on an hourly basis over the course of the term. one-year puris offer a % discount and three-year puris offer a % discount. for t .medium, one-year puris offer a % discount and three-year nuris offer a % discount all upfront reserved instances (auri) – auris offer the highest savings of all of the reserved instance payment options. you pay for the entire reservation with one upfront payment and pay nothing on an hourly basis. one-year auris offer a % discount and three-year auris offer a % discount. for t .medium, one-year auris offer a % discount and three-year nuris offer a % discount reserved instance pricing is specific to each region and depends on the payment option and term that you select. when you purchase a reserved instance, you will be charged the associated upfront (if applicable) and hourly fees (if applicable), even if you are not currently running amazon elasticsearch service. to purchase reserved instances, visit the reserved instance tab in our console. one-year term three-year term * effective hourly pricing is shown to help you calculate the amount of money that a reserved instance will save you over on-demand pricing. when you purchase a reserved instance, you are billed for every hour during the entire reserved instance term that you select, regardless of whether the instance is running or not. the effective hourly price shows the amortized hourly cost of the instance (this takes the total cost of the reserved instance over the entire term, including any upfront payment, and spreads it out over each hour of the reserved instance term). amazon ebs volume pricing (applies if you choose ebs volumes) amazon elasticsearch service allows you to choose the type of amazon ebs volume. if you choose provisioned iops (ssd) storage, you will be charged for the storage as well as the throughput you provision. however, you will not be charged for the i/os you consume. ultrawarm pricing except as otherwise noted, our prices are exclusive of applicable taxes and duties, including vat and applicable sales tax. for customers with a japanese billing address, use of aws is subject to japanese consumption tax. learn more. standard aws data transfer charges you need to pay standard aws data transfer charges for the data transferred in and out of amazon elasticsearch service. you will not be charged for the data transfer between nodes within your amazon elasticsearch service domain. free automated snapshot storage amazon elasticsearch service allows you to add data durability through automated and manual snapshots of your cluster. the service provides storage space for automated snapshots free of charge for each amazon elasticsearch domain and retains these snapshots for a period of days. manual snapshots are stored in amazon s and incur standard amazon s usage charges. data transfer for using the snapshots is free of charge. additional pricing resources aws pricing calculator easily calculate your monthly costs with aws economics resource center additional resources for switching to aws learn how to get started with amazon elasticsearch service visit the getting started page monitor your ri utilization? monitor your reserved instance utilization and coverage using aws budgets have more questions? contact us sign in to the console learn about aws what is aws? what is cloud computing? what is devops? what is a container? what is a data lake? aws cloud security what's new blogs press releases resources for aws getting started training and certification aws solutions portfolio architecture center product and technical faqs analyst reports aws partner network developers on aws developer center sdks & tools .net on aws python on aws java on aws php on aws javascript on aws help contact us aws careers file a support ticket knowledge center aws support overview legal create an aws account amazon is an equal opportunity employer: minority / women / disability / veteran / gender identity / sexual orientation / age. language عربي bahasa indonesia deutsch english español français italiano português tiếng việt türkçe Ρусский ไทย 日本語 한국어 中文 (简体) 中文 (繁體) privacy | site terms | cookie preferences | © , amazon web services, inc. or its affiliates. all rights reserved. dshr's blog dshr's blog i'm david rosenthal, and this is a place to discuss the work i'm doing in digital preservation. friday, february , talk at berkeley's information access seminar once again cliff lynch invited me to give a talk to the information access seminar at uc berkeley's ischool. preparation time was limited because these days i'm a full-time grandparent so the talk, entitled securing the digital supply chain summarizes and updates two long posts from two years ago: certificate transparency securing the software supply chain the abstract was: the internet is suffering an epidemic of supply chain attacks, in which a trusted supplier of content is compromised and delivers malware to some or all of their clients. the recent solarwinds compromise is just one glaring example. this talk reviews efforts to defend digital supply chains. below the fold, the text of the talk with links to the sources. read more » posted by david. at : pm no comments: labels: security thursday, february , chromebook linux update my three acer c chromebooks running linux are still giving yeoman service, although for obvious reasons i'm not travelling these days. but it is time for an update to 's travels with a chromebook. below the fold, an account of some adventures in sysadmin. read more » posted by david. at : am no comments: labels: linux thursday, january , effort balancing and rate limits catalin cimpanu reports on yet another crime wave using bitcoin in as bitcoin price surges, ddos extortion gangs return in force: in a security alert sent to its customers and shared with zdnet this week, radware said that during the last week of and the first week of , its customers received a new wave of ddos extortion emails. extortionists threatened companies with crippling ddos attacks unless they got paid between and bitcoins ($ , to $ , ) ... the security firm believes that the rise in the bitcoin-to-usd price has led to some groups returning to or re-prioritizing ddos extortion schemes. and dan goodin reports on the latest technique the ddos-ers are using in ddosers are abusing microsoft rdp to make attacks more powerful: as is typical with many authenticated systems, rdp responds to login requests with a much longer sequence of bits that establish a connection between the two parties. so-called booter/stresser services, which for a fee will bombard internet addresses with enough data to take them offline, have recently embraced rdp as a means to amplify their attacks, security firm netscout said. the amplification allows attackers with only modest resources to strengthen the size of the data they direct at targets. the technique works by bouncing a relatively small amount of data at the amplifying service, which in turn reflects a much larger amount of data at the final target. with an amplification factor of . to , gigabytes-per-second of requests directed at an rdp server will deliver roughly gbps to the target. i don't know why it took me so long to figure it out, but reading goodin's post i suddenly realized that techniques we described in impeding attrition attacks in p p systems, a follow-up to our award-winning sosp paper on the architecture of the lockss system, can be applied to preventing systems from being abused by ddos-ers. below the fold, brief details. read more » posted by david. at : am no comments: labels: bitcoin, networking, security tuesday, january , isp monopolies for at least the last three years (it isn't about the technology) i've been blogging about the malign effects of the way the faangs dominate the web and the need for anti-trust action to mitigate them. finally, with the recent lawsuits against facebook and google, some action may be in prospect. i'm planning a post on this topic. but when it comes to malign effects of monopoly i've been ignoring the other monopolists of the internet, the telcos. an insightful recent post by john gilmore to dave farber's ip list sparked a response from thomas leavitt and some interesting follow-up e-mail. gilmore was involved in pioneering consumer isps, and leavitt in pioneering web hosting. both attribute the current sorry state of internet connectivity in the us to the lack of effective competition. they and i differ somewhat on how the problem could be fixed. below the fold i go into the details. read more » posted by david. at : am no comments: labels: platform monopolies thursday, january , the bitcoin "price" jemima kelly writes no, bitcoin is not “the ninth-most-valuable asset in the world” and its a must-read. below the fold, some commentary. read more » posted by david. at : am comments: labels: bitcoin thursday, january , two million page views! woohoo! this blog just passed two million all-time page views since april st . posted by david. at : pm no comments: labels: personal tuesday, january , the new oldweb.today two days before christmas ilya kreymer posted announcing the new oldweb.today. the old oldweb.today was released five years ago, and ilya described the details in a guest post here. it was an important step forward in replaying preserved web content because users could view the old web content as it would have been rendered at the time it was published, not as rendered in a modern browser. i showed an example of the difference this made in the internet is for cats. below the fold, i look at why the new oldweb.today is an improvement on the old version, which is still available at classic.oldweb.today read more » posted by david. at : am no comments: labels: digital preservation, emulation, web archiving older posts home subscribe to: posts (atom) blog rules posts and comments are copyright of their respective authors who, by posting or commenting, license their work under a creative commons attribution-share alike . united states license. off-topic or unsuitable comments will be deleted. dshr dshr in anwr recent comments full comments blog archive ▼ ( ) ▼ february ( ) talk at berkeley's information access seminar chromebook linux update ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► july ( ) ► june ( ) ► february ( ) ► ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) lockss system has permission to collect, preserve, and serve this archival unit. simple theme. powered by blogger. none dshr's blog: economic failures of https dshr's blog i'm david rosenthal, and this is a place to discuss the work i'm doing in digital preservation. thursday, december , economic failures of https bruce schneier points me to assessing legal and technical solutions to secure https, a fascinating, must-read analysis of the (lack of) security on the web from an economic rather than a technical perspective by axel arnbak and co-authors from amsterdam and delft universities. do read the whole paper, but below the fold i provide some choice snippets. arnbak et al point out that users are forced to trust all certificate authorities (cas): a crucial technical property of the https authentication model is that any ca can sign certificates for any domain name. in other words, literally anyone can request a certificate for a google domain at any ca anywhere in the world, even when google itself has contracted one particular ca to sign its certificate. many cas are untrustworthy on their face: what’s particularly troubling is that a number of the trusted cas are run by authoritarian governments, among other less trustworthy institutions. their cas can issue a certificate for any web site in the world, which will be accepted as trustworthy by browsers of all internet users. the security practices of even leading cas have proven to be inadequate: three of the four market leaders got hacked in recent years and that some of the “security” features of these services do not really provide actual security. customers can't actually buy security, only the appearance of security: information asymmetry prevents buyers from knowing what cas are really doing. buyers are paying for the perception of security, a liability shield, and trust signals to third parties. none of these correlates verifiably with actual security. given that ca security is largely unobservable, buyers’ demands for security do not necessarily translate into strong security incentives for cas. there's little incentive for cas to invest in better security: negative externalities of the weakest-link security of the system exacerbate these incentive problems. the failure of a single ca impacts the whole ecosystem, not just that ca’s customers. all other things being equal, these interdependencies undermine the incentives of cas to invest, as the security of their customers depends on the efforts of all other cas. they conclude: regardless of major cybersecurity incidents such as ca breaches, and even the snowden revelations, a sense of urgency to secure https seems nonexistent. as it stands, major cas continue business as usual. for the foreseeable future, a fundamentally flawed authentication model underlies an absolutely critical technology used every second of every day by every internet user. on both sides of the atlantic, one wonders what cybersecurity governance really is about. posted by david. at : am labels: security comments: david. said... ars technica has two stories showing how microsoft's failure to take even minimal precautions allowed outsiders to obtain https certificates for microsoft live domains. in one documented case it took microsoft years to respond. security, its just not a priority. march , at : am david. said... here we go again. the china internet network information center (cnnic), a certificate authority trusted by essentially all browsers, delegated authority to mcs, an intermediate authority based in egypt, who went ahead and issued certificates for, among other domains, google properties. browser vendors are rushing to revoke the certificates, except that so far there appears to be no comment to this effect from microsoft. this completely broken system is aptly described by bruce schneier's term "security theater". march , at : pm david. said... google has decided to remove cnnic as a root ca pending improvements to their process. april , at : am david. said... over this weekend, the certificate chain for google's gmail service broke because google forgot to renew one of the intermediate certificates. the system is too complex and fragile for even the experts to maintain properly. april , at : am david. said... i should have remembered and linked to this relevant post from discussing the possible application of lockss-style sampled voting to certificate verification. april , at : pm post a comment newer post older post home subscribe to: post comments (atom) blog rules posts and comments are copyright of their respective authors who, by posting or commenting, license their work under a creative commons attribution-share alike . united states license. off-topic or unsuitable comments will be deleted. dshr dshr in anwr recent comments full comments blog archive ► ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ▼ ( ) ▼ december ( ) crypto-currency as a basis for preservation economic failures of https hardware i/o virtualization "official" senate cia torture report talk at fall cni a note of thanks henry newman's farewell column ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► july ( ) ► june ( ) ► february ( ) ► ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) lockss system has permission to collect, preserve, and serve this archival unit. simple theme. powered by blogger. dshr's blog: rate limits dshr's blog i'm david rosenthal, and this is a place to discuss the work i'm doing in digital preservation. thursday, june , rate limits andrew marantz writes in reddit and the struggle to detoxify the internet: [on 's] april fools’, instead of a parody announcement, reddit unveiled a genuine social experiment. it was called r/place, and it was a blank square, a thousand pixels by a thousand pixels. in the beginning, all million pixels were white. once the experiment started, anyone could change a single pixel, anywhere on the grid, to one of sixteen colors. the only restriction was speed: the algorithm allowed each redditor to alter just one pixel every five minutes. “that way, no one person can take over—it’s too slow,” josh wardle, the reddit product manager in charge of place, explained. “in order to do anything at scale, they’re gonna have to coöperate." the r/place experiment successfully forced coöperation, for example with r/americanflaginplace drawing a stars and stripes, or r/blackvoid trying to rub out everything: toward the end, the square was a dense, colorful tapestry, chaotic and strangely captivating. it was a collage of hundreds of incongruous images: logos of colleges, sports teams, bands, and video-game companies; a transcribed monologue from “star wars”; likenesses of he-man, david bowie, the “mona lisa,” and a former prime minister of finland. in the final hours, shortly before the experiment ended and the image was frozen for posterity, blackvoid launched a surprise attack on the american flag. a dark fissure tore at the bottom of the flag, then overtook the whole thing. for a few minutes, the center was engulfed in darkness. then a broad coalition rallied to beat back the void; the stars and stripes regained their form, and, in the end, the flag was still there. what is important about the r/place experiment? follow me below the fold for an explanation. marantz wrote a long and very interesting article covering a lot of ground, but the r/place part is a great example of the importance to the integrity of the internet of rate limits. this is a topic i've written about before, for example in 's what could possibly go wrong and in 's brittle systems. in the latter i wrote: the design goal of almost all systems is to do what the user wants as fast as possible. this means that when the bad guy wrests control of the system from the user, the system will do what the bad guy wants as fast as possible. doing what the bad guy wants as fast as possible pretty much defines brittleness in a system; failures will be complete and abrupt. in last year's talk at uc berkeley's swarm lab i pointed out that rate limits were essential to lockss, and linked to paul vixie's article rate-limiting state making the case for rate limits on dns, ntp and other internet services. in how china censors the net: by making sure there’s too much information, john naughton reviews censored: distraction and diversion inside china’s great firewall by margaret roberts. the book is about how china "manages" the internet: censorship . is based on the idea that there are three ways of achieving the government’s desire to keep information from the public – fear, friction and flooding. fear is the traditional, analogue approach. it works, but it’s expensive, intrusive and risks triggering a backlash and/or the “streisand effect” – when an attempt to hide a piece of information winds up drawing public attention to what you’re trying to hide (after the singer tried to suppress photographs of her malibu home in ). friction involves imposing a virtual “tax” (in terms of time, effort or money) on those trying to access censored information. if you’re dedicated or cussed enough you can find the information eventually, but most citizens won’t have the patience, ingenuity or stamina to persevere in the search. friction is cheap and unobtrusive and enables plausible denial (was the information not available because of a technical glitch or user error?). flooding involves deluging the citizen with a torrent of information – some accurate, some phoney, some biased – with the aim of making people overwhelmed. in a digital world, flooding is child’s play: it’s cheap, effective and won’t generate backlash. (en passant, it’s what russia – and trump – do.) friction is the technique behind "walled gardens". the defense against friction is net neutrality, which is why the big (isp, content) companies hate it so much. note, as both r/place and vixie did, that the defense against flooding is rate limits, which implies keeping state. here is vixie: every reflection-friendly protocol mentioned in this article is going to have to learn rate limiting. this includes the initial tcp three-way handshake, icmp, and every udp-based protocol. in rare instances it's possible to limit one's participation in ddos reflection and/or amplification with a firewall, but most firewalls are either stateless themselves, or their statefulness is so weak that it can be attacked separately. the more common case will be like dns [response rate limiting], where deep knowledge of the protocol is necessary for a correctly engineered rate-limiting solution applicable to the protocol. engineering economics requires that the cost in cpu, memory bandwidth, and memory storage of any new state added for rate limiting be insignificant compared with an attacker's effort. attenuation also has to be a first-order goal—we must make it more attractive for attackers to send their packets directly to their victims than to bounce them off a ddos attenuator. internet routers could prevent many kinds of ddos attacks by implementing source address verification (sav), which would prevent attackers spoofing their packets' source address. but they don't, which makes rate limiting all stateless protocols essential: this effort will require massive investment and many years. it is far more expensive than sav would be, yet sav is completely impractical because of its asymmetric incentives. universal protocol-aware rate limiting (in the style of dns rrl, but meant for every other presently stateless interaction on the internet) has the singular advantage of an incentive model where the people who would have to do the work are actually motivated to do the work. this effort is the inevitable cost of the internet's "dumb core, smart edge" model and postel's law ("be conservative in what you do, be liberal in what you accept from others"). dns rrl was the first of these efforts. here is a simple explanation of how dns rrl works: if one packet with a forged source address arrives at a dns server, there is no way for the server to tell it is forged. if hundreds of packets per second arrive with very similar source addresses asking for similar or identical information, there is a very high probability of those packets, as a group, being part of an attack. the rrl software has two parts. it detects patterns in arriving queries, and when it finds a pattern that suggests abuse, it can reduce the rate at which the replies are sent. even before the elections social media platforms were discovering that they are just as vulnerable to ddos-style attacks as internet services further down the stack. damn the torpedoes, full speed ahead. andrew marantz writes in reddit and the struggle to detoxify the internet: social-media executives claim to transcend subjectivity, and they have designed their platforms to be feedback machines, giving us not what we claim to want, nor what might be good for us, but what we actually pay attention to. attention begets attention and the feedback loop accelerates. in twitter ceo wants to study platform’s “health,” but is he ignoring the cancer? sam machkovech quotes jack dorsey: we love instant, public, global messaging and conversation. it's what twitter is, and it's why we're here. but we didn't fully predict or understand the real-world negative consequences. ... we aren't proud of how people have taken advantage of our service or our inability to address it fast enough. "instant, public, global messaging and conversation" - twitter and reddit are clearly designed to go as fast as possible. machkovech writes: the current algorithm is designed solely to suggest and auto-forward content that is simply the busiest—the most liked, most seen stuff. actually, this isn't quite true. twitter started trying a form of rate-limiting more than a year ago. in twitter is now temporarily throttling reach of abusive accounts alex kantrowitz wrote: twitter is temporarily decreasing the reach of tweets from users it believes are engaging in abusive behavior via a new protocol that began rolling out last week. the protocol temporarily prevents tweets from users twitter deems abusive from being displayed to people who don't follow them, effectively reducing their reach. if the punished user mentions someone who doesn't follow them, for instance, that person would not see the tweet in their notifications tab. and if the punished user's followers retweet them, those retweets wouldn't be shown to people who don't follow them. those impacted by the new protocol are already tweeting screenshots of twitter's emails detailing their punishments. "we've detected some potentially abusive behavior from your account," the emails read. "so only your follower can see your activity on twitter for the amount of time shown below." the protocol doesn't limit the rate at which the victim can tweet, but it does limit the rate at which their tweets can spread via retweets. it does have an essential feature of rate limits, which acknowledges that abuse detection is probabilistic; the penalty times out. kantrowitz describes other, non-rate-limiting, efforts to curb abuse: in , dorsey made curbing harassment twitter’s top priority, and twitter’s product team released anti-harassment features at an unprecedented pace for the notoriously slow-moving company. they collapsed tweets they thought might be abusive, they built anti-abuse filters into search, they started allowing users to mute people who hadn’t confirmed their email addresses, phone numbers, or were using default profile pictures. they introduced a mute filter that could be applied to specific words. they even killed the default profile photo, doing away with the eggs that had long been synonymous with toxic trolls. despite these efforts, machkovech shows example after example of clearly abusive tweets from bogus accounts, enabled by the ease with which accounts can be created: dorsey pointed to the private research firm cortico, who created a series of conversation "health" metrics based on its studies of twitter data: shared attention; shared reality; variety; and receptivity. ... dorsey's calls for conversation health metrics do not in any way appreciate the apparent next-level disruption tactic already being rolled out on twitter this year: subtler, seemingly real accounts popping up with the express intent of passing those four metrics on their face. i have chronicled an apparent rise in this account type for the past few weeks at my own twitter account, often finding accounts that have existed for as briefly as a few months or as long as nine years. lower down the network stack these are called "sybil attacks"; the defense is to slow or restrict the activity of new identities, or to impose a proof-of-work to render account creation and/or early activity expensive. but, as machkovech is seeing, these defenses aren't effective against well-resourced adversaries with long time horizons: but it could be something even scarier: an effort to test and tease twitter's systems and to harvest innocent bystanders' reactions, thereby dumping fuel into an artificial intelligence-powered botnet. 's twitter is already confusing, in terms of verifying whether a drive-by poster is in any way legitimate. what happens if 's twitter is inundated in tens of thousands of real-sounding, tos-abiding robots—assuming twitter still exists by then? in the design of the lockss system we referred to this as the "wee forest folk" attack, one in which the attacker builds up a good reputation over time through a large number of low-value transactions, then cashes in over a short period via a small number of high-value transactions. it seems that temporarily limiting the rate and reach of posts from new and suspected abusive identities, the analogy of china's "friction", is necessary but not sufficient. it is also problematic for the platforms. facilitating the on-boarding of new customers is important to customer acquisition and retention. abuse detection is fallible, and will inevitably annoy some innocent customers. because the limits affect (at least some) actual people, they are more problematic than rate limits lower down the stack, which merely annoy software. posted by david. at : am labels: fault tolerance, social networks comments: david. said... "i asked ms. gadde in several different ways if there was anything mr. trump could tweet that might result in censure from the company. she declined to answer directly, pointing me instead to a january statement in which the company stated that blocking a world leader’s tweets “would hide important information people should be able to see and debate.” but what if that “important information” conflicts with twitter’s mission to promote a healthy public conversation? sooner or later, twitter’s executives and employees are going to have to make a decision about which is more important, mr. trump’s tweets or the company’s desire to promote a healthy public conversation. it’s hard to see how both are tenable." the conclusion of farhad manjoo's employee uprisings sweep many tech companies. not twitter. july , at : pm david. said... in tesla short-sellers harass pulitzer-winning journalist into deleting twitter account due to review kossack "rei" reports: "the tesla model has gotten no shortage of positive, sometimes glowing reviews from reviewers — but perhaps it was dan’s credentials that made this review one step too far for the short-sellers attempting to take down tesla. or perhaps it was the fact that musk retweeted him. ... dan soon fell under a storm of attacks for his review, both in the comments section of the wsj and more extensively on twitter. he was accused of being duped with a “rigged” car; of being in tesla’s pocket; of having bias against other brands; and a continuous onslaught of other attacks. dan spent much of friday and saturday defending himself on twitter against well-known short sellers such as mark spiegel and popular seeking alpha contributor montana skeptic. and then gave up. and deleted his account to terminate the harassment." twitter has a rate limit problem. july , at : am david. said... andrea james' twitter's nsfw porn spam nightmare for women with common names reports on another area where twitter needs rate limits: "for at least a couple of years, twitter has allowed one porn spam bot to clog up search results for common women's names, as well as for names of young female celebrities. it would not take a lot to create an algorithm to block this specific spam, but it's still here, because twitter can't seem to address the platform's pervasive hostility to women. the porn spambots typically pump out two posts a minute with a random string of sex-related search terms, along with a short video always overlaid with the same text and translucent rectangle to avoid copyright flagging of the clips they use. as soon as you block one, another appears." july , at : pm david. said... the bbc reports on another rate limit issue: "the volume of disinformation on the internet is growing so big that it is starting to crowd out real news, the commons digital, culture, media and sport committee chairman has said. tory mp damian collins said people struggle to identify "fake news". mps in their committee report said the issue threatens democracy and called for tougher social network regulation." july , at : am david. said... "a team of researchers at duo security has unearthed a sophisticated botnet operating on twitter — and being used to spread a cryptocurrency scam. ... the team used twitter’s api and some standard data enrichment techniques to create a large data set of million public twitter accounts, comprising more than half a billion tweets. ... the study led them into some interesting analysis of botnet architectures — and their paper includes a case study on the cryptocurrency scam botnet they unearthed (which they say was comprised of at least , bots “but likely much more”), and which attempts to syphon money from unsuspecting users via malicious “giveaway” link. ... ‘attempts’ being the correct tense because, despite reporting the findings of their research to twitter, they say this crypto scam botnet is still functioning on its platform — by imitating otherwise legitimate twitter accounts, including news organizations (such as the below example), and on a much smaller scale, hijacking verified accounts" from duo security researchers’ twitter ‘bot or not’ study unearths crypto botnet by natasha lomas. the duo team's paper is here. august , at : pm post a comment newer post older post home subscribe to: post comments (atom) blog rules posts and comments are copyright of their respective authors who, by posting or commenting, license their work under a creative commons attribution-share alike . united states license. off-topic or unsuitable comments will be deleted. dshr dshr in anwr recent comments full comments blog archive ► ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ▼ ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ▼ june ( ) cryptocurrencies have limits rate limits software heritage archive goes live the four most expensive words in the english language no-one could have predicted ... the island of misfit toys cryptographers on blockchains: part (updated) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► july ( ) ► june ( ) ► february ( ) ► ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) lockss system has permission to collect, preserve, and serve this archival unit. simple theme. powered by blogger. amazon rds for postgresql – amazon web services (aws) click here to return to amazon web services homepage contact sales support english my account create an aws account products solutions pricing documentation learn partner network aws marketplace customer enablement events explore more عربي bahasa indonesia deutsch english español français italiano português tiếng việt türkçe Ρусский ไทย 日本語 한국어 中文 (简体) 中文 (繁體) aws management console account settings billing & cost management security credentials aws personal health dashboard support center knowledge center aws support overview click here to return to amazon web services homepage products solutions pricing introduction to aws getting started documentation training and certification developer center customer success partner network aws marketplace support log into console download the mobile app amazon rds for postgresql overview features db engines pricing resources faqs customers partners amazon rds amazon aurora rds for mariadb rds for mysql rds for oracle rds for sql server rds for postgresql features performance insights amazon rds for postgresql set up, operate, and scale a relational database in the cloud with just a few clicks get started with amazon rds for postgresql postgresql has become the preferred open source relational database for many enterprise developers and start-ups, powering leading business and mobile applications. amazon rds makes it easy to set up, operate, and scale postgresql deployments in the cloud. with amazon rds, you can deploy scalable postgresql deployments in minutes with cost-efficient and resizable hardware capacity. amazon rds manages complex and time-consuming administrative tasks such as postgresql software installation and upgrades; storage management; replication for high availability and read throughput; and backups for disaster recovery. amazon rds for postgresql gives you access to the capabilities of the familiar postgresql database engine. this means that the code, applications, and tools you already use today with your existing databases can be used with amazon rds. amazon rds supports postgresql major version , which includes a number of enhancements to performance, robustness, transaction management, query parallelism, and more. benefits easy, managed deployments it takes only a few clicks in the aws management console to launch and connect to a production-ready postgresql database in minutes. amazon rds for postgresql database instances are pre-configured with parameters and settings for the server type you have selected. database parameter groups provide granular control and fine-tuning of your postgresql database. learn more » fast, predictable storage amazon rds provides two ssd-backed storage options for your postgresql database. general purpose storage provides cost-effective storage for small or medium-sized workloads. for high-performance oltp applications, provisioned iops delivers consistent performance of up to , ios per second. as your storage requirements grow you can provision additional storage on-the-fly with zero downtime. learn more » backup and recovery the automated backup feature of amazon rds enables recovery of your postgresql database instance to any point in time within your specified retention period of up to thirty five days. in addition, you can perform user-initiated backups of your db instance. these full database backups will be stored by amazon rds until you explicitly delete them. learn more » high availability and read replicas amazon rds multi-az deployments provide enhanced availability and durability for your postgresql databases, making them a natural fit for production database workloads. amazon rds read replicas make it easy to elastically scale out beyond the capacity constraints of a single databse instance for read-heavy database workloads. learn more » monitoring and metrics amazon rds for postgresql provides amazon cloudwatch metrics for your database instances at no additional charge and amazon rds enhanced monitoring provides access to over cpu, memory, file system, and disk i/o metrics. view key operational metrics in aws management console, including compute/memory/storage capacity utilization, i/o activity, and instance connections. learn more » isolation and security as a managed service, amazon rds provides a high level of security for your postgresql databases. these include network isolation using amazon virtual private cloud (vpc), encryption at rest using keys you create and control through aws key management service (kms) and encryption of data in transit using ssl. learn more >> featured customers view more amazon rds for postgresql customers >> check out the product details find out how to work with or migrate to amazon rds for postgresql learn more sign up for a free account instantly get access to the aws free tier. sign up start building in the console get started with amazon rds for postgresql in the aws console. sign in sign in to the console learn about aws what is aws? what is cloud computing? what is devops? what is a container? what is a data lake? aws cloud security what's new blogs press releases resources for aws getting started training and certification aws solutions portfolio architecture center product and technical faqs analyst reports aws partner network developers on aws developer center sdks & tools .net on aws python on aws java on aws php on aws javascript on aws help contact us aws careers file a support ticket knowledge center aws support overview legal create an aws account amazon is an equal opportunity employer: minority / women / disability / veteran / gender identity / sexual orientation / age. language عربي bahasa indonesia deutsch english español français italiano português tiếng việt türkçe Ρусский ไทย 日本語 한국어 中文 (简体) 中文 (繁體) privacy | site terms | cookie preferences | © , amazon web services, inc. or its affiliates. all rights reserved. dshr's blog: travels with a chromebook dshr's blog i'm david rosenthal, and this is a place to discuss the work i'm doing in digital preservation. tuesday, january , travels with a chromebook two years ago i wrote a note of thanks as i switched my disposable travel laptop from an asus seashell to an acer c chromebook running linux. two years later i'm still traveling with a c . below the fold, an update on my experiences. as i wrote two years ago, i started by replacing chromeos with hugegreenbug's port of ubuntu . . this gave me almost exactly the ui i was then running on my desktop, so i was happy with one exception: i'm still not used to the much bigger touch pad, and keep accidentally moving the cursor by touching the touch pad with the balls of my thumbs. this turned out to be a permanent irritation (but see below for what i should have done to fix it). although the combination of the c and ubuntu . gave sterling service for a long time, i started noticing an occasional crash. one of them was a very spectacular snow crash, which caused me to suspect a hardware problem. it was a day and a half before a trip, so i resorted again to amazon's same-day delivery for a second c . by the time it arrived all seemed well again with the first one, so i left with it and had no problem on the trip. now i had two c s, so i could experiment with alternate linux ports. i installed galliumos on the second one, and worked with it for a while. it was totally crash-free, being based on xubuntu . was significantly more up-to-date, and it was certainly lean-and-mean. derrick diener's galliumos: the linux distro specially designed for chromebook is a good review of galliumos. but i just couldn't come to terms with the xfce desktop and the way the touchpad had been configured to mimic the chromeos touchpad. i'd traded one irritation for two. then i found mark solters' ubuntu . on the google pixel , which reported: i hit upon trying linux mint “sarah”, currently beta. this version of mint is . based, and the installer boots on the pixel ! so i downloaded linux mint . "serena" - cinnamon ( -bit), wrote the .iso to a usb drive, popped it into the first c and powered up. the mint live system came right up! i played with it for a while. everything seemed to work, and the cinnamon ui was a lot more to my taste than xfce. worth a try. after backing up my files from . , i double-clicked on "install linux mint". after you answer the questions about keyboard and time zone and so on, the install starts with a worrisome long pause with the spinning cursor, but then things proceed normally with a progress bar. after it finished and i rebooted, i had a default linux mint system with gb of free space. my next step was to connect to my wifi network and sudo apt-get update sudo apt-get upgrade which updated packages. i have no travel for the next month and a working if slightly irritating environment on c # , so i can take the opportunity to clean things up and customize a nice fresh environment on c # . for the programming environment i needed: git - the lockss program is converging on gitlab.com and gitlab ce. emacs - which i still use for programming although i also use vi. the jdk, ant, libxml -utils and junit - needed to build the lockss software. docker for more general needs i added: tex live- still my favorite environment for writing papers. keepassx - everyone should use a password safe. gnucash zoom.us - a program committee i'm on uses this for distributed meetings, it works really well. so far, everything else i need is in the default install. at this point i have gb free. not a lot, but as in the . days i will travel with an external ssd that holds my development environment. then i closed the lid, had a cup of tea, and opened the lid again. things seemed to be working but, as i quickly discovered there was a problem. it turns out that the mint kernel shows symptoms that appeared on chromebook linux wikis back in : suspend/resume works the first time, but the next time the screen locks but the processor keeps running indefinitely. shutdown behaves similarly. and after a resume /var/log/kern.log is continually spammed with messages from ehci-pci saying resume error - . not good. i tried various fixes found by google-ing but none worked. then i had an inspiration. i had two machines: c # with a working kernel and a userland i didn't like. c # with a broken kernel and a userland i liked. all i needed to do was to get c # to boot the galliumos kernel not the mint one. i re-installed vanilla mint . on c # then here is what i did to it. stage was to get the galliumos fixes relevant to the suspend/resume issue into the mint userland: check out the calliumos config files for haswell by: git clone https://github.com/galliumos/galliumos-haswell copy each of the files into its appropriate place. stage was to get the galliumos kernel installed on c # : create a file /etc/apt/sources.list.d/additional-repositories.list containing the line: deb http://apt.galliumos.org/ xenon main copy the file /etc/apt/trusted.gpg.d/galliumos.gpg from c # to c # . run synaptic, reload, search for linux-image-galliumos, mark it for installation, apply. set the grub bootloader to display its menu by default so that if something goes wrong you can boot the mint kernel using the "advanced options" menu item. edit /etc/default/grub to read: #grub_hidden_timeout= grub_hidden_timeout_quiet=false grub_timeout= run update-grub to make the changes take effect. reboot, and the system comes up: $ uname -r . . -galliumos $ now suspend/resume works just the way it does on galliumos! three years ago redditor successincircuit posted list of fixes for xubuntu . on the acer c . some of these fixes still seem to be relevant: galliumos follows successincircuit's advice to optimize for the c s ssd. so i edited the root file system entry in /etc/fstab to be: /dev/mapper/mint--vg-root / ext discard,relatime,errors=remount-ro i find it necessary to disable the touchpad while typing. successincircuit says to do this with syndaemon, but there is a better way for ubuntu . , installing a touchpad control applet in the tray: $ sudo add-apt-repository ppa:atareao/atareao $ sudo apt-get update $ sudo apt-get install touchpad-indicator brave souls have upgraded the m. ssd, reinstalled chromeos, then installed linux, but i'm not yet up for that despite the lack of space. i'm now working on building the nice fresh environment on c # before my next trip. if i encounter anything else interesting i'll comment on this post. update th may : after another strange crash on c # which corrupted its disk, i found c # refurbished for $ . on amazon. this time i also got a gb m. ssd to replace the stock one. update th may i installed the gb ssd, recovered chrome os, installed mint cinnamon . and now have over gb free space on disk. to install the ssd i: turned the c over and removed the screws holding the back cover on. cracked the case open using a small screwdriver to get a front corner open, then slid a guitar plectrum along to free up the the rest of the clips. this step is a bit scary! removed the screw holding down the original ssd, replaced it with the new one, and replaced the screw. put the back cover back in place, pressed down on it to get most of the clips seated. replaced the screws, tightening them gently to seat the remaining clips. so now i have functional c s, albeit one with a habit of occasional mysterious crashes, each with mint userland and galliumos kernel. gratuitous copy and paste i notice that the links to the instructions for installing linux from a note of thanks are now broken, so i'm cutting and pasting them from the archlinux wiki so i can find them easily: enabling developer mode developer mode is necessary in order to access the superuser shell inside chrome os, which is required for making changes to the system like allow booting through seabios. warning: enabling developer mode will wipe all of your data. to enable developer mode: turn on the chrome os device. press and hold the esc + f (refresh) keys, then press the power button. this enters recovery mode. press ctrl + d (no prompt). it will ask you to confirm, then the system will revert its state and enable developer mode. note: press ctrl + d (or wait seconds for the beep and boot) at the white boot splash screen to enter chrome os. accessing the superuser shell after you have enabled the developer mode you will need to access the superuser shell. how you do this depends on whether you have configured chrome os or not. accessing the superuser shell without chrome os configuration if you have not configured chrome os, just press ctrl + alt + f (f is the "forward" arrow on the top row, →), you will see a login prompt. use chronos as the username, it should not prompt you for a password. become superuser with sudo, use the command sudo su -. accessing the superuser shell with chrome os configuration if you have configured chrome os already: open a crosh window with ctrl + alt + t. open a bash shell with the shell command. become superuser with sudo, use the command sudo su - to accomplish that. enabling seabios if your chrome os device did not ship with seabios or you prefer to install a custom firmware, then continue to flashing a custom firmware. this method will allow you to access the pre-installed version of seabios through the developer mode screen in coreboot. inside your superuser shell enter: # crossystem dev_boot_usb= dev_boot_legacy= reboot the machine. booting the installation media plug the usb drive to the chrome os device and start seabios with ctrl + l at the white boot splash screen (if seabios is not set as default). press esc to get a boot menu and select the number corresponding to your usb drive. i'll update this post again after i've tried replacing the ssd. posted by david. at : am labels: amazon, linux comments: david. said... after several trips with the galliumos kernel and the mint user-land everything is mostly fine. suspend has worked almost always· but it is important to note that suspending takes a very variable amount of time, sometimes several minutes. and interrupting the process while it is underway, for example by plugging or unplugging the power, seems to trigger the catatonic state. this may be because the machine decides to hibernate rather than suspend; i still need to figure out how to completely disable hibernate mode. april , at : am david. said... one more anomaly. the power brick is rated . a on the ac side. with the system charging the battery and in use playing a youtube video, it actually draws . a w from a v supply (measured with a kill-a-watt meter). but if you plug it into some sockets, such as the sockets on hotel lamps (claimed to supply a), and some of the seat power sockets on united, it appears to trip the current limiter and it doesn't get power. plug a usb charger into the same socket and it gets power. i don't understand why supplying a bit over half an amp causes a problem. april , at : pm david. said... at ars technica, j. m. porup has a review of two ways to run linux on a chromebook, crouton and the one i use, galliumos. june , at : am david. said... while i was on the road in new zealand, the power brick failed. when i got home, i replaced it with a much better one, smaller, lighter, with an indicator light, only two pins not three. february , at : pm david. said... recently i've been plagued by a failure of the cinnamon screen lock program to allow me to type my password after the machine wakes up. the only way out seems to be power-cycle, which is annoying. so i've replaced cinnamon with mate, which uses a different screen lock, on one of my machines and will use it for a while to see if the problem recurs. june , at : pm david. said... i should have known better. the ztc gb ssd, model ztc-sm - g that i bought from hot deals less(!) on amazon lasted . weeks of light usage in c # before failing with a massive outbreak of bad blocks. i sent a message to the vendor via amazon, but after two business days have yet to hear back from them. a few more days of no response and they'll get a -star review for crappy product and lousy customer support. july , at : pm david. said... the good news is that, so far, mate's screen lock has performed flawlessly. july , at : pm david. said... the saga continues. i replaced the ztc gb ssd with an adata gb ssd. it, and c # have been working fine except that # 's touchpad suffers an annoying jitter, so it was only comfortably usable with a mouse. so, having overcome my qualms about opening c s' cases, i upgraded # with a transcend gb ssd. the one remaining issue is that, very occasionally, when they are asleep each of the c s appear to have a nightmare and wake up. this drains the battery, and if they're in the bag at the time, can get them quite hot. october , at : pm david. said... google has announced official linux support for chromebooks: "google today announced chrome os is getting linux support. as a result, chromebooks will soon be able to run linux apps and execute linux commands. a preview of linux on the pixelbook will be released first, with support for more devices coming soon." linux runs in a vm under chromeos. may , at : pm david. said... this comment has been removed by the author. may , at : pm david. said... this comment has been removed by the author. may , at : pm david. said... i deleted the two previous comments because further experimentation revealed they were wrong. it appears that up-to-date mint (mate) plus the galliumos userland fixes has two problems on the c : ) with the galliumos . . kernel it suffers a gpu hang on resume from sleep that logs out the user. ) with the mint generic kernel . . - it suffers the same long-standing resume problem with ehci. mint . with the galliumos . . kernel works fine. may , at : am david. said... also, installing mint with an encrypted disk works, but the second time the system starts from cold entering the disk passphrase you set does not successfully decrypt the disk. may , at : pm david. said... after some experimentation, i now have mint . running on a c with the vanilla . . - -generic kernel not the galliumos one. instructions: * git clone https://github.com/galliumos/galliumos-haswell * copy files into place * edit /etc/default/grub as above * run blkid to get the uuid of the swap partition * create a file /etc/initramfs-tools/conf.d/resume contents: resume=uuid= * run update-grub * run update-initramfs -u -k all this kernel does not have the gpu hang. august , at : pm david. said... mate . works fine on the c but it can't use full-disk encryption. selecting full-disk encryption during the install process works, but the first or sometimes the second restart corrupts the encrypted volume so that the passphrase set during installation does not decrypt it. home directory encryption works fine on an unencrypted disk. this problem does not happen on a pc; i have verified this by installing mate . on an intel nuc. august , at : am david. said... tim anderson points out another reason to run linux on your chromebook in buying a chromebook? don't forget to check that best-before date: "it is unlikely to be printed on the box, but every chromebook has an "auto update expiration (aue) date" after which the operating system is unsupported by google. the authoritative document on the subject is here, where google explains that it "provides each new hardware platform with . years of auto update support". while . years sounds reasonable, google starts the clock ticking "when the first device on the platform is released". ... what happens when the dreaded aue date passes? this means there will be no more automatic software updates from google, no technical support from google, and "business and education customers... should not expect that they can manage their devices as expected using the google admin console"." it is (deliberately?) hard to figure out what the aue date is for your chromebook, but it looks as though my acer c s aue was last june. but they still work fine and get software updates because they run linux! august , at : am david. said... another good source of c information is here. december , at : am david. said... the c 's touchpad is one of its downsides, so you need to travel with a mouse. the tracpoint is way the best travel mouse i've ever used, and one of my kickstarter successes. it is tiny, yet the ergonomics are great - the grip looks strange but it feels like a pen. it is so small it is easily usable on the c 's space alongside the touchpad, which measures a scant . " wide by . " high. march , at : am david. said... the battery in the first of my c s expired. i ordered a dentsing ap j k battery from amazon ($ . including tax). i opened the case, undid the two screws holding the battery, replaced it and the screws, and reassembled the case. it works fine. october , at : pm david. said... this comment has been removed by the author. october , at : pm david. said... mint kernels up to and including . . - -generic go to sleep normally when the lid is closed, and wake up normally when it is opened. subsequent kernels . . - and . . - appear to go to sleep normally when the lid is closed but when it is opened do not wake up, they do a cold boot. this is possibly related to the known problem in the latest mint that prevents it working with encrypted swap, with which my c s are configured. october , at : pm post a comment newer post older post home subscribe to: post comments (atom) blog rules posts and comments are copyright of their respective authors who, by posting or commenting, license their work under a creative commons attribution-share alike . united states license. off-topic or unsuitable comments will be deleted. dshr dshr in anwr recent comments full comments blog archive ► ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ▼ ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ▼ january ( ) preservable emulations rick whitt on digital preservation the long tail of non-english science gresham's law star wars storage media transition (personal) error : blogger is bloggered travels with a chromebook ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► july ( ) ► june ( ) ► february ( ) ► ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) lockss system has permission to collect, preserve, and serve this archival unit. simple theme. powered by blogger. amazon ec t instances – amazon web services (aws) click here to return to amazon web services homepage contact sales support english my account create an aws account products solutions pricing documentation learn partner network aws marketplace customer enablement events explore more عربي bahasa indonesia deutsch english español français italiano português tiếng việt türkçe Ρусский ไทย 日本語 한국어 中文 (简体) 中文 (繁體) aws management console account settings billing & cost management security credentials aws personal health dashboard support center knowledge center aws support overview click here to return to amazon web services homepage products solutions pricing introduction to aws getting started documentation training and certification developer center customer success partner network aws marketplace support log into console download the mobile app amazon ec overview features pricing instance types faqs getting started resources developer resources product videos customers cost optimization instance type details instance type explorer amazon ec t instances low-cost general purpose amazon ec instances get started with t instances t instances are the low cost burstable general purpose instance type that provide a baseline level of cpu performance with the ability to burst cpu usage at any time for as long as required. t instances are designed for applications with moderate cpu usage that experience temporary spikes in use. t instances offer a balance of compute, memory, and network resources and are a very cost effective way to run a broad spectrum of general purpose workloads including large scale micro-services, small and medium databases, virtual desktops, and business-critical applications. t instances are also an affordable option to run your code repositories and development and test environments. amazon ec t overview video ( : ) click here for an informative amazon ec t infographic features low cost burstable cpu performance t instances are designed to run the majority of general purpose workloads at a much lower cost. t instances work by providing a baseline level of cpu performance to address many common workloads while providing the ability to burst above the baseline for times when more performance is required. t instances makes use of credits to track how much cpu is used. t instances accumulate cpu credits when a workload is operating below the baseline threshold and uses credits when running above the baseline threshold. t instances are unlike any other burstable instance available in the market today since customers can sustain high cpu performance, whenever and however long required. unlimited and standard mode t instances start in unlimited mode by default, giving users the ability to sustain high cpu performance over any desired time frame while keeping cost as low as possible. for most general-purpose workloads, t unlimited instances provide ample performance without any additional charges. if the average cpu utilization of a t instance is lower than the baseline over a -hour period, the hourly instance price automatically covers all interim spikes in usage. in the cases that the t instances needs to run at higher cpu utilization for a prolonged period, it can do so for a small additional charge of $ . per vcpu-hour. you can also choose to run in standard mode where a t instance can burst until it uses up all of its earned credits. for more details on t credits, please see the ec documentation page. multiple processor choices t instances feature either the st or nd generation intel xeon platinum series processor (skylake-sp or cascade lake) with a sustained all core turbo cpu clock speed of up to . ghz, and deliver up to % improvement in price performance compared to t instances. t instances provide support for the new intel advanced vector extensions (avx- ) instruction set, offering up to x the flops per core compared to the previous generation t instances. t a instances feature the amd epyc series processor with an all core turbo clock speed of up to . ghz. t a offers a % lower price than t instances for customers who are looking to further cost optimize their amazon ec compute environments. built on the aws nitro system t instances are built on the aws nitro system, a rich collection of building blocks that offloads many of the traditional virtualization functions to dedicated hardware. by doing so, the aws nitro system enables high performance, high availability, and high security while also reducing virtualization overhead. product details t t a t the t instances feature either the st or nd generation intel xeon platinum series processor (skylake-sp or cascade lake) with a sustained all core turbo cpu clock speed of up to . ghz. additionally there is support for the new intel advanced vector extensions (avx- ) instruction set, offering up to x the flops per core compared to the previous generation t instances. name vcpus memory (gib) baseline performance/vcpu cpu credits earned/hr network burst bandwidth (gbps) ebs burst bandwidth (mbps) on-demand price/hr* -yr reserved instance effective hourly* -yr reserved instance effective hourly* t .nano . % up to , $ . $ . $ . t .micro . % up to , $ . $ . $ . t .small . % up to , $ . $ . $ . t .medium . % up to , $ . $ . $ . t .large . % up to , $ . $ . $ . t .xlarge . % up to , $ . $ . $ . t . xlarge . % up to , $ . $ . $ . t a the t a instances feature amd epyc series processors with an all core turbo clock speed of . ghz. the amd-based instances provide additional options for customers that do not fully utilize the compute resources and can benefit from a cost savings of %. $name vcpus memory (gib) baseline performance/vcpu cpu credits earned/hr network burst bandwidth (gbps) ebs burst bandwidth (mbps) on-demand price/hr* -yr reserved instance effective hourly* -yr reserved instance effective hourly* t a.nano . % up to , $ . $ . $ . t a.micro . % up to , $ . $ . $ . t a.small . % up to , $ . $ . $ . t a.medium . % up to , $ . $ . $ . t a.large . % up to , $ . $ . $ . t a.xlarge . % up to , $ . $ . $ . t a. xlarge . % up to , $ . $ . $ . *prices shown are for linux/unix in us east (northern virginia) aws region. prices for -year and -year reserved instances are for "partial upfront" payment options or "no upfront" for instances without the partial upfront option. for full pricing details, see the amazon ec pricing page. t instances accumulate cpu credits when operating below baseline threshold. one cpu credit is equal to one vcpu running at % utilization for one minute. see t unlimited documentation for details on cpu credits and how they work. resources re:invent session: amazon ec t instances - burstable, cost-effective performance webinar: introducing new amazon ec t instances - general purpose burstable instances amazon ec t user guide get started with aws sign up for an aws account instantly get access to the aws free tier. learn with -minute tutorials explore and learn with simple tutorials. start building with aws begin building with step-by-step guides to help you launch your aws project. try amazon ec for free! click here to try for free aws free tier includes hours of linux and windows t .micro instances, ( t .micro for the regions in which t .micro is unavailable) each month for one year. to stay within the free tier, use only ec micro instances. view aws free tier details >> ready to get started? sign up have more questions? contact us sign in to the console learn about aws what is aws? what is cloud computing? what is devops? what is a container? what is a data lake? aws cloud security what's new blogs press releases resources for aws getting started training and certification aws solutions portfolio architecture center product and technical faqs analyst reports aws partner network developers on aws developer center sdks & tools .net on aws python on aws java on aws php on aws javascript on aws help contact us aws careers file a support ticket knowledge center aws support overview legal create an aws account amazon is an equal opportunity employer: minority / women / disability / veteran / gender identity / sexual orientation / age. language عربي bahasa indonesia deutsch english español français italiano português tiếng việt türkçe Ρусский ไทย 日本語 한국어 中文 (简体) 中文 (繁體) privacy | site terms | cookie preferences | © , amazon web services, inc. or its affiliates. all rights reserved. ranti. centuries.org ranti. centuries.org eternally yours on centuries home articles hello! archives contact keeping the dream alive - freiheit written by ranti - - t : : z i did not recall when the first time i heard it, but i remembered it was introduced by my cousin. this song from münchener freiheit became one of the songs i listen a lot. the lyrics (see below) resonate stronger nowadays. keeping the dream alive (single version) cover by david groeneveld: cover by kim wilde: lyrics: freiheit - keeping the dream alive tonight the rain is falling full of memories of people and places and while the past is calling in my fantasy i remember their faces the hopes we had were much too high way out of reach but we had to try the game will never be over because we're keeping the dream alive i hear myself recalling things you said to me the night it all started and still the rain is falling makes me feel the way i felt when we parted the hopes we had were much too high way out of reach but we have to try no need to hide no need to run 'cause all the answers come one by one the game will never be over because we're keeping the dream alive i need you i love you the game will never be over because we're keeping the dream alive the hopes we had were much too high way out of reach but we had to try no need to hide no need to run 'cause all the answers come one by one the hopes we had were much too high way out of reach but we had to try no need to hide no need to run 'cause all the answers come one by one the game will never be over because we're keeping the dream alive the game will never be over because we're keeping the dream alive the game will never be over… tags: (none) edit lou reed's walk on the wild side written by ranti - - t : : z if my memory serves me right, i heard about this walk on the wild side song (wikipedia) sometime during my college year in the s. of course, the bass and guitar reef were the one that captured my attention right away. at that time, being an international student here in the us, i was totally oblivious with the lyrics and the references on it. when i finally understood what the lyrics are about, listening to the song makes more sense. here's the footage of the walk on the wild side song (youtube) but what prompted me to write this was started by the version that amanda palmer sang for neil gaiman. i was listening to her cd "several attempts to cover songs by the velvet underground & lou reed for neil gaiman as his birthday approaches" and one of the songs was walk on the wild side. i like her rendition of the songs, which prompted me to find it on youtube. welp, that platform does not disappoint; it's a quite a nice piano rendition. of course, like any other platform that wants you to stay there, youtube also listed various walk on the wild side cover songs. one of them is from alice phoebe lou a singer-songwriter. her rendition using a guitar is also quite enjoyable (youtube) and now i have a new singer-songwriter to keep an eye on. among other videos that were listed on youtube is the one that kinda blew my mind, walk on the wild side - the story behind the classic bass intro featuring herbie flowers which explained that those are two basses layered on top of each other. man, what a nice thing to learn something new about this song. :-) tags: (none) edit tao written by ranti - - t : : z read it from the lazy yogi tags: (none) edit on climate change written by ranti - - t : : z read the whole poem tags: (none) edit tv news archive from the internet archive written by ranti - - t : : z i just learned about the existence of the tv news archive (covering news from until the day before today's date) containing news shows from us tv such as pbs, cbs, abc, foxnews, cnn, etc. you can search by the captions. they also have several curated collections like news clips regarding nsa or snippets or tv around the world i think some of you might find this useful. quite a nice collection, imo. tags: (none) edit public domain day (january , ): what could have entered it in and what did get released written by ranti - - t : : z copyright law is messy, yo. we won't see a lot of notable and important works entering public domain here in the us until . other countries, however, got to enjoy many of them first. public domain reviews put a list of creators whose work are entering the public domain for canada, european union (eu), and many other countries (https://publicdomainreview.org/collections/class-of- /.) for those in eu, nice to see h.g. wells name there (if uk do withdraw, this might end up not applicable to them. but, my knowledge about uk copyright law is zero, so, who knows.) as usual, center of study for the public domain from duke university put a list of some quite well-known works that are still under the extended copyright restriction: http://web.law.duke.edu/cspd/publicdomainday/ /pre- . those works would have been entered the public domain if we use the law that was applicable when they were published. i'm still baffled how current copyright hinders research done and published in to be made available freely. greedy publishers… so, thanks to that, usa doesn't get to enjoy many published works yet. "yet" is the operative word here because we don't know what the incoming administration would do on this topic. considering the next potus is a businessman, i fear the worst. i know: gloomy first of the year thought, but it is what it is. on a cheerful side, check the list from john mark ockerbloom on his online books project. it's quite an amazing project he's been working on. of course, there are also writings made available from hathitrust and gutenberg project, among other things. here's to the next days. xoxo tags: (none) edit for written by ranti - - t : : z read the full poem tags: (none) edit light written by ranti - - t : : z “light thinks it travels faster than anything but it is wrong. no matter how fast light travels, it finds the darkness has always got there first, and is waiting for it.” ― terry pratchett, reaper man tags: (none) edit dot-dot-dot written by ranti - - t : : z more about bertolt brecht poem tags: (none) edit assistive technology written by ranti - - t : : z many people would probably think assistive technology (at) are computer software, applications, or tools that are designed to help blind or deaf people. typically, the first thing that one might have in mind was screen readers, braille display, screen magnifier app for desktop reading, or physical objects like hearing aid, wheel chair, or crutches, a lot of people probably won't think glasses as an at. perhaps because glasses can be highly personalized to fit one's fashion style. tags: (none) edit recent popular posts keeping the dream alive - freiheit - - t : : z public domain day (january , ): what could have entered it in and what did get released - - t : : z dot-dot-dot - - t : : z the story of the chinese farmer - - t : : z the awesome things @ michigan state university - - t : : z lou reed's walk on the wild side - - t : : z tao - - t : : z on climate change - - t : : z tv news archive from the internet archive - - t : : z © — site powered by strong coffee & pictures of happy puppies this work is licensed under a creative commons attribution-noncommercial-noderivatives . international license. subscribe login searching jared mauch didn’t have good broadband—so he built his own fiber isp | ars technica skip to main content biz & it tech science policy cars gaming & culture store forums subscribe close navigate store subscribe videos features reviews rss feeds mobile site about ars staff directory contact us advertise with ars reprints filter by topic biz & it tech science policy cars gaming & culture store forums settings front page layout grid list site theme black on white white on black sign in comment activity sign up or login to join the discussions! stay logged in | having trouble? sign up to comment and more sign up one-man isp — jared mauch didn’t have good broadband—so he built his own fiber isp "i had to start a telephone company to get [high-speed] internet access." jon brodkin - jan , : pm utc reader comments with posters participating share this story share on facebook share on twitter share on reddit jared mauch installed about five miles' worth of this fiber cable. jared mauch closeup of the cables, which contain strands of fiber each. jared mauch mauch installed two sets of conduits, essentially empty tubes that fiber cables are installed into. they are different colors to make it obvious which one is which but are otherwise the same. jared mauch empty fiber conduits. jared mauch sometimes, there's water. jared mauch mauch's construction crew. jared mauch a ditch witch cable plow, which uses a vibrating blade to cut into the surface and drag fiber cables or fiber conduits through the ground. jared mauch a directional drill machine that installs cables or conduit under driveways and roads without digging giant holes. jared mauch a fiber blower—used to blow fiber cables through empty conduits—rigged together with parts from a hardware store. jared mauch a rented air compressor that mauch hooked up to his fiber blower. jared mauch the old saying "if you want something done right, do it yourself" usually isn't helpful when your problem is not having good internet service. but for one man in rural michigan named jared mauch, who happens to be a network architect, the solution to not having good broadband at home was in fact building his own fiber-internet service provider. "i had to start a telephone company to get [high-speed] internet access at my house," mauch explained in a recent presentation about his new isp that serves his own home in scio township, which is next to ann arbor, as well as a few dozen other homes in washtenaw county. mauch, a senior network architect at akamai in his day job, moved into his house in . at that point, he got a t line when . mbps was "a really great internet connection," he said. as broadband technology advanced, mauch expected that an isp would eventually wire up his house with cable or fiber. it never happened. enlarge / jared mauch jared mauch he eventually switched to a wireless internet service provider that delivered about mbps. mauch at one point contacted comcast, which told him it would charge $ , to extend its cable network to his house. "if they had priced it at $ , , i would have written them a check," mauch told ars. "it was so high at $ , that it made me consider if this is worthwhile. why would i pay them to expand their network if i get nothing back out of it?" at&t, the incumbent phone company, finally offered dsl to mauch about five years ago, he said. however, at&t's advertised plans for his neighborhood topped out at a measly . mbps—a good speed in , not in . at&t stopped offering basic dsl to new customers in october and hasn't upgraded many rural areas to modern replacements, leaving users like mauch without any great options. but about four years ago, mauch started planning to build his own provider that now offers fiber-to-the-home broadband in parts of scio township and lima township. mauch has installed five miles of fiber so far and began hooking up his first customers a few months ago. as of early january, mauch told us he had connected homes and had about more homes to wire up. he initially figured he'd get about percent of potential customers to buy service, but it's been about percent in reality. the customers that mauch has not yet hooked up are generally relying on cellular service, he said. advertisement washtenaw fiber properties llc the name of mauch's company is washtenaw fiber properties llc, and it's registered as a competitive access provider with the michigan state government. while technically a phone company, mauch provides only internet service without any phone or tv offerings. "my tariff is really funny," mauch said, explaining that the document he was required to file with the state explains that his company provides services only on an individual, case-by-case basis. mauch said he has spent about $ , , of which $ , went to the contractor that installed most of the fiber conduits. the fiber lines are generally about six feet underground and in some cases or feet underground to avoid gas pipes and other obstacles. mauch received a stop-work order from the county because he hadn't installed enough stakes along the right of way. jared mauch at customer homes, mauch installs a mikrotik rbftc media converter (the white box on the left) with a ubiquiti pon-to-ethernet module. the gray box on the right is a splice enclosure where fiber wires connect to the home. jared mauch control panel for a fusion splicer, which is used to splice fiber wires together. jared mauch an arista router at mauch's home connects to his own customers and to mauch's bandwidth supplier. jared mauch a flier advertising mauch's isp. a fiber distribution panel at mauch's house. each wire is a strand of fiber that goes to a customer or to mauch's bandwidth provider. jared mauch the inside of a splice tray that holds spliced fibers; there are numerous splice trays spread throughout the network. jared mauch an official washtenaw fiber properties pickup truck. jared mauch the biggest construction phase began in march . mauch had the contractor install two sets of conduits running side by side because it didn't cost much more than installing one set of conduits. having the extra currently empty conduit gives mauch the option of adding more fiber later; he could also lease or sell the empty conduit to another phone company down the line. installing the actual fiber cables into the conduits was a task that mauch did himself. a fiber blower can cost over $ , , but mauch said he built one using a rented air compressor and about $ worth of parts from a hardware store. mauch said he also spent $ , on a directional drill machine that installs cables or conduit under driveways and roads without digging giant holes. mauch buys internet connectivity and bandwidth for his isp from acd.net, a large network provider, but acd.net hasn't deployed fiber lines to mauch's neighborhood. mauch thus installed two miles of fiber from his home to acd.net's closest underground cable vaults, where he connected his fiber to their network. bandwidth supplied by acd.net now travels to a fiber distribution panel at mauch's property, allowing mauch's house to act as the hub that provides connectivity to his customers. mauch also bought a backup connection from net to provide redundancy. if mauch ever sells his house, he said he plans to grant himself an easement to access certain isp-related equipment on the property. the isp gear at mauch's home includes an arista router for talking to acd.net; a ubiquiti optical line terminal; an intel nuc server for network monitoring, graphing, and customer speed tests; a mac mini for backups; and a raspberry pi that serves as a backup dhcp server. he also has a whole-home backup generator, though his customers can still lose connectivity when their power goes out. advertisement at customer homes, mauch installs a mikrotik rbftc media converter with a ubiquiti pon-to-ethernet module. customers can supply their own wireless routers or buy one from mauch at cost—he doesn't do router rentals, which is generally a bad deal for customers anyway. costly construction mauch originally estimated the project would cost $ , , but it ended up being more than twice that. some customers spent $ , up front to help offset building costs and will receive service credits for multiple years in exchange now that the network is built. based on the amount mauch invested and his expected revenue, he estimates he'll break even within months. "i copied a prepay model from an existing isp who had experience with it," mauch said, noting that he learned from the experiences of several isps. one of the isps mauch learned from is vergennes broadband in michigan, a provider we wrote about in . now that mauch has built an isp, he said he has provided advice to several other people who are working on their own similar projects. further reading when at&t promises broadband—but delivers only kbps construction wasn't a breeze. mauch received one stop-work order from the county because he hadn't installed enough stakes along the right of way. mauch also ran into confusion over a requirement to provide hours' notice before work—he said he didn't realize he needed to provide that notice each time his crew did work. "permitting agencies are not always very clear about what their requirements are... and this is a barrier of entry for newer providers like me," mauch told us. there was another snag when a machine was stolen from one of mauch's work sites. "we actually found it for sale on facebook, and we managed to recover it as well due to diligent work on the part of the police and our own research," he said. the pandemic helped mauch a bit because there was less road traffic and people were generally at home, making it easier to run fiber to their houses, he said. the pandemic also helped local residents realize just how important broadband access is, which may have boosted the sign-up rate for mauch's service. mauch has been charging $ a month for symmetrical mbps service, $ for mbps, and $ for mbps, with an installation fee of up to $ depending on the installation's complexity (recently raised to $ for new signups). if a house is more than feet from the road, he charges an extra cents per foot to extend the cable. page: next → reader comments with posters participating share this story share on facebook share on twitter share on reddit jon brodkin jon is ars technica's senior it reporter, covering the fcc and broadband, telecommunications, tech policy, and more. email jon.brodkin@arstechnica.com // twitter @jbrodkin advertisement you must login or create an account to comment. channel ars technica ← previous story next story → related stories sponsored stories powered by today on ars store subscribe about us rss feeds view mobile site contact us staff advertise with us reprints newsletter signup join the ars orbital transmission mailing list to get weekly updates delivered to your inbox. sign me up → cnmn collection wired media group © condé nast. all rights reserved. use of and/or registration on any portion of this site constitutes acceptance of our user agreement (updated / / ) and privacy policy and cookie statement (updated / / ) and ars technica addendum (effective / / ). ars may earn compensation on sales from links on this site. read our affiliate link policy. your california privacy rights | do not sell my personal information the material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of condé nast. ad choices q dlg update - google docs javascript isn't enabled in your browser, so this file can't be opened. enable and reload. some word features can't be displayed in google docs and will be dropped if you make changes view details q dlg update share sign in the version of the browser you are using is no longer supported. please upgrade to a supported browser.dismiss file edit view tools help accessibility debug see new changes acrl techconnect acrl techconnect broken links in the discovery layer—pt. ii: towards an ethnography of broken links this post continues where my last one left off, investigating broken links in our discovery layer. be forewarned—most of it will be a long, dry list of all the mundane horrors of librarianship. metadata mismatches, ezproxy errors, and openurl resolvers, oh my! what does it mean when we say a link is broken? the simplest &# ; continue reading "broken links in the discovery layer—pt. ii: towards an ethnography of broken links" broken links in the discovery layer—pt. i: researching a problem like many administrators of discovery layers, i&# ;m constantly baffled and frustrated when users can&# ;t access full text results from their searches. after implementing summon, we heard a few reports of problems and gradually our librarians started to stumble across them on their own. at first, we had no formal system for tracking these errors. eventually, &# ; continue reading "broken links in the discovery layer—pt. i: researching a problem" orcid for system interoperability in scholarly communication workflows what is orcid? if you work in an academic library or otherwise provide support for research and scholarly communication, you have probably heard of orcid (open contributor &# ; researcher identifier) in terms of “orcid id,” a unique -digit identifier that represents an individual in order to mitigate name ambiguity. the orcid id number is presented &# ; continue reading "orcid for system interoperability in scholarly communication workflows" creating presentations with beautiful.ai updated - - at : pm with accessibility information. beautiful.ai is a new website that enables users to create dynamic presentations quickly and easily with “smart templates” and other design optimized features. so far the service is free with a paid pro tier coming soon. i first heard about beautiful.ai in an advertisement on npr and was &# ; continue reading "creating presentations with beautiful.ai" national forum on web privacy and web analytics we had the fantastic experience of participating in the national forum on web privacy and web analytics in bozeman, montana last month. this event brought together around forty people from different areas and types of libraries to do in-depth discussion and planning about privacy issues in libraries. our hosts from montana state university, scott young, &# ; continue reading "national forum on web privacy and web analytics" the ex libris knowledge center and orangewashing two days after proquest completed their acquisition of ex libris in december , ex libris announced the launch of their new online customer knowledge center. in the press release for the knowledge center, the company describes it as “a single gateway to all ex libris knowledge resources,” including training materials, release notes, and product manuals. &# ; continue reading "the ex libris knowledge center and orangewashing" managing ils updates we&# ;ve done a few screencasts in the past here at techconnect and i wanted to make a new one to cover a topic that&# ;s come up this summer: managing ils updates. integrated library systems are huge, unwieldy pieces of software and it can be difficult to track what changes with each update: new settings are &# ; continue reading "managing ils updates" blockchain: merits, issues, and suggestions for compelling use cases blockchain holds a great potential for both innovation and disruption. the adoption of blockchain also poses certain risks, and those risks will need to be addressed and mitigated before blockchain becomes mainstream. a lot of people have heard of blockchain at this point. but many are unfamiliar with how this new technology exactly works and &# ; continue reading "blockchain: merits, issues, and suggestions for compelling use cases" introducing our new best friend, gdpr you&# ;ve seen the letters gdpr in every single email you&# ;ve gotten from a vendor or a mailing list lately, but you might not be exactly sure what it is. with gdpr enforcement starting on may , it&# ;s time for a crash course in what gdpr is, and why it could be your new best friend &# ; continue reading "introducing our new best friend, gdpr" names are hard a while ago i stumbled onto the post &# ;falsehoods programmers believe about names&# ; and was stunned. personal names are one of the most deceptively difficult forms of data to work with and this article touched on so many common but unaddressed problems. assumptions like &# ;people have exactly one canonical name&# ; and &# ;my system will never &# ; continue reading "names are hard" google forms - create and analyze surveys, for free. one account. all of google. sign in to continue to forms enter your email find my account sign in with a different account create account one google account for everything google about google privacy terms help data unbound data unbound helping organizations access and share data effectively. special focus on web apis for data integration. some of what i missed from the cmd-d automation conference the cmd-d&# ;masters of automation one-day conference in early august would have been right up my alley: it’ll be a full day of exploring the current state of automation technology on both apple platforms, sharing ideas and concepts, and showing what’s possible—all with the goal of inspiring and furthering development of your own automation projects. fortunately, [&# ;] fine-tuning a python wrapper for the hypothes.is web api and other #ianno followup in anticipation of #ianno hack day, i wrote about my plans for the event, one of which was to revisit my own python wrapper for the nascent hypothes.is web api. instead of spending much time on my own wrapper, i spent most of the day working with jon udell&# ;s wrapper for the api. i&# ;ve been [&# ;] revisiting hypothes.is at i annotate i&# ;m looking forward to hacking on web and epub annotation at the #ianno hack day. i won&# ;t be at the i annotate conference per se but will be curious to see what comes out of the annual conference. i continue to have high hopes for digital annotations, both on the web and in non-web [&# ;] my thoughts about fargo.io using fargo.io organizing your life with python: a submission for pycon ? i have penciled into my calendar a trip to montreal to attend pycon . in my moments of suboptimal planning, i wrote an overly ambitious abstract for a talk or poster session i was planning to submit. as i sat down this morning to meet the deadline for submitting a proposal for a poster [&# ;] current status of data unbound llc in pennsylvania i&# ;m currently in the process of closing down data unbound llc in pennsylvania. i submitted the paperwork to dissolve the legal entity in april and have been amazed to learn that it may take up to a year to get the final approval done. in the meantime, as i establishing a similar california legal [&# ;] must get cracking on organizing your life with python talk and tutorial proposals for pycon are due tomorrow ( / ) . i was considering submitting a proposal until i took the heart the appropriate admonition against &# ;conference-driven&# ; development of the program committee. i will nonetheless use the oct and nov deadlines for lightning talks and proposals respectively to judge whether to [&# ;] embedding github gists in wordpress as i gear up i to write more about programming, i have installed the embed github gist plugin. so by writing &#x b;gist id= &#x d; in the text of this post, i can embed https://gist.github.com/rdhyee/ into the post to get: working with open data i&# ;m very excited to be teaching a new course working with open data at the uc berkeley school of information in the spring semester: open data — data that is free for use, reuse, and redistribution — is an intellectual treasure-trove that has given rise to many unexpected and often fruitful applications. in this [&# ;] a mundane task: updating a config file to retain old settings i want to have a hand in creating an excellent personal information manager (pim) that can be a worthy successor to ecco pro. so far, running eccoext (a clever and expansive hack of ecco pro) has been a eminently practical solution. you can download the most recent version of this actively developed extension from [&# ;] acrl techconnect skip to content acrl techconnect menu about authors broken links in the discovery layer—pt. ii: towards an ethnography of broken links this post continues where my last one left off, investigating broken links in our discovery layer. be forewarned—most of it will be a long, dry list of all the mundane horrors of librarianship. metadata mismatches, ezproxy errors, and openurl resolvers, oh my! what does it mean when we say a link is broken? the simplest definition would be: when a link that claims to lead to full text does not. but the way that many discovery layers work is by translating article metadata into a query in a separate database, which leads to some gray areas. what if the link leads to a search with only a single result, the resource in question? what if the link leads to a search with two results, a dozen, a hundred…and the resource is among them? what if the link leads to a journal index and it takes some navigation to get to the article’s full text? where do we draw the line? the user’s expectation is that selecting something that says “full text” leads to the source itself. i think all of the above count as broken links, though they obviously range in severity. some mean that the article simply cannot be accessed while others mean that the user has to perform a little more work. for the purposes of this study, i am primarily concerned with the first case: when the full text is nowhere near the link’s destination. as we discuss individual cases reported by end users, it will solidify our definition. long list i’m going to enumerate some types of errors i’ve seen, providing a specific example and detailing its nature as much as possible to differentiate the errors from each other. . the user selects a full text link but is taken to a database query that doesn’t yield the desired result. we had someone report this with an article entitled “land use: u.s. soil erosion rates–myth and reality” in summon which was translated into a query on the article’s issn, publication title, and an accidentally truncated title (just “land use”). the query fails to retrieve the article but does show other results. the article is present in the database and can be retrieved by editing the query, for instance by changing the title parameter to “u.s. soil erosion rates”. indeed, the database has the title as “u.s. soil erosion rates–myth and reality”. the article appears to be part of a recurring column and is labelled “policy forum: land use” which explains the discovery layer’s representation of the title. fundamentally, the problem is a disagreement about the title between the discovery layer and database. as another example, i’ve seen this problem occur with book reviews where one side prefixes the title with “review:” while the other does not. in a third instance of this, i’ve seen a query title = "julia brannen peter moss "and" ann mooney working "and" caring over the twentieth century palgrave macmillan basingstoke hampshire pp hbk £ isbn " where a lot of ancillary text spilled into the title. . the user is looking for a specific piece except the destination database combines this piece with similar ones into a single record with a generic title such that incoming queries fail. so, for instance, our discovery layer’s link might become a title query for book review: bad feminist by roxane gay in the destination, which only has an article named “book reviews” in the same issue of the host publication. in my experience, this is one of the more common discovery layer problems and can be described as a granularity mismatch. the discovery layer and subscription database disagree about what the fundamental unit of the publication is. while book reviews often evince this problem, so too do letters to the editor, opinion pieces, and recurring columns. . an article present in one of our subscription databases is not represented in the discovery layer, despite the database being correctly selected in the knowledgebase that informs the discovery system’s index. we’re able to read the article “kopfkino: julia phillips’ sculptures beyond the binary” in an ebsco database that provides access to the journal flash art international but no query in summon can retrieve it as a result. i suppose this is not technically a broken link as a non-existent link but it falls under the general umbrella of discovery layer content problems. . the exact inverse of the above: an article is correctly represented by the discovery layer index as being part of a database subscription that the user should have access to, but the article does not actually exist within the source database due to missing content. this occurred with an interview of howard willard in american artist from . while our subscription to art & architecture source does indeed include the issue of american artist in question, and one can read other articles from it, there was no record for the interview itself in ebscohost nor are its pages present in any of the pdf scans of the issue. . the user is looking for an article that is combined with another, even though the source seems to agree that they should be treated separately. for instance, one of our users was looking for the article “musical curiosities in athanasius kircher’s antiquarian visions” in the journal music in art but summon’s link lands on a broken link resolver page in the destination ebsco database. it turns out, upon closer inspection, that the pages for this article are appended to the pdf of the article that appears before it. all other articles for the issue have their own record. this is an interesting hybrid metadata/content problem similar to granularity mismatch: while there is no record for the article itself in the database, the article’s text is present. yet unlike some granularity mismatches it is impossible to circumvent via search; you have to know to browse the issue and utilize page numbers to locate it. . the user selects a link to an article published within the past year in a journal with a year-long embargo. the discovery layer shows a “full text online” link but because the source’s link resolver doesn’t consider an embargoed article to be a valid destination, the link lands on an error page. this is an instance where summon would, ideally, at least take to you to the article’s citation page but in any case the user won’t be able to retrieve the full text. . the user selects an article that is in a journal not contained within any of the library’s database subscriptions. this is usually simple knowledge base error where the journal lists for a database changed without being updated in the discovery layer index. still, it’s quite common because not all subscription changes are published in a machine-readable manner that would allow discovery layers to automate their ingestion. . the user selects an article listed as being published in in the discovery layer, while the source database has so the openurl fails to resolve properly. upon investigation, this date mismatch can be traced back to the journal’s publisher which lists the individual articles as being published in while the issue in which they are contained comes from . the summon support staff rightly points out to me that they can’t simply change the article dates to match one source; while it might fix some links, it will break others, and this date mismatch is a fundamentally unsolvable disagreement. this issue highlights the brittleness of real world metadata; publishers, content aggregators, and discovery products do not live in harmony. reviewing the list of problems, this dual organization seems to helpfully group like issues: metadata & linking problems metadata mismatch ( , , ) granularity mismatch ( ) link resolver error ( ) index problems article not in database/journal/index ( , , , ) journal not in database ( ) of these three, the first category accounts for the vast majority of problems according to my anecdata. it’s notable that issues overlap and their classification is inexact. when a link to an embargoed article fails, should we say that is due to the article being “missing” or a link resolver issue? whatever the case, it is often clear when a link is broken even if we could argue endlessly about how exactly. there are also a host of problems that we, as librarians, cause. we might misconfigure ezproxy for a database or fail to keep our knowledge base holdings up to date. the difference with these problems is that they tend to happen once and then be resolved forever; i fix the ezproxy stanza, i remove access to the database we unsubscribed from. so the proportion of errors we account for is vanishingly low, while these other errors are eternal. no matter how many granularity mismatches or missing articles in i point out, there are always millions more waiting to cause problems for our users. notes this sort of incredibly poor handling of punctuation in queries is sadly quite common. even though, in this instance, the source database and discovery layer are made by the same company the link between them still isn’t prepared to handle a colon in a text string. consider how many academic articles have colons in their title. this is not good. ↩ author eric phetteplaceposted on july , categories discovery, metadata comment on broken links in the discovery layer—pt. ii: towards an ethnography of broken links broken links in the discovery layer—pt. i: researching a problem like many administrators of discovery layers, i’m constantly baffled and frustrated when users can’t access full text results from their searches. after implementing summon, we heard a few reports of problems and gradually our librarians started to stumble across them on their own. at first, we had no formal system for tracking these errors. eventually, i added a script which inserted a “report broken link” form into our discovery layer’s search results. i hoped that collecting reported problems and then reporting then would identify certain systemic issues that could be resolved, ultimately leading to fewer problems. pointing out patterns in these errors to vendors should lead to actual progress in terms of user experience. from the broken links form, i began to cull some data on the problem. i can tell you, for instance, which destination databases experience the most problems or what the character of the most common problems is. the issue is the sample bias—are the problems that are reported really the most common? or are they just the ones that our most diligent researchers (mostly our librarians, graduate students, and faculty) are likely to report? i long for quantifiable evidence of the issue without this bias. how i classify the broken links that have been reported via our form. n = select searches & search results so how would one go about objectively studying broken links in a discovery layer? the first issue to solve is what searches and search results to review. luckily, we have data on this—we can view in our analytics what the most popular searches are. but a problem becomes apparent when one goes to review those search terms: artstor hours jstor kanopy of course, the most commonly occurring searches tend to be single words. these searches all trigger “best bet” or database suggestions that send users directly to other resources. if their result lists do contain broken links, those links are unlikely to ever be visited, making them a poor choice for our study. if i go a little further into the set of most common searches, i see single-word subject searches for “drawing” followed by some proper nouns (“suzanne lacy”, “chicago manual of style”). these are better since it’s more likely users actually select items from their results but still aren’t a great representation of all the types of searches that occur. why are these types of single-word searches not the best test cases? because search phrases necessarily have a long tail distribution; the most popular searches aren’t that popular in the context of the total quantity of searches performed . there are many distinct search queries that were only ever executed once. our most popular search of “artstor”? it was executed times over the past two years. yet we’ve had somewhere near , searches in the past six months alone. this supposedly popular phrase has a negligible share of that total. meanwhile, just because a search for “how to hack it as a working parent. jaclyn bedoya, margaret heller, christina salazar, and may yan. code lib ( ) iss. ″ has only been run once doesn’t mean it doesn’t represent a type of search—exact citation search—that is fairly common and worth examining, since broken links during known item searches are more likely to be frustrating. even our most popular searches evince a long tail distribution. so let’s say we resolve the problem of which searches to choose by creating a taxonomy of search types, from single-word subjects to copy-pasted citations. we can select a few real world samples of each type to use in our study. yet we still haven’t decided which search results we’re going to examine! luckily, this proves much easier to resolve. people don’t look very far down in the search results , rarely scrolling past the first “page” listed (summon has an infinite scroll so there technically are no pages, but you get the idea). only items within the first ten results are likely to be selected. once we have our searches and know that we want to examine only the first ten or so results, my next thought is that it might be worth filtering our results that are unlikely to have problems. but does skipping the records from our catalog, institutional repository, libguides, etc. make other problems abnormally more apparent? after all, these sorts of results are likely to work since we’re providing direct links to the summon link. also, our users do not heavily employ facets—they would be unlikely to filter out results from the library catalog. in a way, by focusing a study on search results that are the most likely to fail and thus give us information about underlying linking issues, we’re diverging away from the typical search experience. in the end, i think it’s worthwhile to stay true to more realistic search patterns and not apply, for instance, a “full text online” filter which would exclude our library catalog. next time on tech connect—oh how many ways can things go wrong?!? i’ll start investigating broken links and attempt to enumerate their differing natures. notes this script was largely copied from robert hoyt of fairfield university, so all credit due to him. ↩ for instance, see: beitzel, s. m., jensen, e. c., chowdhury, a., frieder, o., & grossman, d. ( ). temporal analysis of a very large topically categorized web query log. journal of the american society for information science and technology, ( ), – . “… it is clear that the vast majority of queries in an hour appear only one to five times and that these rare queries consistently account for large portions of the total query volume” ↩ ignore, for the moment, that this taxonomy’s constitution is an entire field of study to itself. ↩ pan, b., hembrooke, h., joachims, t., lorigo, l., gay, g., & granka, l. ( ). in google we trust: users’ decisions on rank, position, and relevance. journal of computer-mediated communication, ( ), – . ↩ in fact, the most common facet used in our discovery layer is “library catalog” showing that users often want only bibliographic records; the precise opposite of a search aimed at only retrieving article database results. ↩ author eric phetteplaceposted on march , march , categories data, discovery comments on broken links in the discovery layer—pt. i: researching a problem orcid for system interoperability in scholarly communication workflows what is orcid? if you work in an academic library or otherwise provide support for research and scholarly communication, you have probably heard of orcid (open contributor & researcher identifier) in terms of “orcid id,” a unique -digit identifier that represents an individual in order to mitigate name ambiguity. the orcid id number is presented as a uri (unique resource identifier) that serves as the link to a corresponding orcid record, where disambiguating data about an individual is stored. for example, https://orcid.org/ - - - x is the orcid id for the late stephen hawking, and clicking on this link will take you to hawking’s orcid record. data within orcid records can include things like names(s) and other identifiers, biographical information, organizational affiliations, and works. figure : this screenshot shows the types of data that can be contained in an orcid record. anyone can register for an orcid id for free, and individuals have full control over what data appears in their record, the visibility of that data, and whether other individuals or organizations are authorized to add data to their orcid record on their behalf. individuals can populate information in their orcid record themselves, or they can grant permission to organizations, like research institutions, publishers, and funding agencies, to connect with their orcid record as trusted parties, establishing an official affiliation between the individual and the organization. for example, figures and illustrate an authenticated orcid connection between an individual author and the university of virginia (uva) as represented in libraopen, the uva library’s samvera institutional repository. figure : the university of virginia library’s libraopen institutional repository is configured to make authenticated connections with authors’ orcid records, linking the author to their contributions and to the institution. once an author authenticates/connects their orcid id in the system, orcid id uris are displayed next to the authors’ names. image source: doi.org/ . /v fb t figure : by clicking on the author’s orcid id uri in libraopen, we can see the work listed on the individual’s orcid record, with “university of virginia” as the source of the data, which means that the author gave permission for uva to write to their orcid record. this saves time for the author, ensures integrity of metadata, and contributes trustworthy data back to the scholarly communication ecosystem that can then be used by other systems connected with orcid. image courtesy of sherry lake, uva https://orcid.org/ - - - orcid ecosystem & interoperability these authenticated connections are made possible by configuring software systems to communicate with the orcid registry through the orcid api, which is based on oauth . . with individual researchers/contributors at the center, and their affiliated organizations connecting with them through the orcid api, all participating organizations’ systems can also communicate with each other. in this way, orcid not only serves as a mechanism for name disambiguation, it also provides a linchpin for system interoperability in the research and scholarly communication ecosystem. figure : orcid serves as a mechanism for interoperability between systems and data in the scholarly communication ecosystem. graphic courtesy of the orcid organization. publishers, funders, research institutions (employers), government agencies, and other stakeholders have been adopting and using orcid increasingly in their systems over the past several years. as a global initiative, over million individuals around the world have registered for an orcid id, and that number continues to grow steadily as more organizations start to require orcid ids in their workflows. for example, over publishers have signed on to an open letter committing to use orcid in their processes, and grant funders are continuing to come on board with orcid as well, having recently released their own open letter demonstrating commitment to orcid. a full list of participating orcid member organizations around the globe can be found at https://orcid.org/members. orcid integrations orcid can be integrated into any system that touches the types of data contained within an orcid record, including repositories, publishing and content management platforms, data management systems, central identity management systems, human resources, grants management, and current research information systems (cris). orcid integrations can either be custom built into local systems, such as the example from uva above, or made available through a vendor system out of the box. several vendor-hosted cris such as pure, faculty , digital measures, and symplectic elements, already have built-in support for authenticated orcid connections that can be utilized by institutional orcid members, which provides a quick win for pulling orcid data into assessment workflows with no development required. while orcid has a public api that offers limited functionality for connecting with orcid ids and reading public orcid data, the orcid member api allows organizations to read from, write to, and auto-update orcid data for their affiliated researchers. the orcid institutional membership model allows organizations to support the orcid initiative and benefit from the more robust functionality that the member api provides. orcid can be integrated with disparate systems, or with one system from which data flows into others, as illustrated in figure . figure : this graphic from the czech technical university in prague illustrates how a central identity management system is configured to connect with the orcid registry via the orcid api, with orcid data flowing internally to other institutional systems. image source: czech technical university in prague central library & computing and information centre , : solving a problem of authority control in dspace during orcid implementation orcid in us research institutions in january of , four consortia in the us – the northeast research libraries (nerl), the greater western library alliance (gwla), the big ten academic alliance (btaa), and lyrasis – joined forces to form a national partnership for a consortial approach to orcid membership among research institutions in the us, known as the orcid us community. the national partnership allows non-profit research institutions to become premium orcid member organizations for a significantly discounted fee and employs staff to provide dedicated technical and community support for its members. as of december , , there are member organizations in the orcid us community. in addition to encouraging adoption of orcid, a main goal of the consortium approach is to build a community of practice around orcid in the us. prior to , any institutions participating in orcid were essentially going it alone and there were no dedicated communication channels or forums for discussion and sharing around orcid at a national level. however, with the formation of the orcid us community, there is now a website with community resources for orcid adoption specific to the us, dedicated communication channels, and an open door to collaboration between member institutions. among orcid us community member organizations, just under half have integrated orcid with one or more systems, and the other slightly more than half are either in early planning stages or technical development. (see the orcid us community newsletter for more information.) as an ecosystem, orcid relies not only on organizations but also the participation of individual researchers, so all members have also been actively reaching out to their affiliated researchers to encourage them to register for, connect, and use their orcid id. getting started with orcid orcid can benefit research institutions by mitigating confusion caused by name ambiguity, providing an interoperable data source that can be used for individual assessment and aggregated review of institutional impact, allowing institutions to assert authority over their institutional name and verify affiliations with researchers, ultimately saving time and reducing administrative burden for both organizations and individuals. to get the most value from orcid, research institutions should consider the following three activities as outlined in the orcid us planning guide: forming a cross-campus orcid committee or group with stakeholders from different campus units (libraries, central it, research office, graduate school, grants office, human resources, specific academic units, etc.) to strategically plan orcid system integration and outreach efforts assessing all of the current systems used on campus to determine which workflows could benefit from orcid integration conducting outreach and education around research impact and orcid to encourage researchers to register for and use their orcid id the more people and organizations/systems using orcid, the more all stakeholders can benefit from orcid by maintaining a record of an individuals’ scholarly and cultural contributions throughout their career, mitigating confusion caused by name ambiguity, assessing individual contributions as well as institutional impact, and enabling trustworthy and efficient sharing of data across scholarly communication workflows. effectively, orcid represents a paradigm shift from siloed, repetitive workflows to the ideal of being able to “enter once, re-use often” by using orcid to transfer data between systems, workflows, and individuals, ultimately making everyone’s lives easier. sheila rabun is the orcid us community specialist at lyrasis, providing technical and community support for + institutional members of the orcid us community. in prior roles, she managed community and communication for the international image interoperability framework (iiif) consortium, and served as a digital project manager for several years at the university of oregon libraries’ digital scholarship center. learn more at https://orcid.org/ - - - author sheila rabunposted on december , december , categories digital scholarship, publication, scholarly communication creating presentations with beautiful.ai updated - - at : pm with accessibility information. beautiful.ai is a new website that enables users to create dynamic presentations quickly and easily with “smart templates” and other design optimized features. so far the service is free with a paid pro tier coming soon. i first heard about beautiful.ai in an advertisement on npr and was immediately intrigued. the landscape of presentation software platforms has broadened in recent years to include websites like prezi, emaze, and an array of others beyond the tried and true powerpoint. my preferred method of creating presentations for the past couple of years has been to customize the layouts available on canva and download the completed pdfs for use in powerpoint. i am also someone who enjoys tinkering with fonts and other design elements until i get a presentation just right, but i know that these steps can be time consuming and overwhelming for many people. with that in mind, i set out to put beautiful.ai to the test by creating a short “prepare and share” presentation about my first experience at ala’s annual conference this past june for an upcoming meeting. a title slide created with beautiful.ai. features to help you get started, beautiful.ai includes an introductory “design tips for beautiful slides” presentation. it is also fully customizable so you can play around with all of of the features and options as you explore, or you can click on “create new presentation” to start from scratch. you’ll then be prompted to choose a theme, and you can also choose a color palette. once you start adding slides you can make use of beautiful.ai’s template library. this is the foundation of the site’s usefulness because it helps alleviate guesswork about where to put content and that dreaded “staring at the blank slide” feeling. each individual slide becomes a canvas as you create a presentation, similar to what is likely familiar in powerpoint. in fact, all of the most popular powerpoint features are available in beautiful.ai, they’re just located in very different places. from the navigation at the left of the screen users can adjust the colors and layout of each slide as well as add images, animation, and presenter notes. options to add, duplicate, or delete a slide are available on the right of the screen. the organize feature also allows you to zoom out and see all of the slides in the presentation. beautiful.ai offers a built-in template to create a word cloud. one of beautiful.ai’s best features, and my personal favorite, is its built-in free stock image library. you can choose from pre-selected categories such as data, meeting, nature, or technology or search for other images. an import feature is also available, but providing the stock images is extremely useful if you don’t have your own photos at the ready. using these images also ensures that no copyright restrictions are violated and helps add a professional polish to your presentation. the options to add an audio track and advance times to slides are also nice to have for creating presentations as tutorials or introductions to a topic. when you’re ready to present, you can do so directly from the browser or export to pdf or powerpoint. options to share with a link or embed with code are also available. usability while intuitive design and overall usability won’t necessarily make or break the existence of a presentation software platform, each will play a role in influencing whether someone uses it more than once. for the most part, i found beautiful.ai to be easy and fun to use. the interface is bold, yet simplistic, and on trend with current website design aesthetics. still, users who are new to creating presentations online in a non-powerpoint environment may find the beautiful.ai interface to be confusing at first. most features are consolidated within icons and require you to hover over them to reveal their function. icons like the camera to represent “add image” are pretty obvious, but others such as layout and organize are less intuitive. some of beautiful.ai’s terminology may also not be as easily recognizable. for example, the use of the term “variations” was confusing to me at first, especially since it’s only an option for the title slide. the absence of any drag and drop capability for text boxes is definitely a feature that’s missing for me. this is really where the automated design adaptability didn’t seem to work as well as i would’ve expected given that it’s one of the company’s most prominent marketing statements. on the title slide of my presentation, capitalizing a letter in the title caused the text to move closer to the edge of the slide. in canva, i could easily pull the text block over to the left a little or adjust the font size down by a few points. i really am a stickler for spacing in my presentations, and i would’ve expected this to be an element that the “design ai” would pick up on. each template also has different pre-set design elements, and it can be confusing when you choose one that includes a feature that you didn’t expect. yet, text sizes that are pre-set to fit the dimensions of each template does help not only with readability in the creation phase but with overall visibility for audiences. again, this alleviates some of the guesswork that often happens in powerpoint with not knowing exactly how large your text sizes will appear when projected onto larger screens. a slide created using a basic template and stock photos available in beautiful.ai. one feature that does work really well is the export option. exporting to powerpoint creates a perfectly sized facsimile presentation, and being able to easily download a pdf is very useful for creating handouts or archiving a presentation later on. both are nice to have as a backup for conferences where internet access may be spotty, and it’s nice that beautiful.ai understands the need for these options. unfortunately, beautiful.ai doesn’t address accessibility on its faq page nor does it offer alternative text or other web accessibility features. users will need to add their own slide titles and alt text in powerpoint and adobe acrobat after exporting from beautiful.ai to create an accessible presentation. conclusion beautiful.ai challenged me to think in new ways about how best to deliver information in a visually engaging way. it’s a useful option for librarians and students who are looking for a presentation website that is fun to use, engaging, and on trend with current web design. click here to view “my first ala”presentation created with beautiful.ai. jeanette sewell is the database and metadata management coordinator at fondren library, rice university. author jeanette sewellposted on november , november , categories conferences, library, presentation, technology, tools national forum on web privacy and web analytics we had the fantastic experience of participating in the national forum on web privacy and web analytics in bozeman, montana last month. this event brought together around forty people from different areas and types of libraries to do in-depth discussion and planning about privacy issues in libraries. our hosts from montana state university, scott young, jason clark, sara mannheimer, and jacqueline frank, framed the event with different (though overlapping) areas of focus. we broke into groups based on our interests from a pre-event survey and worked through a number of activities to identify projects. you can follow along with all the activities and documents produced during the forum in this document that collates all of them. float your boat exercise while initially worried that the activities would feel too forced, instead they really worked to release creative ideas. here’s an example: our groups drew pictures of boats with sails showing opportunities, and anchors showing problems. we started out in two smaller subgroups of our subgroups and drew a boat, then met with the large subgroup to combine the boat ideas. this meant that it was easy to spot the common themes—each smaller group had written some of the same themes (like gdpr). working in metaphor meant we could express some more complex issues, like politics, as the ocean—something that always surrounds the issue and can be helpful or unhelpful without much warning. this helped us think differently about issues and not get too focused on our own individual perspective. the process of turning metaphor into action was hard. we had to take the whole world of problems and opportunities and come up with how these could be realistically accomplished. good and important ideas had to get left behind because they were so big there was no way to feasibly plan them, certainly not in a day or two. the differing assortment of groups (which were mixable where ideas overlapped) ensured that we were able to question each other’s assumptions and ask some hard questions. for example, one of the issues margaret’s group had identified as a problem was disagreement in the profession about what the proper limits were on privacy. individually identifiable usage metrics are a valuable commodity to some, and a thing not to be touched to others. while everyone in the room was probably biased more in favor of privacy than perhaps the profession at large is, we could share stories and realities of the types of data we were collecting and what it was being used for. considering the realities of our environments, one of our ideas to bring everyone from across the library and archives world to create a unified set of privacy values was not going to happen. despite that, we were able to identify one of the core problems that led to a lack of unity, which was, in many cases, lack of knowledge about what privacy issues existed and how these might affect institutions. when you don’t completely understand something, or only half understand it, you are more likely to be afraid of it. on the afternoon of the second day and continuing into the morning of the third day, we had to get serious and pick just one idea to focus on to create a project plan. again, the facilitators utilized a few processes that helped us take a big idea and break it down into more manageable components. we used “big scai” thinking to frame the project: what is the status quo, what are the challenges, what actions are required, and what are the ideals. from there we worked through what was necessary for the project, nice to have, unlikely to get, and completely unnecessary to the project. this helped focus efforts and made the process of writing a project implementation plan much easier. what the workday looked like. writing the project implementation plan as a group was made easier by shared documents, but we all commented on the irony of using google docs to write privacy plans. on the other hand, trying to figure out how to write in groups and easily share what we wrote using any other platform was a challenge in the moment. this reality illustrates the problems with privacy: the tool that is easiest to use and comes to mind first will be the one that ends up being used. we have to create tools that make privacy easy (which was a discussion many of us at the forum had), but even more so we need to think about the tradeoffs that we make in choosing a tool and educate ourselves and others about this. in this case, since all the outcomes of the project were going to be public anyway, going on the “quick and easy” side was ok. the forum project leaders recently presented about their work at the dlf forum conference. in this presentation, they outlined the work that they did leading up to the forum, and the strategies that emerged from the day. they characterized the strategies as privacy badging and certifications, privacy leadership training, privacy for tribal communities and organizations, model license for vendor contracts, privacy research institute, and a responsible assessment toolkit. you can read through the thought process and implementation strategies for these projects and others yourself at the project plan index. the goal is to ensure that whoever wants to do the work can do it. to quote scott young’s follow-up email, “we ask only that you keep in touch with us for the purposes of community facilitation and grant reporting, and to note the provenance of the idea in future proposals—a sort of cc by designation, to speak in copyright terms.” for us, this three-day deep dive into privacy was an inspiration and a chance to make new connections (while also catching up with some old friends). but even more, it was a reminder that you don’t need much of anything to create a community. provided the right framing, as long as you have people with differing experiences and perspectives coming together to learn from each other, you’ve facilitated the community-building. author margaret hellerposted on october , october , categories conferences, privacy the ex libris knowledge center and orangewashing two days after proquest completed their acquisition of ex libris in december , ex libris announced the launch of their new online customer knowledge center. in the press release for the knowledge center, the company describes it as “a single gateway to all ex libris knowledge resources,” including training materials, release notes, and product manuals. a defining feature is that there has never been any paywall or log-on requirement, so that all knowledge center materials remain freely accessible to any site visitor. historically, access to documentation for automated library systems has been restricted to subscribing institutions, so the knowledge center represents a unique change in approach. within the press release, it is also readily apparent how ex libris aims to frame the openness of the knowledge center as a form of support for open access. as the company states in the second paragraph, “demonstrating the company’s belief in the importance of open access, the site is open to all, without requiring any logon procedure.” former ex libris ceo matti shem tov goes a step further in the following paragraph: “we want our resources and documentation to be as accessible and as open as our library management, discovery, and higher-education technology solutions are.” the problem with how ex libris frames their press release is that it elides the difference between mere openness and actual open access. they are a for-profit company, and their currently burgeoning market share is dependent upon a software-as-a-service (saas) business model. therefore, one way to describe their approach in this case is orangewashing. during a recent conversation with me, margaret heller came up with the term, based on the color of the plos open access symbol. similar in concept to greenwashing, we can define orangewashing as a misappropriation of open access rhetoric for business purposes. what perhaps makes orangewashing more initially difficult to diagnose in ex libris’s (and more broadly, proquest’s) case is that they attempt to tie support for open access to other product offerings. even before purchasing ex libris, proquest had been including an author-side paid open-access publishing option to its electronic thesis and dissertation platform, though we can question whether this is actually a good option for authors. for its part, ex libris has listened to customer feedback about open access discovery. as an example, there are now open access filters for both the primo and summon discovery layers. ex libris has also, generally speaking, remained open to customer participation regarding systems development, particularly with initiatives like the developer network and idea exchange. perhaps the most credible example is in a june , press release, where the company declares “support of the open discovery initiative (odi) and conformance with odi’s recommended practice for pre-indexed ‘web-scale’ discovery services.” a key implication is that “conforming to odi regulations about ranking of search results, linking to content, inclusion of materials in primo central, and discovery of open access content all uphold the principles of content neutrality.” given the above information, in the case of the knowledge center, it is tempting to give ex libris the benefit of the doubt. as an access services librarian, i understand how much of a hassle it can be to find and obtain systems documentation in order to properly do my job. i currently work for an ex libris institution, and can affirm that the knowledge center is of tangible benefit. besides providing easier availability for their materials, ex libris has done fairly well in keeping information and pathing up to date. notably, as of last month, customers can also contribute their own documentation to product-specific community knowledge sections within the knowledge center. nevertheless, this does not change the fact that while the knowledge center is unique in its format, it represents a low bar to clear for a company of ex libris’s size. their systems documentation should be openly accessible in any case. moreover, the knowledge center represents openness—in the form of company transparency and customer participation—for systems and products that are not open. this is why when we go back to the knowledge center press release, we can identify it as orangewashing. open access is not the point of a profit-driven company offering freely accessible documentation, and any claims to this effect ultimately ring hollow. so what is the likely point of the knowledge center, then? we should consider that alma has become the predominant service platform within academic libraries, with primo and summon being the only supported discovery layers for it. while oclc and ebsco offer or support competing products, ex libris already held an advantageous position even before the proquest purchase. therefore, besides the knowledge center serving as supportive measure for current customers, we can view it as a sales pitch to future ones. this may be a smart business strategy, but again, it has little to do with open access. two other recent developments provide further evidence of ex libris’s orangewashing. the first is mla’s announcement that ebsco will become the exclusive vendor for the mla international bibliography. on the primo-l listserv, ex libris posted a statement [listserv subscription required] noting that the agreement “goes against the goals of niso’s open discovery initiative…to promote collaboration and transparency among content and discovery providers.” nevertheless, despite not being involved in the agreement, ex libris shares some blame given the long-standing difficulty over ebsco not providing content to the primo central index. as a result, what may occur is the “siloing” of an indispensable research database, while ex libris customers remain dependent on the company to help determine an eventual route to access. secondly, in addition to offering research publications through proquest and discovery service through primo/summon, ex libris now provides end-to-end content management through esploro. monetizing more aspects of the research process is certainly far from unusual among academic publishers and service providers. elsevier arguably provides the most egregious example, and as lisa janicke hinchliffe notes, their pattern of recent acquisitions belies an apparent goal of creating a vertical stack service model for publication services. in considering what elsevier is doing, it is unsurprising—from a business standpoint—for ex libris and proquest to pursue profits in a similar manner. that said, we should bear in mind that libraries are already losing control over open access as a consequence of the general strategy that elsevier is employing. esploro will likely benefit from having strong library development partners and “open” customer feedback, but the potential end result could place its customers in a more financially disadvantageous and less autonomous position. this is simply antithetical to open access. over the past few years, ex libris has done well not just in their product development, but also their customer support. making the knowledge center “open to all” in late was a very positive step forward. yet the company’s decision to orangewash through claiming support for open access as part of a product unveiling still warrants critique. peter suber reminds us that open access is a “revolutionary kind of access”—one that is “unencumbered by a motive of financial gain.” while ex libris can perhaps talk about openness with a little more credibility than their competitors, their bottom line is still what really matters. author chris martinposted on september , september , categories open access, scholarly communication managing ils updates we’ve done a few screencasts in the past here at techconnect and i wanted to make a new one to cover a topic that’s come up this summer: managing ils updates. integrated library systems are huge, unwieldy pieces of software and it can be difficult to track what changes with each update: new settings are introduced, behaviors change, bugs are (hopefully) fixed. the video belows shows my approach to managing this process and keeping track of ongoing issues with our koha ils. author eric phetteplaceposted on august , august , categories library blockchain: merits, issues, and suggestions for compelling use cases blockchain holds a great potential for both innovation and disruption. the adoption of blockchain also poses certain risks, and those risks will need to be addressed and mitigated before blockchain becomes mainstream. a lot of people have heard of blockchain at this point. but many are unfamiliar with how this new technology exactly works and unsure about under which circumstances or on what conditions it may be useful to libraries. in this post, i will provide a brief overview of the merits and the issues of blockchain. i will also make some suggestions for compelling use cases of blockchain at the end of this post. what blockchain accomplishes blockchain is the technology that underpins a well-known decentralized cryptocurrency, bitcoin. to simply put, blockchain is a kind of distributed digital ledger on a peer-to-peer (p p) network, in which records are confirmed and encrypted. blockchain records and keeps data in the original state in a secure and tamper-proof manner[ ] by its technical implementation alone, thereby obviating the need for a third-party authority to guarantee the authenticity of the data. records in blockchain are stored in multiple ledgers in a distributed network instead of one central location. this prevents a single point of failure and secures records by protecting them from potential damage or loss. blocks in each blockchain ledger are chained to one another by the mechanism called ‘proof of work.’ (for those familiar with a version control system such as git, a blockchain ledger can be thought of as something similar to a p p hosted git repository that allows sequential commits only.[ ]) this makes records in a block immutable and irreversible, that is, tamper-proof. in areas where the authenticity and security of records is of paramount importance, such as electronic health records, digital identity authentication/authorization, digital rights management, historic records that may be contested or challenged due to the vested interests of certain groups, and digital provenance to name a few, blockchain can lead to efficiency, convenience, and cost savings. for example, with blockchain implemented in banking, one will be able to transfer funds across different countries without going through banks.[ ] this can drastically lower the fees involved, and the transaction will take effect much more quickly, if not immediately. similarly, adopted in real estate transactions, blockchain can make the process of buying and selling a property more straightforward and efficient, saving time and money.[ ] disruptive potential of blockchain the disruptive potential of blockchain lies in its aforementioned ability to render the role of a third-party authority obsolete, which records and validates transactions and guarantees their authenticity, should a dispute arise. in this respect, blockchain can serve as an alternative trust protocol that decentralizes traditional authorities. since blockchain achieves this by public key cryptography, however, if one loses one’s own personal key to the blockchain ledger holding one’s financial or real estate asset, for example, then that will result in the permanent loss of such asset. with the third-party authority gone, there will be no institution to step in and remedy the situation. issues this is only some of the issues with blockchain. other issues include (a) interoperability between different blockchain systems, (b) scalability of blockchain at a global scale with large amount of data, (c) potential security issues such as the % attack [ ], and (d) huge energy consumption [ ] that a blockchain requires to add a block to a ledger. note that the last issue of energy consumption has both environmental and economic ramifications because it can cancel out the cost savings gained from eliminating a third-party authority and related processes and fees. challenges for wider adoption there are growing interests in blockchain among information professionals, but there are also some obstacles to those interests gaining momentum and moving further towards wider trial and adoption. one obstacle is the lack of general understanding about blockchain in a larger audience of information professionals. due to its original association with bitcoin, many mistake blockchain for cryptocurrency. another obstacle is technical. the use of blockchain requires setting up and running a node in a blockchain network, such as ethereum[ ], which may be daunting to those who are not tech-savvy. this makes a barrier to entry high to those who are not familiar with command line scripting and yet still want to try out and test how a blockchain functions. the last and most important obstacle is the lack of compelling use cases for libraries, archives, and museums. to many, blockchain is an interesting new technology. but even many blockchain enthusiasts are skeptical of its practical benefits at this point when all associated costs are considered. of course, this is not an insurmountable obstacle. the more people get familiar with blockchain, the more ways people will discover to use blockchain in the information profession that are uniquely beneficial for specific purposes. suggestions for compelling use cases of blockchain in order to determine what may make a compelling use case of blockchain, the information profession would benefit from considering the following. (a) what kind of data/records (or the series thereof) must be stored and preserved exactly the way they were created. (b) what kind of information is at great risk to be altered and compromised by changing circumstances. (c) what type of interactions may need to take place between such data/records and their users.[ ] (d) how much would be a reasonable cost for implementation. these will help connecting the potential benefits of blockchain with real-world use cases and take the information profession one step closer to its wider testing and adoption. to those further interested in blockchain and libraries, i recommend the recordings from the library . online mini-conference, “blockchain applied: impact on the information profession,” held back in june. the blockchain national forum, which is funded by imls and is to take place in san jose, ca on august th, will also be livestreamed. notes [ ] for an excellent introduction to blockchain, see “the great chain of being sure about things,” the economist, october , , https://www.economist.com/news/briefing/ -technology-behind-bitcoin-lets-people-who-do-not-know-or-trust-each-other-build-dependable. [ ] justin ramos, “blockchain: under the hood,” thoughtworks (blog), august , , https://www.thoughtworks.com/insights/blog/blockchain-under-hood. [ ] the world food programme, the food-assistance branch of the united nations, is using blockchain to increase their humanitarian aid to refugees. blockchain may possibly be used for not only financial transactions but also the identity verification for refugees. russ juskalian, “inside the jordan refugee camp that runs on blockchain,” mit technology review, april , , https://www.technologyreview.com/s/ /inside-the-jordan-refugee-camp-that-runs-on-blockchain/. [ ] joanne cleaver, “could blockchain technology transform homebuying in cook county — and beyond?,” chicago tribune, july , , http://www.chicagotribune.com/classified/realestate/ct-re- -blockchain-homebuying- -story.html. [ ] “ % attack,” investopedia, september , , https://www.investopedia.com/terms/ / -attack.asp. [ ] sherman lee, “bitcoin’s energy consumption can power an entire country — but eos is trying to fix that,” forbes, april , , https://www.forbes.com/sites/shermanlee/ / / /bitcoins-energy-consumption-can-power-an-entire-country-but-eos-is-trying-to-fix-that/# ff aa bc . [ ] osita chibuike, “how to setup an ethereum node,” the practical dev, may , , https://dev.to/legobox/how-to-setup-an-ethereum-node- a . [ ] the interaction can also be a self-executing program when certain conditions are met in a blockchain ledger. this is called a “smart contract.” see mike orcutt, “states that are passing laws to govern ‘smart contracts’ have no idea what they’re doing,” mit technology review, march , , https://www.technologyreview.com/s/ /states-that-are-passing-laws-to-govern-smart-contracts-have-no-idea-what-theyre-doing/. author bohyun kimposted on july , july , categories coding, data, technologytags bitcoin, blockchain, distributed ledger technology comment on blockchain: merits, issues, and suggestions for compelling use cases introducing our new best friend, gdpr you’ve seen the letters gdpr in every single email you’ve gotten from a vendor or a mailing list lately, but you might not be exactly sure what it is. with gdpr enforcement starting on may , it’s time for a crash course in what gdpr is, and why it could be your new best friend whether you are in the eu or not. first, you can check out the eu gdpr information site (though it probably will be under heavy load for a few days!) for lots of information on this. it’s important to recognize, however, that for universities like mine with a campus located in the eu, it has created additional oversight to ensure that our own data collection practices are gdpr compliant, or that we restrict people residing in the eu from accessing those services. you should definitely work with legal counsel on your own campus in making any decisions about gdpr compliance. so what does the gdpr actually mean in practice? the requirements break down this way: any company which holds the data of any eu citizen must provide data controls, no matter where the company or the data is located. this means that every large web platform and pretty much every library vendor must comply or face heavy fines. the gdpr offers the following protections for personally identifiable information, which includes things like ip address: privacy terms and conditions must be written in easy to understand language, data breaches require quick notifications, the right to know what data is being collected and to receive a copy of it, the “right to be forgotten” or data erasure (unless it’s in the public interest for the data to be retained), ability to transfer data between providers, systems to be private by design and only collect necessary data, and for companies to appoint data privacy officers without conflicts of interest. how this all works in practice is not consistent, and there will be a lot to be worked out in the courts in the coming years. note that google recently lost several right to be forgotten cases, and were required to remove information that they had originally stated was in the public interest to retain. the gdpr has actually been around for a few years, but may , was set as the enforcement date, so many people have been scrambling to meet that deadline. if you’re reading this today, there’s probably not a lot of time to do anything about your own practices, but if you haven’t yet reviewed what your vendors are doing, this would be a good time. note too that there are no rights guaranteed for any americans, and several companies, including facebook, have moved data governance out of their irish office to california to be out of reach of suits brought in irish courts. where possible, however, we should be using all the features at our disposal. as librarians, we already tend to the “privacy by design” philosophy, even though we aren’t always perfect at it. as i wrote in my last post, my library worked on auditing our practices and creating a new privacy policy, and one of the last issues was trying to figure out how we would approach some of our third-party services which we need to provide services to our patrons but that did not allow deleting data. now some of those features are being made available. for example, google analytics now has a data retention feature, which allows you to set data to expire and be deleted after a certain amount of time. google provides some more detailed instructions to ensure that you are not accidentally collecting personally-identifiable information in your analytics data. lots of our library vendors provide personal account features, and those too are subject to these new gdpr features. this means that there are new levels of transparency about what kinds of tracking they are doing, and greater ability for patrons to control data, and for you to control data on the behalf of patrons. here are a few example vendor gdpr compliance statements or faqs: ebsco ex libris proquest springshare note that some vendors, like ebsco, are moving to https for all sites that weren’t before, and so this may require changes to proxy servers or other links. i am excited about gdpr because no matter where we are located, it gives us new tools to defend the privacy of our patrons. even better than that, it is providing lots of opportunities on our campuses to talk about privacy with all stakeholders. at my institution, the library has been able to showcase our privacy expertise and have some good conversations about data governance and future goals for privacy. it doesn’t mean that all our problems will be solved, but we are moving in a more positive direction. author margaret hellerposted on may , may , categories administration, privacytags gdpr names are hard a while ago i stumbled onto the post “falsehoods programmers believe about names” and was stunned. personal names are one of the most deceptively difficult forms of data to work with and this article touched on so many common but unaddressed problems. assumptions like “people have exactly one canonical name” and “my system will never have to deal with names from china/japan/korea” were apparent everywhere. i consider myself a fairly critical and studious person, i devote time to thinking about the consequences of design decisions and carefully attempt to avoid poor assumptions. but i’ve repeatedly run into trouble when handling personal names as data. there is a cognitive dissonance surrounding names; we treat them as rigid identifiers when they’re anything but. we acknowledge their importance but struggle to take them as seriously. names change. they change due to marriage, divorce, child custody, adoption, gender identity, religious devotion, performance art, witness protection, or none of these at all. sometimes people just want a new name. and none of these reasons for change are more or less valid than others, though our legal system doesn’t always treat them equally. we have students who change their legal name, which is often something systems expect, but then they have the audacity to want to change their username, too! and that works less often because all sorts of system integrations expect usernames to be persistent. names do not have a universal structure. there is no set quantity of components in a name nor an established order to those components. at my college, we have students without surnames. in almost all our systems, surname is a required field, so we put a period “.” there to satisfy that requirement. then, on displays in our digital repository where surnames are assumed, we end up with bolded section headers like “., johnathan” which look awkward. many western names might follow a [given name] – [middle name] – [surname] structure and an unfortunate number of the systems i have to deal with assume all names share this structure. it’s easy to see how this yields problematic results. for instance, if you want to a see a sorted list of users, you probably want to sort by family name, but many systems sort by the name in the last position causing, for instance, chinese names to be handled differently from western ones. but it’s not only that someone might not have a middle name, or might have two middle names, or might have a family name in the first position—no, even that would be too simple! some name components defy simple classifications. i once met a person named “bus stop”. “stop” is clearly not a family affiliation, despite coming in the final position of the name. sometimes the second component of a tripartite western name isn’t a middle name at all, but a maiden name or the second word of a two-word first name (e.g. “mary anne” or “lady bird”)! one cannot even determine by looking at a familiar structure the roles of all of a name’s pieces! names are also contextual. one’s name with family, with legal institutions, and with classmates can all differ. many of our international students have alternative westernized first names. their family may call them qiáng but they introduce themselves as brian in class. we ask for a “preferred name” in a lot of systems, which is a nice step forward, but don’t ask when it’s preferred. names might be meant for different situations. we have no system remotely ready for this, despite the personalization that’s been seeping into web platforms for decades. so if names are such a trouble, why not do our best and move on? aren’t these fringe cases that don’t affect the vast majority of our users? these issues simply cannot be ignored because names are vital. what one is called, even if it’s not a stable identifier, has great effects on one’s life. it’s dispiriting to witness one’s name misspelled, mispronounced, treated as an inconvenience, botched at every turn. a system that won’t adapt to suit a name delegitimizes the name. it says, “oh that’s not your real name” as if names had differing degrees of reality. but a person may have multiple names—or many overlapping names over time—and while one may be more institutionally recognized at a given time, none are less real than the others. if even a single student a year is affected, it’s the absolute least amount of respect we can show to affirm their name(s). so what do we to do? endless enumerations of the difficulties of working with names does little but paralyze us. honestly, when i consider about the best implementation of personal names, the mods metadata schema comes to mind. having a element with any number of children is the best model available. the s can be ordered in particular ways, a “@type” attribute can define a part’s function , a record can include multiple names referencing the same person, multiple names with distinct parts can be linked to the same authority record, etc. mods has a flexible and comprehensive treatment of name data. unfortunately, returning to “falsehoods programmers believe”, none of the library systems i administer do anywhere near as good a job as this metadata schema. nor is it necessarily a problem with western bias—even the chinese government can’t develop computer systems to accurately represent the names of people in the country, or even agree on what the legal character set should be! it seems that programmers start their apps by creating a “users” database table with columns for unique identifier, username, “firstname”/”lastname” [sic], and work from there. on the bright side, the name isn’t used as the identifier at least! we all learned that in databases class but we didn’t learn to make “names” a separate table linked to “users” in our relational databases. in my day-to-day work, the best i’ve done is to be sensitive to the importance of names changes specifically and how our systems handle them. after a few meetings with a cross-departmental team, we developed a name change process at our college. system administrators from across the institution are on a shared listserv where name changes are announced. in the libraries, i spoke with our frontline service staff about assisting with name changes. our people at the circulation desk know to notice name discrepancies—sometimes a name badge has been updated but not our catalog records, we can offer to make them match—but also to guide students who may need to contact the registrar or other departments on campus to initiate the top-down name change process. while most of our the library’s systems don’t easily accommodate username changes, i can write administrative scripts for our institutional repository that alter the ownership of a set of items from an old username to a new one. i think it’s important to remember that we’re inconveniencing the user with the work of implementing their name change and not the other way around. so taking whatever extra steps we can do on our own, without pushing labor onto our students and staff, is the best way we can mitigate how poorly our tools are able to support the protean nature of personal names. notes chinese names typically have the surname first, followed by the given name. ↩ another poor implementation can be seen in the chicago manual of style‘s indexing instructions, which has an extensive list of exceptions to the western norm and how to handle them. but cmos provides no guidance on how one would go about identifying a name’s cultural background or, for instance, identifying a compound surname. ↩ although the mods user guidelines sadly limit the use of the type attribute to a fixed list of values which includes “family” and “given”, rendering it subject to most of the critiques in this post. substantially expanding this list with “maiden”, “patronymic/matronymic” (names based on a parental given name, e.g. mikhailovich), and more, as well as some sort of open-ended “other” option, would be a great improvement. ↩ https://www.nytimes.com/ / / /world/asia/ china.html ↩ author eric phetteplaceposted on may , may , categories change, data, diversity comments on names are hard posts navigation page page … page next page search for: search about acrl techconnect is a moderated blog written by librarians and archivists covering innovative projects, emerging tech tools, coding, usability, design, and more. acrl techconnect serves as your source for technology-related content from the association of college and research libraries, a division of the american library association, and c&rl news magazine. cc-by-nc-nd this work is licensed under a creative commons attribution-noncommercial-noderivs . unported license. based on a work at acrl.ala.org/techconnect. recent posts broken links in the discovery layer—pt. ii: towards an ethnography of broken links broken links in the discovery layer—pt. i: researching a problem orcid for system interoperability in scholarly communication workflows creating presentations with beautiful.ai national forum on web privacy and web analytics acrl technology resources articles, books, and podcasts c&rl news column communities and groups categories academic librarianship accessibility administration algorithms api book review books careers change coding conferences continuing education copyright data design digital libraries digital scholarship digitization discovery diversity hacking ils information architecture library library as makerspace library design library instruction linked-data management marketing metadata mobile open access pedagogy planning presentation privacy publication reference scholarly communication social media technology the setup tools tutorial uncategorized usability use study version control web what-if workflow writing about authors acrl techconnect proudly powered by wordpress bethany nowviskie bethany nowviskie foreword (to the past) congratulations to melissa terras and paul gooding on the publication of an important new collection of essays entitled electronic legal... a pledge: self-examination and concrete action in the jmu libraries “the beauty of anti-racism is that you don’t have to pretend to be free of racism to be an anti-racist.... change us, too [the following is a brief talk i gave at the opening plenary of rbms , a meeting of the rare... from the grass roots [this is a cleaned-up version of the text from which i spoke at the conference of research libraries uk,... how the light gets in i took a chance on a hackberry bowl at a farmer’s market—blue-stained and turned like a drop of water. it’s... reconstitute the world [what follows is the text of a talk i gave in two different contexts last week, as &# ;reconstitute the world:... spectra for speculative knowledge design [last weekend, i joined the inspiring, interdisciplinary ecotopian toolkit gathering hosted by penn&# ;s program in environmental humanities. (how lucky was i? we even got... we raise our voices [crossposted statement on us administration budget proposal from the &# ;director&# ;s desk&# ; at the digital library federation blog.] last night, the... iv. coda: speculative computing ( ) [shannon mattern&# ;s wry observation that &# ;speculative now seems to be the universal prefix&# ; got me thinking about time and unpredictability, and reminded me... inauguration day january th has inaugurated the worst and longest case of writer’s block of my life. i hate to write, under... open invitations [these are unedited remarks from the closing plenary of the dlf forum, written about minutes before it began,... speculative collections [this is the text of a talk i gave last week, as &# ;speculative collections and the emancipatory library,&# ; to close... alternate futures/usable pasts [while i&# ;m cleaning up the text of a talk i gave at harvard&# ;s hazen symposium last week (see #hazenatharvard or merrilee&# ;s... everywhere, every when this is the text of a presentation i made yesterday at a wonderful columbia university symposium called insuetude (still ongoing),... capacity through care [this is the draft of an invited contribution to a forum on &# ;care&# ; that will appear in debates in the... hallowmas [trigger warning: miscarriage.] ten years ago today, i lost the baby that might have come after my son, and not... on capacity and care [this is the blended and edited text of two talks i gave last week. one, titled &# ;on capacity and care,&# ;... supporting practice in community [here&# ;s a cleaned-up version of brief remarks i made in a panel discussion on &# ;cultivating digital library professionals,&# ; at tuesday&# ;s... a game nonetheless [i recently had the pleasure of responding to a creative and beautifully grounded talk by kevin hamilton of the university... open and shut i recently collaborated on a project a little outside the ordinary for me: a case study for a chapter in... all at once thirteen years ago, i was a graduate student in english literature when the twin towers collapsed, a fireball erupted from... charter-ing a path [cross-posted from the re:thinking blog at clir, the council on library and information resources, where i&# ;m honored to serve as... speculative computing & the centers to come [this is a short talk i prepared for a panel discussion today with brett bobley, ed ayers, and stephen robertson,... johannes factotum & the ends of expertise [this—more or less—is the text of a keynote talk i delivered last week in atlanta, at the dlf forum:... neatline & visualization as interpretation [this post is re-published from an invited response to a february mediacommons question of the week: &# ;how can we better... a kit for hosting speaking in code [cross-posted from the re:thinking blog at clir, the council on library and information resources, where i&# ;m honored to be serving... digital humanities in the anthropocene [update: i&# ;ve made low-res versions of my slides and an audio reading available for download on vimeo, alex gil has... anthropocene abstract i am deeply honored to have been invited to give a plenary lecture at this year&# ;s digital humanities conference, planned... asking for it a report published this week by oclc research asks the burning question of no one, no where: &# ;does every research... on the origin of “hack” and “yack” one of the least helpful constructs of our “digital humanities” moment has been a supposed active opposition, drawn out over... in the library with the lead pipe in the library with the lead pipe an open access, peer reviewed journal culturally responsive community engagement programming and the university library: lessons learned from half a decade of vtditc by craig e. arthur, dr. freddy paige, la’ portia perkins, jasmine weiss, and dr. michael williams (good homie signs’ “hip hop @ vt” mural / ) in brief vtditc: hip hop studies at virginia tech is an award-winning series of experiential learning-focused, culturally responsive community engagement programs. it is deeply rooted in hip hop culture and... read more creating a student-centered alternative to research guides: developing the infrastructure to support novice learners in brief: research and course guides typically feature long lists of resources without the contextual or instructional framework to direct novice researchers through the research process. an investigation of guide usage and user interactions at a large university in the southwestern u.s. revealed a need to reexamine the way research guides can be developed and... read more power and status (and lack thereof) in academe: academic freedom and academic librarians in brief academic librarians do not experience full academic freedom protections, despite the fact that they are expected to exercise independent judgment, be civically engaged, and practice applied scholarship. academic freedom for academic librarians is not widely studied or well understood. to learn more, we conducted a survey which received over responses from academic... read more the library commons: an imagination and an invocation by jennie rose halperin in brief commons theory can provide important interventions within neoliberal managerial information capitalism when applied to the library as an institution. the commons and its associated practices provide a model of abundance, sharing, and cooperation. libraries can and should participate in alternative economic and management models to create an inclusive vision... read more “information has value”: the political economy of information capitalism in brief information capitalism dominates the production and flow of information across the globe. it produces massive information institutions that are as harmful to everyday people as they are powerful. to this point, information literacy (il) educators do not have a theory and pedagogy of information capitalism. this article appraises the current state of political... read more training matters: student employment and learning in academic libraries in brief conceiving of student employment in academic libraries as an educationally purposeful experience requires adopting a learner-centered pedagogical approach to student employee job training. adopting such an approach is triply beneficial: it makes that job training more effective; it identifies training as an opportunity to pursue learning goals that support the growth of students... read more creating a library wide culture and environment to support mlis students of color: the diversity scholars program at oregon state university libraries in brief the work of social justice, equity, and inclusion is not a short-term investment by a limited number of people; instead, it should be a part of every library’s and librarian’s work. at the oregon state university libraries (osul), we felt that in order to create a program dedicated to employing mlis students of... read more it’s not imposter syndrome: resisting self-doubt as normal for library workers in brief library workers, as with other professions, are quick to diagnose ourselves and others with imposter syndrome when we doubt or devalue our everyday work.&# ; however, methods of coping with imposter syndrome have changed little in the forty years since the term was first theorized, and often centre on feel-good fixes which do not... read more multilingualism, neoliberalism, and language ideologies in libraries in briefthis article calls for a more holistic and inclusive approach to the under-examined issue of language in libraries. it begins by foregrounding language as a category of difference and arguing for its consideration in discussions of access, equity, diversity, and inclusion. by drawing on literature from applied linguistics and library and information studies, it... read more communicating with information: creating inclusive learning environments for students with asd in brief the focus of this article is twofold: it ) considers how digital humanities techniques and methodologies increase accessibility and scholarship opportunities for students with autism spectrum disorder; and ) outlines how libraries can collaborate with existing services to provide subsequently appropriate supports for students. autism spectrum disorder (asd), one of the increasingly prevalent... read more google forms - create and analyze surveys, for free. one account. all of google. sign in to continue to forms enter your email find my account sign in with a different account create account one google account for everything google about google privacy terms help google forms - create and analyze surveys, for free. one account. all of google. sign in to continue to forms enter your email find my account sign in with a different account create account one google account for everything google about google privacy terms help none what i learned today… skip to the content search what i learned today... menu about me publications & presentations library mashups the accidental systems librarian open source software for libraries my presenting/learning calendar blog archives search search for: close search close menu about me publications & presentationsshow sub menu library mashups the accidental systems librarian open source software for libraries my presenting/learning calendar blog archives facebook twitter linkedin categories about me taking a break post author by nicole c. baratta post date may , no comments on taking a break i’m sure those of you who are still reading have noticed that i haven’t been updating this site much in the past few years. i was sharing my links with you all but now delicious has started adding ads to that. i’m going to rethink how i can use this site effectively going forward. for now you can read my regular content on opensource.com at https://opensource.com/users/nengard. share this: email twitter facebook tumblr linkedin reddit pocket pinterest categories link sharing bookmarks for may , post author by nicole c. baratta post date may , no comments on bookmarks for may , today i found the following resources and bookmarked them on delicious. start a fire grow and expand your audience by recommending your content within any link you share digest powered by rss digest share this: email twitter facebook tumblr linkedin reddit pocket pinterest categories link sharing bookmarks for april , post author by nicole c. baratta post date april , no comments on bookmarks for april , today i found the following resources and bookmarked them on delicious. mattermost mattermost is an open source, self-hosted slack-alternative mblock program your app, arduino projects and robots by dragging & dropping fidus writer fidus writer is an online collaborative editor especially made for academics who need to use citations and/or formulas. beek social network for booklovers open ebooks open ebooks is a partnership between digital public library of america, the new york public library, and first book, with content support from digital books distributor baker & taylor. digest powered by rss digest share this: email twitter facebook tumblr linkedin reddit pocket pinterest categories link sharing bookmarks for february , post author by nicole c. baratta post date february , no comments on bookmarks for february , today i found the following resources and bookmarked them on delicious. connfa open source ios & android app for conferences & events paperless scan, index, and archive all of your paper documents foss serve foss serve promotes student learning via participation in humanitarian free and open source software (foss) projects. disk inventory x disk inventory x is a disk usage utility for mac os x . (and later). it shows the sizes of files and folders in a special graphical way called “treemaps”. loomio loomio is the easiest way to make decisions together. loomio empowers organisations and communities to turn discussion into action, wherever people are. democracyos democracyos is an online space for deliberation and voting on political proposals. it is a platform for a more open and participatory government. the software aims to stimulate better arguments and come to better rulings, as peers. digest powered by rss digest share this: email twitter facebook tumblr linkedin reddit pocket pinterest categories link sharing bookmarks for january , post author by nicole c. baratta post date january , no comments on bookmarks for january , today i found the following resources and bookmarked them on delicious. superpowers the open source, extensible, collaborative html d+ d game maker sequel pro sequel pro is a fast, easy-to-use mac database management application for working with mysql databases. digest powered by rss digest share this: email twitter facebook tumblr linkedin reddit pocket pinterest categories link sharing bookmarks for december , post author by nicole c. baratta post date december , no comments on bookmarks for december , today i found the following resources and bookmarked them on delicious. open broadcaster software free, open source software for live streaming and recording digest powered by rss digest share this: email twitter facebook tumblr linkedin reddit pocket pinterest categories link sharing bookmarks for november , post author by nicole c. baratta post date november , no comments on bookmarks for november , today i found the following resources and bookmarked them on delicious. numfocus foundation numfocus promotes and supports the ongoing research and development of open-source computing tools through educational, community, and public channels. digest powered by rss digest share this: email twitter facebook tumblr linkedin reddit pocket pinterest categories link sharing bookmarks for november , post author by nicole c. baratta post date november , no comments on bookmarks for november , today i found the following resources and bookmarked them on delicious. smore smore makes it easy to design beautiful and effective online flyers and newsletters. ninite install and update all your programs at once digest powered by rss digest share this: email twitter facebook tumblr linkedin reddit pocket pinterest categories link sharing bookmarks for november , post author by nicole c. baratta post date november , no comments on bookmarks for november , today i found the following resources and bookmarked them on delicious. vim adventures learning vim while playing a game digest powered by rss digest share this: email twitter facebook tumblr linkedin reddit pocket pinterest categories link sharing bookmarks for november , post author by nicole c. baratta post date november , no comments on bookmarks for november , today i found the following resources and bookmarked them on delicious. star wars: building a galaxy with code digest powered by rss digest share this: email twitter facebook tumblr linkedin reddit pocket pinterest posts navigation ← newer posts … older posts → facebook twitter linkedin search for: tags amazon android ato blogging chrome cil cil cil cil cil code lib facebook feedburner firefox gmail google il il il il il koha kohacon kohacon kohacon libraries mapping nfais njla open source oscon oscon php pinterest rss sla sla special libraries association sxsw twitter valenj webinar windows wordpress zotero learn library mashups learn systems librarianship learn open source my sites library mashups practical open source for libraries the accidental systems librarian © what i learned today… powered by wordpress to the top ↑ up ↑ send to email address your name your email address cancel post was not sent - check your email addresses! email check failed, please try again sorry, your blog cannot share posts by email. q / dlg update - google docs javascript isn't enabled in your browser, so this file can't be opened. enable and reload. some word features can't be displayed in google docs and will be dropped if you make changes view details q / dlg update share sign in the version of the browser you are using is no longer supported. please upgrade to a supported browser.dismiss file edit view tools help accessibility debug see new changes bethany nowviskie skip to content bethany nowviskie menu bio minor arcana jmu libraries cv search for: search reconstitute the world speculative collections on capacity and care foreword (to the past) posted on october october by bethany nowviskie congratulations to melissa terras and paul gooding on the publication of an important new collection of essays entitled electronic legal deposit: shaping the library collections of the future! this volume takes a global outlook on challenges and successes in preserving digital information, and stems from their digital library futures ahrc project, which first analyzed the impact of electronic legal deposit legislation on academic libraries and their users in the uk. more from melissa here, including “an ark to save learning from deluge? reconceptualising legal deposit after the digital turn,” an oa version of the opening chapter she & paul contributed to the collection. i was honored to be asked to write a foreword to the book, which i share here, under facet publishing’s green oa agreement, as my own author’s last copy of a single chapter from an edited collection. i thought i’d post it, particularly, now — as next week not only marks world digital preservation day, but another highly significant election day in the united states. we are four years on from the moment i describe below… on the morning of november th, , i looked out over a milwaukee ballroom crowded with librarians, archivists, and specialists in digital preservation. some were pensive. many were weeping. others seemed stricken. my audience had gathered for the first joint conference of the digital library federation (dlf, the us-based nonprofit organization i then directed) with its new partner, the national digital stewardship alliance (ndsa)—a cross-industry group that had recently come under dlf’s wing from its place of genesis at the library of congress. we were strangers and friends, largely though not exclusively american, united in a community of practice and the common cause of a dedication to the future of libraries, archives, and their holdings and information services in the digital age. but it suddenly felt as if we didn’t know what information was, and whether—despite all our efforts, expertise, and the shared infrastructure that our memory institutions represented—its future could be made secure. the unexpected outcome of the us presidential election, announced in the wee hours the night before, had cast a pall over this professional audience that crossed party lines. how could so many confident, data-driven predictions have been so wrong? what shared social understandings—built from the seeming common landscape of ubiquitous digital information that we had met to manage and survey—had never, in fact, been shared or were even commonly legible at all? and what evidentiary traces of this time would remain, in a political scene of post-truth posturing, the devaluation of expert knowledge, and the willingness of our new authorities—soon to become as evident on federal websites as in press conferences and cable news punditry—to revise and resubmit the historical record? the weeks and months that followed, for dlf and ndsa members, were filled with action. while the end of term web archive project sprang to its regular work of harvesting us federal domains at moments of presidential transition, reports that trump administration officials had ordered the removal of information on climate change and animal welfare from the websites of the environmental protection agency and us department of agriculture fostered a fear of the widespread deletion of scientific records, and prompted emergency ‘data rescue’ download parties. a new dlf government records transparency and accountability working group was launched. its members began watch-dogging preparations for the us census and highlighting house and senate bills meant to curtail scientific and demographic data creation; scrutinizing proposed changes to the records retention schedules of federal agencies and seeking ways to make the arcanum of their digital preservation workflows more accessible to the general public; and—amid new threats of the deportation of immigrants and the continued rise of violent nationalism—asking crucial questions about what electronic information should be made discoverable and accessible, for the protection of vulnerable persons. the social sciences research council convened a meeting on challenges to the digital preservation of documents of particular value to historians, economists, cultural anthropologists, and other social scientists, and the pegi project—focusing on the preservation of electronic government information—commissioned a wide-ranging report on at-risk, born-digital information meant to be held by us federal depository libraries and other cultural memory institutions for long-term public access and use. over time, reflective, pedagogical, and awareness-raising projects like endangered data week emerged, ties among the ndsa and international organizations like the uk-based digital preservation coalition were strengthened, and conversations on college campuses (fueled by the cambridge analytica scandal and the work of scholars of race, technology, and social media like safiya noble and siva vaidhyanathan) turned more squarely to data ethics and algorithmic literacy. frenetic data rescue parties gave over to the more measured advocacy and storytelling approach of the data refuge movement. and in the uk, an ahrc-funded ‘digital library futures’ project led by paul gooding and melissa terras (the seed of this edited collection) offered a golden opportunity to reflect—in the light of altered global understandings of the preservation and access challenges surrounding digital information—on the parliamentary legal deposit libraries (non print works) regulations of , which extended collecting practices dating to the early modern period to new media formats beyond the book. you hold in your hands (or view on your screens, or listen to through e-readers, or encounter in some other way i can’t yet foresee) an important and timely volume. it is well balanced between reflection-and-outlook and practice-and-method in what our editors call the ‘contested space’ of e-legal deposit—taking on the international and very long-term consequences of our present-day conception, regulation, assembly, positioning, and use of library-held digital collections. in other words, the essays assembled here cross space and time. the editors take a necessarily global view in bringing together a broad array of national approaches to the legal deposit of materials that already circulate in world-wide networks. and while the authors they’ve invited to contribute certainly take a long view of digital information, they also frequently address, head-on, the ways that electronic legal deposit forces our attention not just on posterity, but on the here-and-now of what media consumption means and how it works in the digital age. rather than asking us to rest our imaginations on a far-future prospect in which reading is conducted as it ever was in print (was any such act, as jerome mcgann would ask, self-identical?), the authors of these essays, collectively, assert that the kaleidoscopic mediations of e-legal deposit show us we’ve never really known what reading is. the best thinkers on libraries question the very assumptions that our memory institutions rest upon, while elevating and honoring both their promise and the centuries of labor and careful (if not always disinterested or benign) intent that have made them what they are. melissa terras and paul gooding are among the best, and the perspectives they have assembled here—from publishers, eminent librarians and archivists, technologists, organizers, and scholars—make this edited collection an essential contribution to the literature on digital preservation. it is a necessary book that grapples with legal, practical, technical, and conceptual problems: with the distinctive visions and values of libraries; with the necessarily concomitant development of policies and platforms; and even with the very nature of our documentary heritage, at a moment when print-era logics break down. what i most appreciate is that this book—like the notion of e-legal deposit itself—calls for careful consideration of both present-day services and research possibilities not yet dreamt of. in this, it serves the true mission of legal deposit libraries: to be a stable bridge between a past that is perpetually constructed by our acts of preservation and erasure—and the many futures we may mediate but can barely imagine. posted in higher ed, infrastructure a pledge: self-examination and concrete action in the jmu libraries posted on june june by bethany nowviskie “the beauty of anti-racism is that you don’t have to pretend to be free of racism to be an anti-racist. anti-racism is the commitment to fight racism wherever you find it, including in yourself. and it’s the only way forward.” — ijeoma oluo, author of so you want to talk about race. black lives matter. too long have we allowed acts of racism and deeply ingrained, institutionalized forces of white supremacy to devalue, endanger, and grievously harm black people and members of other minoritized and marginalized groups. state-sanctioned violence and racial terror exist alongside slower and more deep-seated forces of inequality, anti-blackness, colonization, militarization, class warfare, and oppression. as members of the jmu libraries dean’s council and council on diversity, equity, and inclusion, we acknowledge these forces to be both national and local, shaping the daily lived experiences of our students, faculty, staff, and community members. as a blended library and educational technology organization operating within a pwi, the jmu libraries both participates in and is damaged by the whiteness and privilege of our institutions and fields. supporting the james madison university community through a global pandemic has helped us see imbalances, biases, and fault lines of inequality more clearly. we pledge self-examination and concrete action. libraries and educational technology organizations hold power, and can share or even cede it. as we strive to create welcoming spaces and services for all members of our community, we assert the fundamental non-neutrality of libraries and the necessity of taking visible and real action against the forces of racism and oppression that affect bipoc students, faculty, staff, and community members. specifically, and in order to “fight racism wherever [we] find it, including in [ourselves],” we commit to: listen to bipoc and student voices, recognizing that they have long spoken on these issues and have too often gone unheard. educate ourselves and ask questions of all the work we do. (“to what end? to whose benefit? whose comfort is centered? who has most agency and voice? who is silenced, ignored, or harmed? who is elevated, honored, and made to feel safe? who can experience and express joy?”) set public and increasingly measurable goals related to diversity, equity, inclusion, and anti-racism, so that we may be held accountable. continue to examine, revise, and augment our collections, services, policies, spending patterns, and commitments, in order to institutionalize better practices and create offerings with enduring impact. learn from, and do better by, our own colleagues. we are a predominantly white organization and it is likely that we will make mistakes as we try to live up to this pledge. when that happens, we will do the work to learn and rectify. we will apologize, examine our actions and embedded power structures, attempt to mitigate any harm caused by our actions, and we will do better. continue reading “a pledge: self-examination and concrete action in the jmu libraries” posted in higher ed change us, too posted on june may by bethany nowviskie [the following is a brief talk i gave at the opening plenary of rbms , a meeting of the rare books and manuscripts section of the acrl/ala. this year’s theme was “response and responsibility: special collections and climate change,” and my co-panelists were frances beinecke of the national resources defense council and brenda ekwurzel of the union of concerned scientists. many thanks to conference chairs ben goldman and kate hutchens, session chair melissa hubbard, and outgoing rbms chair shannon supple. the talk draws together some of my past writings, all of which are linked to and freely available. images in my slide deck, as here, were by catherine nelson.] six years ago, i began writing about cultural heritage and cultural memory in the context of our ongoing climate disaster. starting to write and talk publicly was a frank attempt to assuage my terror and my grief—my personal grief at past and coming losses in the natural world, and the sense of terror growing inside me, both at the long-term future of the digital and physical collections in my charge, and at the unplanned-for environmental hardships and accelerating social unrest my two young children, then six and nine years old, would one day face. i latched, as people trained as scholars sometimes do, onto a set of rich and varied theoretical frameworks. these were developed by others grappling with the exact same existential dread: some quite recent, some going back to the s, the s, even the s—demonstrating, for me, not just the continuity of scientific agreement on the facts of climate change and the need for collective action (as my co-panelists have demonstrated), but scholarly and artistic agreement on the generative value of responses from what would become the environmental humanities and from practices i might call green speculative design. the concepts and theories i lighted on, however, served another function. they allowed me simultaneously to elevate and to sublimate many of my hardest-hitting feelings. in other words, i put my fears into a linguistic machine labeled “the anthropocene”—engineered to extract angst and allow me to crank out historicized, lyrical melancholy on the other end. since then i’ve also become concerned that, alongside and through the explicit, theoretical frameworks i found in the literature, i leaned unconsciously—as cis-gender white women and other members of dominant groups almost inevitably do—on implicit frameworks of white supremacy, on my gender privilege, and on the settler ideologies that got us here in the first place, all of which uphold and support the kind of emotional and fundamentally self-centered response i was first disposed to make. i see more clearly now that none of this is about my own relatively vastly privileged children and well-tended collections—except insofar as both of them exist within broader networks and collectives of care, as one achingly beloved and all-too-transitory part. please don’t misunderstand me: it remains absolutely vital that we honor our attachments, and acknowledge the complexity and deep reality of our emotional responses to living through the sixth great mass extinction of life on this planet—vital to compassionate teaching and leadership, to responsible stewardship, and to defining value systems that help us become more humane in the face of problems of inhuman scale. grappling with our emotions as librarians and archivists (and as curators, conservators, collectors, community organizers, scholars, and scientists) will be a major part of the work of this conference. it is also vital to doing work that appreciates its own inner standing point, and uses its positionality to promote understanding and effect change. but i’ve felt my own orientation changing. for me, all of this is, every day, less and less about my feelings on special collections and climate change—except to the degree that those feelings drive me toward actions that have systemic impact and are consonant with a set of values we may share. so this is a brief talk that will try to walk you (for what it’s worth) along the intellectual path i’ve taken over the past six years—in the space of about sixteen minutes. continue reading “change us, too” posted in design, infrastructuretagged embodied from the grass roots posted on march june by bethany nowviskie [this is a cleaned-up version of the text from which i spoke at the conference of research libraries uk, held at the wellcome collection in london last week. i’d like to thank my wonderful hosts for an opportunity to reflect on my time at dlf. as i said to the crowd, i hope the talk offers some useful—or at least productively vexing—ideas.] at a meeting in which the status of libraries as “neutral spaces” has been asserted and lauded, i feel obligated to confess: i’m not a believer in dispassionate and disinterested neutrality—not for human beings nor for the institutions that we continually reinforce or reinvent, based on our interactions in and through them. my training as a humanities scholar has shown me all the ways that it is in fact impossible for us to step wholly out of our multiple, layered, subjective positions, interpretive frameworks, and embodied existence. it has also taught me the dangers of assuming—no matter how noble our intentions—that socially constructed institutions might likewise escape their historical and contemporary positioning, and somehow operate as neutral actors in neutral space. happily, we don’t need neutrality to move constructively from independent points of view to shared understandings and collective action. there are models for this. the ones i will focus on today are broadly “dh-adjacent,” and they depend, sometimes uncomfortably, on the vulnerability, subjectivity, and autonomy of the people who engage with them—foregrounding the ways that individual professional roles intersect with personal lives as they come together around shared missions and goals. and as i discuss them, please note that i’ll be referring to the digital humanities and to digital librarianship somewhat loosely—in their cultural lineaments—speaking to the diffuse and socially constructed way both are practiced on the ground. in particular, i’ll reference a dh that is (for my purposes today) relatively unconcerned with technologies, methods, and objects of study. it’s my hope that shifting our focus—after much fruitful discussion, this week, of concrete research support—to a digital humanities that can also be understood as organizational, positional, and intersubjective might prompt some structural attunement to new ways of working in libraries. and i do this here, at a consortial gathering of “the most significant research libraries in the uk and ireland,” because i think that self-consciously expanding our attention in library leadership from the pragmatic provision of data, platforms, skills-teaching, and research support for dh, outward to its larger organizational frame is one way of cracking open serious and opportune contributions by people who would not consider themselves digital humanists at all. this likely includes many of you, your colleagues in university administration across areas and functions, and most members of your libraries’ personnel. such a change in focus invites all of us to be attentive to the deeper and fundamentally different kinds of engagement and transformation we might foster through dh as a vector and perhaps with only simple re-inflections of the resources we already devote to the field. it could also open our organizations up to illuminating partnerships with communities of practice who frankly don’t give a fig about academic disciplinary labels or whether they are or are not “doing dh.” i also speak to library leaders because my call is not for work to be done by individual scholars as researchers and teachers alone, nor even by small teams of librarians laboring in support of the research and cultural heritage enterprise—but rather by our fully-engaged institutions as altered structures of power. continue reading “from the grass roots” posted in administrivia, higher edtagged community-archives, digital humanities, libraries, politics how the light gets in posted on january january by bethany nowviskie i took a chance on a hackberry bowl at a farmer’s market—blue-stained and turned like a drop of water. it’s a good name for it. he had hacked it down at the bottom of his garden. (they’re filling in the timber where the oaks aren’t coming back.) but the craftsman had never worked that kind of wood before, kiln-dried at steamy summer’s height. “will it split?” it did. now it’s winter, and i make kintsukuroi, a golden repair. i found the wax conservators use on gilded picture-frames, and had some mailed from london. it softens in the heat of hands. go on. let the dry air crack you open. you can break and be mended again. posted in infrastructure, past lives posts navigation … next travel/talks march -present: global pandemic and a revolutionary air (speaking/travel hiatus) february , : featured talk, aaad : “black temporalities: past, present, and future” july – january : speaking/travel hiatus while starting my new position at james madison university june , : tensions of europe keynote on machine learning & historical understanding, luxembourg june , : rmbs opening plenary on climate change & libraries/archives, baltimore june - , : teaching rare book school in philadelphia: “community archives and digital cultural memory” themes themesselect category administrivia design documents geospatial higher ed infrastructure past lives soft circuits & code swinburne twittering unfiltered archives archives select month october june june march january june april march february november october april february november october may march february november july may february january october september august may january november october june april march january november october september june may april january december october september june april march january december october july june may recent posts foreword (to the past) a pledge: self-examination and concrete action in the jmu libraries change us, too from the grass roots how the light gets in reconstitute the world spectra for speculative knowledge design we raise our voices iv. coda: speculative computing ( ) inauguration day open invitations speculative collections alternate futures/usable pasts everywhere, every when oldies but goodies digital humanities in the anthropocene asking for it toward a new deal resistance in the materials too small to fail reality bytes lazy consensus a skunk in the library why, oh why, cc-by? what do girls dig? standard disclaimer this site and its contents are my responsibility alone, and may not reflect the opinions of my employer, colleagues, students, children, or imaginary friends. yours everything here is free to use under a creative commons attribution . international license. twitter linkedin github flickr instagram powered by miniva wordpress theme the thingology blog the thingology blog new syndetics unbound feature: mark and boost electronic resources proquest and librarything have just introduced a major new feature to our catalog-enrichment suite, syndetics unbound, to meet the needs of libraries during the covid- crisis. our friends at proquest blogged about it briefly on the proquest blog. this blog post goes into greater detail about what we did, how we did it, and what [&# ;] introducing syndetics unbound short version today we&# ;re going public with a new product for libraries, jointly developed by librarything and proquest. it&# ;s called syndetics unbound, and it makes library catalogs better, with catalog enrichments that provide information about each item, and jumping-off points for exploring the catalog. to see it in action, check out the hartford public library [&# ;] alamw in boston (and free passes)! abby and kj will be at ala midwinter in boston this weekend, showing off librarything for libraries. since the conference is so close to librarything headquarters, chances are good that a few other lt staff members may appear, as well! visit us. stop by booth # to meet abby &# ; kj (and potential mystery guests!), [&# ;] for ala : three free opac enhancements for a limited time, librarything for libraries (ltfl) is offering three of its signature enhancements for free! there are no strings attached. we want people to see how librarything for libraries can improve your catalog. check library. the check library button is a &# ;bookmarklet&# ; that allows patrons to check if your library has a book [&# ;] ala in san francisco (free passes) our booth. but this is kate, not tim or abby. she had the baby. tim and i are headed to san francisco this weekend for the ala annual conference. visit us. stop by booth # to talk to us, get a demo, and learn about all the new and fun things we&# ;re up to with [&# ;] new &# ;more like this&# ; for librarything for libraries we&# ;ve just released &# ;more like this,&# ; a major upgrade to librarything for libraries’ &# ;similar items&# ; recommendations. the upgrade is free and automatic for all current subscribers to librarything for libraries catalog enhancement package. it adds several new categories of recommendations, as well as new features. we&# ;ve got text about it below, but here&# ;s a short [&# ;] subjects and the ship of theseus i thought i might take a break to post an amusing photo of something i wrote out today: the photo is a first draft of a database schema for a revamp of how librarything will do library subjects. all told, it has tables. gulp. about eight of the tables do what a good cataloging [&# ;] librarything recommends in bibliocommons does your library use bibliocommons as its catalog? librarything and bibliocommons now work together to give you high-quality reading recommendations in your bibliocommons catalog. you can see some examples here. look for &# ;librarything recommends&# ; on the right side. not that kind of girl (daniel boone regional library) carthage must be destroyed (ottowa public library) the [&# ;] new: annotations for book display widgets our book display widgets is getting adopted by more and more libraries, and we&# ;re busy making it better and better. last week we introduced easy share. this week we&# ;re rolling out another improvement—annotations! book display widgets is the ultimate tool for libraries to create automatic or hand-picked virtual book displays for their home page, blog, [&# ;] send us a programmer, win $ , in books. we just posted a new job post job: library developer at librarything (telecommute). to sweeten the deal, we are offering $ , worth of books to the person who finds them. that&# ;s a lot of books. rules! you get a $ , gift certificate to the local, chain or online bookseller of your choice. to qualify, you [&# ;] diversity, equity and inclusion core competencies: influence others (part of ) - tara robertson consulting skip to content tara robertson consulting diversity, equity + inclusion menu about services presentations blog contact posted on january , january , by tara robertson diversity, equity and inclusion core competencies: influence others (part of ) this is the fourth post in a week-long series exploring dei professional competencies. i believe the five key competencies for dei professionals are: be strategic translate academic research into action and measure the impact of initiatives meet people where they are at and help them move to be more inclusive influence others get cross functional projects done. yesterday i wrote about being a professional change agent and the need to meet people where they’re at and help them change their perspective and behaviour to be more inclusive. today i’m going to explore the ability to influence others. dei leaders need to be able to influence beyond their small team, at all levels of an organization. the way i did this was by building authentic relationships, learning about what other people’s priorities are, and negotiating how to be mutually successful. for example, i reached out to the av operations team to advocate for live captioning for our big internal meetings to increase access, both for people who were hard of hearing, people who process content better with text, and for people for whom english was an additional language. they worked to make this part of the workflow and handled the administration with the captioning vendor. over a year later the av operations team reached out to me to partner on the sound quality in the office meetings rooms. as a distributed workforce we spent a lot of time in zoom meetings and some rooms had better sound quality than others. also, for some neurodiverse people they were too noisy and echoey and made it exhausting to be in meetings, so it was an accessibility issue too. i recently did cliftonstrengths and one of my top strengths is woo, or winning others over. cliftonstrenghts describes this as: “people exceptionally talented in the woo theme love the challenge of meeting new people and winning them over. they derive satisfaction from breaking the ice and making a connection with someone.” dei leaders have little if no formal power, so being skilled at this is necessary. this is the fourth in a series of five posts. tomorrow’s post, the last one in this series, is about getting cross functional projects done. share this: click to share on twitter (opens in new window) click to share on facebook (opens in new window) related leave a reply cancel reply your email address will not be published. required fields are marked * comment name * email * website notify me of follow-up comments by email. notify me of new posts by email. post navigation previous postprevious core competencies in dei: meet people where they are at and help them move to be more inclusive (part of ) next postnext diversity, equity and inclusion core competencies: get cross functional projects done (part of ) search for: search tara robertson consulting is located in vancouver, canada on the unceded xʷməθkwəy̓əm (musqueam), skwxwú mesh (squamish), səl̓ílwətaʔ/selilwitulh (tsleil-waututh) territories twitter linkedin proudly powered by wordpress geojson - wikipedia geojson from wikipedia, the free encyclopedia jump to navigation jump to search geojson filename extensions .json, .geojson internet media type application/geo+json[ ] type of format gis file format extended from json standard rfc open format? yes website geojson.org geojson[ ] is an open standard format designed for representing simple geographical features, along with their non-spatial attributes. it is based on the json format. the features include points (therefore addresses and locations), line strings (therefore streets, highways and boundaries), polygons (countries, provinces, tracts of land), and multi-part collections of these types. geojson features need not represent entities of the physical world only; mobile routing and navigation apps, for example, might describe their service coverage using geojson.[ ] the geojson format differs from other gis standards in that it was written and is maintained not by a formal standards organization, but by an internet working group of developers.[ ] a notable offspring of geojson is topojson, an extension of geojson that encodes geospatial topology and that typically provides smaller file sizes. contents history example . geometries topojson . topojson schema see also references sources external links history[edit] the geojson format working group and discussion were begun in march [ ] and the format specification was finalized in june . in april the internet engineering task force has founded the geographic json working group[ ] which released geojson as rfc in august . example[edit] { "type": "featurecollection", "features": [ { "type": "feature", "geometry": { "type": "point", "coordinates": [ . , . ] }, "properties": { "prop ": "value " } }, { "type": "feature", "geometry": { "type": "linestring", "coordinates": [ [ . , . ], [ . , . ], [ . , . ], [ . , . ] ] }, "properties": { "prop ": "value ", "prop ": . } }, { "type": "feature", "geometry": { "type": "polygon", "coordinates": [ [ [ . , . ], [ . , . ], [ . , . ], [ . , . ], [ . , . ] ] ] }, "properties": { "prop ": "value ", "prop ": { "this": "that" } } } ] } geometries[edit] geometry primitives type examples point { "type": "point", "coordinates": [ , ] } linestring { "type": "linestring", "coordinates": [ [ , ], [ , ], [ , ] ] } polygon { "type": "polygon", "coordinates": [ [[ , ], [ , ], [ , ], [ , ], [ , ]] ] } { "type": "polygon", "coordinates": [ [[ , ], [ , ], [ , ], [ , ], [ , ]], [[ , ], [ , ], [ , ], [ , ]] ] } multipart geometries type examples multipoint { "type": "multipoint", "coordinates": [ [ , ], [ , ], [ , ], [ , ] ] } multilinestring { "type": "multilinestring", "coordinates": [ [[ , ], [ , ], [ , ]], [[ , ], [ , ], [ , ], [ , ]] ] } multipolygon { "type": "multipolygon", "coordinates": [ [ [[ , ], [ , ], [ , ], [ , ]] ], [ [[ , ], [ , ], [ , ], [ , ], [ , ]] ] ] } { "type": "multipolygon", "coordinates": [ [ [[ , ], [ , ], [ , ], [ , ]] ], [ [[ , ], [ , ], [ , ], [ , ], [ , ], [ , ]], [[ , ], [ , ], [ , ], [ , ]] ] ] } geometrycollection { "type": "geometrycollection", "geometries": [ { "type": "point", "coordinates": [ , ] }, { "type": "linestring", "coordinates": [ [ , ], [ , ], [ , ] ] }, { "type": "polygon", "coordinates": [ [[ , ], [ , ], [ , ], [ , ]] ] } ] } topojson[edit] topojson is an extension of geojson that encodes topology. rather than representing geometries discretely, geometries in topojson files are stitched together from shared line segments called arcs.[ ] arcs are sequences of points, while line strings and polygons are defined as sequences of arcs. each arc is defined only once, but can be referenced several times by different shapes, thus reducing redundancy and decreasing the file size.[ ] in addition, topojson facilitates applications that use topology, such as topology-preserving shape simplification, automatic map coloring, and cartograms. a reference implementation of the topojson specification is available as a command-line tool to encode topojson from geojson (or esri shapefiles) and a client side javascript library to decode topojson back to geojson again. topojson is also supported by the popular ogr tool as of version . [ ] and postgis as of version . . .[ ] topojson schema[edit] given a gis shape near coordinates latitude ° and longitude °, a simple but valid and complete topojson file containing all metadata, polygon, linestring, point elements, arcs and properties is defined as follows: topojson shapes { "type":"topology", "transform":{ "scale": [ , ], "translate": [ , ] }, "objects":{ "two-squares":{ "type": "geometrycollection", "geometries":[ {"type": "polygon", "arcs":[[ , ]],"properties": {"name": "left_polygon" }}, {"type": "polygon", "arcs":[[ ,- ]],"properties": {"name": "right_polygon" }} ] }, "one-line": { "type":"geometrycollection", "geometries":[ {"type": "linestring", "arcs": [ ],"properties":{"name":"under_linestring"}} ] }, "two-places":{ "type":"geometrycollection", "geometries":[ {"type":"point","coordinates":[ , ],"properties":{"name":"origine_point"}}, {"type":"point","coordinates":[ ,- ],"properties":{"name":"under_point"}} ] } }, "arcs": [ [[ , ],[ ,- ]], [[ , ],[- , ],[ , ],[ , ]], [[ , ],[ , ],[ ,- ],[- , ]], [[ ,- ],[ , ]] ] } see also[edit] geography markup language gis vector file format references[edit] ^ a b butler, howard; daly, martin; doyle, allan; gillies, sean; hagen, stefan; schaub, tim (august ). rfc . ietf. doi: . /rfc . ^ "providing directions". developer.apple.com. ^ "geojson info page". lists.geojson.org. ^ "the geojson march archive by thread". lists.geojson.org. ^ "geographic json (geojson) -". datatracker.ietf.org. ^ "topojson/topojson-specification". december , – via github. ^ "topojson/topojson". github. ^ "release/ . . -news – gdal". trac.osgeo.org. ^ "astopojson". postgis.net. sources[edit] andrews, christopher ( september ). "emerging technology: ajax and geojson". archived from the original on may . "geoweb guru: technical overview: geojson". february . archived from the original on - - . lambert, chris ( may ). "google geo developers blog: build on top of your public latitude location with geojson & kml". external links[edit] official website topojson/topojson topojson/topojson-specification retrieved from "https://en.wikipedia.org/w/index.php?title=geojson&oldid= " categories: json gis vector file formats hidden categories: pages using rfc magic links navigation menu personal tools not logged in talk contributions create account log in namespaces article talk variants views read edit view history more search navigation main page contents current events random article about wikipedia contact us donate contribute help learn to edit community portal recent changes upload file tools what links here related changes upload file special pages permanent link page information cite this page wikidata item print/export download as pdf printable version languages català Čeština deutsch español français 한국어 italiano nederlands 日本語 português Русский Српски / srpski Українська 中文 edit links this page was last edited on january , at : (utc). text is available under the creative commons attribution-sharealike license; additional terms may apply. by using this site, you agree to the terms of use and privacy policy. wikipedia® is a registered trademark of the wikimedia foundation, inc., a non-profit organization. privacy policy about wikipedia disclaimers contact wikipedia mobile view developers statistics cookie statement archival connections archival connections project site platform monopolies and archives i am at the interpares trust north american team meeting in vancouver, and the issue of platform monopolies has risen to the top of my mind. here is a quick list of readings i&# ;ve thrown together while listening to and engaging in the discussion: for now, i don&# ;t have much to say, other than this: as a &# ; continue reading platform monopolies and archives sia workshop links just sharing a few links for use during the sia workshop i&# ;ll be teaching later today: google form for exercises sia workshop slides scaling machine-assisted description of historical records one of the questions i&# ;ve been grappling with as part of the archival connections research project is simple: is there a future for the finding aid? i&# ;m inclined to think not, at least not in the form we are used to. looking to the future, i recently had the chance to propose something slightly different, and &# ; continue reading scaling machine-assisted description of historical records social feed manager takeaways later this week, i&# ;ll be introducing the archival connections project at the society of indiana archivists meeting. during the first year of this project, one focus of my work was evaluating and developing some recommendations for using social feed manager, a tool developed by george washington university libraries. my full report is here, for those interested: https://gwu-libraries.github.io/sfm-ui/resources/sfmreportprom .pdf. without &# ; continue reading social feed manager takeaways arrangement and description in the cloud: a preliminary analysis i&# ;m posting a preprint of some early work related to the archival connections project. this work will be published as a book chapter/proceedings by the archiveschule in marburg. in the meantime, here is the preprint: archival arrangement and description in the cloud a preliminary analysis installing social feed manager locally the easiest way to get started with social feed manager is to install docker on a local machine, such as a laptop or (preferably) desktop computer with a persistent internet connection. running sfm locally for anything other than testing purposes is not recommended. it will not be sufficient for a long-term documentation project and would &# ; continue reading installing social feed manager locally preserving email report summary earlier today, i provided a summary of preserving email, a technology watch report i wrote back in . i'll leave it to others to judge how well that report holds up, but i had the following takeaways when re-reading it: introducing archival connections welcome! this shares information from a five-year research project that i am coordinating at the university of illinois at urbana-champaign. the project aims to make it easier for people to find and use the materials managed by archival repositories like the university of illinois archives, where i work. you can read more about the project on the &# ; continue reading introducing archival connections evaluating sources tutorial evaluating sources tutorial this tutorial will help you be able to select quality sources for any assignment. your credibility in your research project depends on the sources you use to support it. if you use sources that are of poor quality, what reason would anyone have to trust the points you are making? this five-minute video will teach you three questions you can ask to determine whether a source is a good fit for your needs. your needs depend on the types of evidence you are looking for, the points you want to make, and the requirements of your assignment. example news article evaluation now, let's look at how we might evaluate an article we found on the web. you can access the article we're exploring -- will daca parents be forced to leave their u.s.-citizen children behind? -- at https://www.theatlantic.com/politics/archive/ / /donald-trump-daca/ / first, i'm going to look into the author. i can click on her name in the article and get a short biography that says that she's a former assistant editor at the atlantic (the magazine in which this article is published). i'm also going to google her name to find out more about her. from my google search, i learn that she is a reporter for cnn currently and used to work for usa today. a journalist who has been hired by at least three major national news sources is probably a pretty good journalist and i see that she specializes in immigration news, so this is an area she spends a lot of time researching. journalists are not necessarily experts in everything they write about, but they are experts at pulling information together from other expert sources (like researchers, people working in that area, people affected by the thing she's writing about) to tell a story. i feel pretty good about trusting priscilla alvarez writing on the issue of daca (deferred action for childhood arrivals). next, i'm going to look at the publication to see what their reputation is. i could just look at their "about" page, but of course that is going to say positive things about them. so instead, i'm going to google the atlantic. what i see in the google search results (beyond their own website) is a wikipedia article. usually that will tell me a lot about the publication and if there are any concerns or controversies. nothing i see in the wikipedia article concerns me and it shows me that the atlantic is a respected, award-winning national news publication that has been around for over years. if i wanted to dig deeper, i might go to the website media bias fact check https://mediabiasfactcheck.com/ to see how they rate the publication. they suggest that it has a left-center bias and that their factual reporting is high. most publications you will find will have some bias, but you want to make sure that, even if they have a bias, what they are publishing is truthful. this suggests that the atlantic posts factual stories with a left-center bias. if i were writing a paper trying to convince a very conservative audience of something, the atlantic might not be the best source to use because it is a somewhat liberal publication, but if i'm writing for a more general audience, this is a solid and well-respected source. because journalists are not usually experts on the particular things they are writing about, their work is only as good as the sources they use. so it's worth looking at where their information came from. in some articles and books, you'll find a list of cited sources as the end of the article or chapter, but news articles usually list their sources more informally within the body of the article. i'll highlight a few examples below: you can see just in these three paragraphs that the author got information from ) a mother who is a daca recipient, ) the u.s. department of homeland security, and ) a research study. this alone suggests that this is a well-researched article where she is looking at the government response, scholarly research on the topic, and has talked to real people who are impacted by the issue. that is a great mix of sources and is a great example actually of the sort of mix many instructors want to see in your own research. based on the three areas i looked into (the author, the publication or website, and the sources used), i feel pretty confident using this in my research. next, you're going to look at two other articles about daca and will analyze them using the same methods i used here. next never submit passwords through google forms. this form was created inside of portland community college. report abuse forms cmoa collection skip to main content the carnegie museum of art bracket logo with wordmark the carnegie museum of art wordmark logo menu visitvisit sub menu accessibility & amenities directions & parking tours health & safety exhibitionsexhibitions sub menu current upcoming past collection events & programs supportsupport sub menu membership sponsorship volunteer shop collection our collection contains over , objects featuring a broad spectrum of visual art. the museum also houses over , negatives by photographer charles “teenie” harris. works are currently available online. searchsearch filter by: creator date classification department location show has image on view results searching... sorted by random. options sort by:random relevance creator (a–z) creator (z–a) title (a–z) title (z–a) creation date (old–new) creation date (new–old) acquisition date (old–new) acquisition date (new–old) grid view list view pagination results per page the cmoa logo forbes avenue pittsburgh, pa ( ) - one of the four carnegie museums of pittsburgh about accessibility & amenities careers event rentals press visitor policies © carnegie museum of art, carnegie instituteterms of useprivacy policynon-discrimination policycontact learning (lib)tech learning (lib)tech stories from my life as a technologist choosing not to go into management (again) often, to move up and get a higher pay, you have to become a manager, but not everyone is suited to become a manager, and sometimes given the preference, it&# ;s not what someone wants to do. thankfully at gitlab, in every engineering team including support, we have two tracks: technical (individual contributor), and management. progression &# ; continue reading "choosing not to go into management&# ;(again)" prioritization in support: tickets, slack, issues, and more i mentioned in my gitlab reflection that prioritization has been quite different working in support compared to other previous work i&# ;ve done. in most of my previous work, i&# ;ve had to take &# ;desk shifts&# ; but those are discreet where you&# ;re focused on providing customer service during that period of time and you can focus on &# ; continue reading "prioritization in support: tickets, slack, issues, and&# ;more" reflection part : my second year at gitlab and on becoming senior again this reflection is a direct continuation of part of my time at gitlab so far. if you haven&# ;t, please read the first part before beginning this one. becoming an engineer ( months) the more time i spent working in support, the more i realized that the job was much more technical than i originally &# ; continue reading "reflection part : my second year at gitlab and on becoming senior&# ;again" reflection part : my first year at gitlab and becoming senior about a year ago, i wrote a reflection on summit and contribute, our all staff events, and later that year, wrote a series of posts on the gitlab values and culture from my own perspective. there is a lot that i mention in the blog post series and i&# ;ll try not to repeat myself (too &# ; continue reading "reflection part : my first year at gitlab and becoming&# ;senior" is blog reading dead? there was a bit more context to the question, but a friend recently asked me: what you do think? is blogging dead? i think blogging the way it used to work is (mostly) dead. back in the day, we had a bunch of blogs and people who subscribe to them via email and rss feeds. &# ; continue reading "is blog reading&# ;dead?" working remotely at home as a remote worker during a pandemic i&# ;m glad that i still have a job, that my life isn&# ;t wholly impacted by the pandemic we&# ;re in, but to say that nothing is different just because i was already a remote worker would be wrong. the effect the pandemic is having on everyone around you has affects your life. it seems obvious to &# ; continue reading "working remotely at home as a remote worker during a&# ;pandemic" code libbc lightning talk notes: day code libbc day lightning talk notes! code club for adults/seniors &# ; dethe elza richmond public library, digital services technician started code clubs, about years ago used to call code and coffee, chain event, got little attendance had code codes for kids, teens, so started one for adults and seniors for people who have done &# ; continue reading "code libbc lightning talk notes: day&# ; " code libbc lightning talk notes: day code libbc day lightning talk notes! scraping index pages and vufind implementation &# ; louise brittain boisvert systems librarian at legislative collection development policy: support legislators and staff, receive or collect publications, many of them digital but also some digitized (mostly pdf, but others) accessible via link in marc record previously, would create an index page &# ; continue reading "code libbc lightning talk notes: day&# ; " presentation: implementing values in practical ways this was presented at code libbc . slides slides on github hi everyone, hope you’re enjoying code libbc so far. while i’m up here, i just want to take a quick moment to thank the organizers past and present. we’re on our th one and still going strong. i hope to continue attending and see this event &# ; continue reading "presentation: implementing values in practical&# ;ways" implementing values: learning from gitlab: transparency this is the sixth value covered in a series of blog posts on what we can learn in implementing values that are the same or similar to gitlab&# ;s credit values. for background and links to the other posts, please check out the overview post. transparency i have never encountered an organization that was more open &# ; continue reading "implementing values: learning from gitlab:&# ;transparency" library tech talk - u-m library library tech talk - u-m library technology innovations and project updates from the u-m library i.t. division library it services portfolio academic library service portfolios are mostly a mix of big to small strategic initiatives and tactical projects. systems developed in the past can become a durable bedrock of workflows and services around the library, remaining relevant and needed for five, ten, and sometimes as long as twenty years. there is, of course, never enough time and resources to do everything. the challenge faced by library it divisions is to balance the tension of sustaining these legacy systems while continuing to innovate and develop new services. the university of michigan’s library it portfolio has legacy systems in need of ongoing maintenance and support, in addition to new projects and services that add to and expand the portfolio. we, at michigan, worked on a process to balance the portfolio of services and projects for our library it division. we started working on the idea of developing a custom tool for our needs since all the other available tools are oriented towards corporate organizations and we needed a light-weight tool to support our process. we went through a complete planning process first on whiteboards and paper, then developed an open source tool called tracc for helping us with portfolio management. keys to a dazzling library website redesign the u-m library launched a completely new primary website in july after years of work. the redesign project team focused on building a strong team, internal communication, content strategy, and practicing needs informed design and development to make the project a success. sweet sixteen: digital collections completed july - june digital content & collections (dcc) relies on content and subject experts to bring us new digital collections. this year, digital collections were created or significantly enhanced. here you will find links to videos and articles by the subject experts speaking in their own words about the digital collections they were involved in and why they found it so important to engage in this work with us. thank you to all of the people involved in each of these digital collections! adding ordered metadata fields to samvera hyrax how to add ordered metadata fields in samvera hyrax. includes example code and links to actual code. sinking our teeth into metadata improvement like many attempts at revisiting older materials, working with a couple dozen volumes of dental pamphlets started very simply but ended up being an interesting opportunity to explore the challenges of making the diverse range of materials held in libraries accessible to patrons in a digital environment. and while improving metadata may not sound glamorous, having sufficient metadata for users to be able to find what they are looking for is essential for the utility of digital libraries. collaboration and generosity provide the missing issue of the american jewess what started with a bit of wondering and conversation within our unit of the library led to my reaching out to princeton university with a request but no expectations of having that request fulfilled. individuals at princeton, however, considered the request and agreed to provide us with the single issue of the american jewess that we needed to complete the full run of the periodical within our digital collection. especially in these stressful times, we are delighted to bring you a positive story, one of collaboration and generosity across institutions, while also sharing the now-complete digital collection itself. how to stop being negative, or digitizing the harry a. franck film collection this article reviews how , + frames of photographic negatives from the harry a. franck collection are being digitally preserved. combine metadata harvester: aggregate all the data! the digital public library of america (dpla) has collected and made searchable a vast quantity of metadata from digital collections all across the country. the michigan service hub works with cultural heritage institutions throughout the state to collect their metadata, transform those metadata to be compatible with the dpla’s online library, and send the transformed metadata to the dpla, using the combine aggregator software, which is being developed here at the u of m library. hacks with friends retrospective: a pitch to hitch in when the students go on winter break i go to hacks with friends (hwf) and highly recommend and encourage everyone who can to participate in hwf . not only is it two days of free breakfast, lunch, and snacks at the ross school of business, but it’s a chance to work with a diverse cross section of faculty, staff, and students on innovative solutions to complex problems. u-m library’s digital collection items are now included in library search the university library’s digital collections, encompassing more than collections with over a million items, are now discoverable through the library’s articles discovery tool, powered by summon. read on to learn about searching this trove of images and text, and how to add it to your library’s summon instance. redirecting falvey memorial library blog falvey memorial library blog the collection of blogs published by falvey memorial library, villanova university falvey memorial library :: the collection of blogs published by falvey memorial library, villanova university skip navigation falvey memorial library visit / apply / give my library account collections research services using the library about falvey memorial library search everything books & media title journal title author subject call number isbn/issn tag articles & more article title article author other libraries (ill) ill title ill author ill subject ill call number ill isbn/issn library website guides digital library search for books, articles, library site, almost anything advanced you are exploring: home > blogs falvey memorial library blog falvey library blogs announcing the literary festival lineup! february , library newsliterary festival, virtual events the rd annual villanova literary festival begins thursday, feb. ! please register for these exciting virtual readings and talks. after registering, you will receive a confirmation email containing information about joining the event. all ... read more foto friday: studyin’ in the snow february , library newsfoto friday, groundhogs day, readin room, studying even if punxsutawney phil, the groundhog who foretells whether winter will be long or short, saw his shadow and called for more weeks of winter chill, so what? our students don’t need will d. cat to let them know that after “snow” ... read more view recordings of workshops on course materials assistance programs february , library newsaffordable materials project, amp, textbooks existing and emerging financial limitations, covid- , and the pivot to online learning has exposed challenges many students face securing the course materials they need to succeed and thrive. villanova faculty and academic support staff are ... read more tbt: snow-covered campus february , library newscampus, digital library, snow, tbt, throwback, throwback thursday, villanova campus we have officially had our first (two) snow days of the semester! in case you weren’t able to wander around campus in the snow earlier this week, here’s a picture of the snowy campus from . it can be found in falvey’s digital library ... read more cat in the stax: connecting virtually february , library news'cat in the stacks, cat in the stack, cat in the stax, covid, covid- , library events, social distancing, virtual events, zoom college is all about meeting and connecting with new people from all over the country that normally you wouldn’t get an opportunity to interact with if you stayed in your small town. unfortunately, the last year has made it really difficult to ... read more black history month february , blue electrode: sparking between silicon and paperblack history month, digital library, university archives black history month is a celebration of the contributions of african americans have made to our history and a time of deep reflection on the continued struggle for racial justice. has shown our society has fallen short on racial equality and ... read more register for the bible and black lives matter event on feb. th! february , library newsblack history month, virtual event faculty, staff, and students are invited to join us on sunday, feb. from : - : pm for a virtual talk and conversation led by the rev. naomi washington-leapheart and black campus ministry students on the “bible and black lives.� ... read more service alert: tuesday, february february , library news service alert: due to the weather, library services will continue to be unavailable on tuesday, feb. . the building is open / for villanova students, faculty, and staff. a wildcard is required for entry and a mask must be worn while visiting. ... read more villanova commemorates dr. martin luther king, jr.— freedom school and mlk keynote address february , library newsfreedom school, keeanga-yamahtta taylor, library resources, martin luther king jr., mlk keynote address dr. keeanga-yamahtta taylor will deliver the dr. martin luther king, jr. keynote address on wednesday, feb. , at p.m. her lecture entitled, “the radical king and the quest to change america,” will be available through the mlk keynote ... read more search falvey library blogs categories blue electrode: sparking between silicon and paper library news resources technology developments feeds content comments meta log in last modified: december , lancaster ave., villanova, pa . . contact directions privacy & security diversity higher education act my nova villanova a-z directory work at villanova accessibility ask us: live chat coffee|code: dan scott's blog - coding coffee|code: dan scott's blog - coding librarian · developer our nginx caching proxy setup for evergreen details of our nginx caching proxy settings for evergreen enriching catalogue pages in evergreen with wikidata an openly licensed javascript widget that enriches library catalogues with wikidata data wikidata, canada , and music festival data at caml , stacy allison-cassin and i presented our arguments in favour of using wikidata is a good fit for communities who want to increase the visibility of canadian music in wikimedia foundation projects. wikidata workshop for librarians interested in learning about wikidata? i delivered a workshop for librarians and archivists at the caml preconference. perhaps you will find the materials i developed useful for your own training purposes. truly progressive webvr apps are available offline! i've been dabbling with the a-frame framework for creating webvr experiences for the past couple of months, ever since patrick trottier gave a lightning talk at the gdg sudbury devfest in november and a hands-on session with aframe in january. the &# ;aframevr twitter feed regularly highlights cool new webvr apps … schema.org, wikidata, knowledge graph: strands of the modern semantic web my slides from ohio devfest : schema.org, wikidata, knowledge graph: strands of the modern semantic web and the video, recorded and edited by the incredible amazing patrick hammond: in november, i had the opportunity to speak at ohio devfest . one of the organizers, casey borders, had invited me … google scholar's broken recaptcha hurts libraries and their users update - - : the brilliant folk at unc figured out how to fix google scholar using a pre-scoped search so that, if a search is launched from the library web site, it will automatically associate that search with the library's licensed resources. no ezproxy required! for libraries, proxying user requests is … php's file_marc gets a new release ( . . ) yesterday, just one day before the anniversary of the . . release, i published the . . release of the pear file_marc library. the only change is the addition of a convenience method for fields called getcontents() that simply concatenates all of the subfields together in order, with … php's file_marc gets a new release ( . . ) yesterday, just one day before the anniversary of the . . release, i published the . . release of the pear file_marc library. the only change is the addition of a convenience method for fields called getcontents() that simply concatenates all of the subfields together in order, with … chromebooks and privacy: not always at odds on friday, june th i gave a short talk at the olita digital odyssey conference, which had a theme this year of privacy and security. my talk addressed the evolution of our public and loaner laptops over the past decade, from bare windows xp, to linux, windows xp with … chromebooks and privacy: not always at odds on friday, june th i gave a short talk at the olita digital odyssey conference, which had a theme this year of privacy and security. my talk addressed the evolution of our public and loaner laptops over the past decade, from bare windows xp, to linux, windows xp with … library stories: vision: "professional research tools" for a recent strategic retreat, i was asked to prepare (as homework) a story about a subject that i'm passionate about, with an idea of where we might see the library in the next three to five years. here's one of the stories i came up with, in the form … library stories: vision: "professional research tools" for a recent strategic retreat, i was asked to prepare (as homework) a story about a subject that i'm passionate about, with an idea of where we might see the library in the next three to five years. here's one of the stories i came up with, in the form … querying evergreen from google sheets with custom functions via apps script our staff were recently asked to check thousands of isbns to find out if we already have the corresponding books in our catalogue. they in turn asked me if i could run a script that would check it for them. it makes me happy to work with people who believe … querying evergreen from google sheets with custom functions via apps script our staff were recently asked to check thousands of isbns to find out if we already have the corresponding books in our catalogue. they in turn asked me if i could run a script that would check it for them. it makes me happy to work with people who believe … that survey about ezproxy oclc recently asked ezproxy clients to fill out a survey about their experiences with the product and to get feedback on possible future plans for the product. about half-way through, i decided it might be a good idea to post my responses. because hey, if i'm working to help them … that survey about ezproxy oclc recently asked ezproxy clients to fill out a survey about their experiences with the product and to get feedback on possible future plans for the product. about half-way through, i decided it might be a good idea to post my responses. because hey, if i'm working to help them … "the librarian" - an instruction session in the style of "the martian" i had fun today. a colleague in computer science has been giving his c++ students an assignment to track down an article that is only available in print in the library. when we chatted about it earlier this year, i suggested that perhaps he could bring me in as a … "the librarian" - an instruction session in the style of "the martian" i had fun today. a colleague in computer science has been giving his c++ students an assignment to track down an article that is only available in print in the library. when we chatted about it earlier this year, i suggested that perhaps he could bring me in as a … we screwed up: identities in loosely-coupled systems a few weeks ago, i came to the startling and depressing realization that we had screwed up. it started when someone i know and greatly respect ran into me in the library and said "we have a problem". i'm the recently appointed chair of our library and archives department, so … we screwed up: identities in loosely-coupled systems a few weeks ago, i came to the startling and depressing realization that we had screwed up. it started when someone i know and greatly respect ran into me in the library and said "we have a problem". i'm the recently appointed chair of our library and archives department, so … research across the curriculum the following post dates back to january , , when i had been employed at laurentian for less than a year and was getting an institutional repository up and running.... i think old me had some interesting thoughts! abstract the author advocates an approach to university curriculum that re-emphasizes the … research across the curriculum the following post dates back to january , , when i had been employed at laurentian for less than a year and was getting an institutional repository up and running.... i think old me had some interesting thoughts! abstract the author advocates an approach to university curriculum that re-emphasizes the … library and archives canada: planning for a new union catalogue update - - : clarified (in the privacy section) that only nrcan runs evergreen. i attended a meeting with library and archives canada today in my role as an ontario library association board member to discuss the plans around a new canadian union catalogue based on oclc's hosted services. following are some … library and archives canada: planning for a new union catalogue update - - : clarified (in the privacy section) that only nrcan runs evergreen. i attended a meeting with library and archives canada today in my role as an ontario library association board member to discuss the plans around a new canadian union catalogue based on oclc's hosted services. following are some … library catalogues and http status codes i noticed in google's webmaster tools that our catalogue had been returning some soft s. curious, i checked into some of the uris suffering from this condition, and realized that evergreen returns an http status code of ok when it serves up a record details page for a record … library catalogues and http status codes i noticed in google's webmaster tools that our catalogue had been returning some soft s. curious, i checked into some of the uris suffering from this condition, and realized that evergreen returns an http status code of ok when it serves up a record details page for a record … dear database vendor: defending against sci-hub.org scraping is going to be very difficult our library receives formal communications from various content/database vendors about "serious intellectual property infringement" on a reasonably regular basis, that urge us to "pay particular attention to proxy security". here is part of the response i sent to the most recent such request: we use the usagelimit directives that … dear database vendor: defending against sci-hub.org scraping is going to be very difficult our library receives formal communications from various content/database vendors about "serious intellectual property infringement" on a reasonably regular basis, that urge us to "pay particular attention to proxy security". here is part of the response i sent to the most recent such request: we use the usagelimit directives that … putting the "web" back into semantic web in libraries i was honoured to lead a workshop and speak at this year's edition of semantic web in bibliotheken (swib) in bonn, germany. it was an amazing experience; there were so many rich projects being described with obvious dividends for the users of libraries, once again the european library community fills … putting the "web" back into semantic web in libraries i was honoured to lead a workshop and speak at this year's edition of semantic web in bibliotheken (swib) in bonn, germany. it was an amazing experience; there were so many rich projects being described with obvious dividends for the users of libraries, once again the european library community fills … social networking for researchers: researchgate and their ilk the centre for research in occupational safety and health asked me to give a lunch'n'learn presentation on researchgate today, which was a challenge i was happy to take on... but i took the liberty of stretching the scope of the discussion to focus on social networking in the context of … social networking for researchers: researchgate and their ilk the centre for research in occupational safety and health asked me to give a lunch'n'learn presentation on researchgate today, which was a challenge i was happy to take on... but i took the liberty of stretching the scope of the discussion to focus on social networking in the context of … how discovery layers have closed off access to library resources, and other tales of schema.org from lita forum at the lita forum yesterday, i accused (presentation) most discovery layers of not solving the discoverability problems of libraries, but instead exacerbating them by launching us headlong to a closed, unlinkable world. coincidentally, lorcan dempsey's opening keynote contained a subtle criticism of discovery layers. i wasn't that subtle. here's why … how discovery layers have closed off access to library resources, and other tales of schema.org from lita forum at the lita forum yesterday, i accused (presentation) most discovery layers of not solving the discoverability problems of libraries, but instead exacerbating them by launching us headlong to a closed, unlinkable world. coincidentally, lorcan dempsey's opening keynote contained a subtle criticism of discovery layers. i wasn't that subtle. here's why … dcmi : schema.org holdings in open source library systems my slides from dcmi : schema.org in the wild: open source libraries++. last week i was at the dublin core metadata initiative conference, where richard wallis, charles maccathie nevile and i were slated to present on schema.org and the work of the w c schema.org bibliographic extension … my small contribution to schema.org this week version . of the http://schema.org vocabulary was released a few days ago, and i once again had a small part to play in it. with the addition of the workexample and exampleofwork properties, we (richard wallis, dan brickley, and i) realized that examples of these creativework example … my small contribution to schema.org this week version . of the http://schema.org vocabulary was released a few days ago, and i once again had a small part to play in it. with the addition of the workexample and exampleofwork properties, we (richard wallis, dan brickley, and i) realized that examples of these creativework example … posting on the laurentian university library blog since returning from my sabbatical, i've felt pretty strongly that one of the things our work place is lacking is open communication about the work that we do--not just outside of the library, but within the library as well. i'm convinced that the more that we know about the demands … posting on the laurentian university library blog since returning from my sabbatical, i've felt pretty strongly that one of the things our work place is lacking is open communication about the work that we do--not just outside of the library, but within the library as well. i'm convinced that the more that we know about the demands … cataloguing for the open web: schema.org in library catalogues and websites tldr; my slides are href="http://stuff.coffeecode.net/ /understanding_schema">here, and the slides from jenn and jason are also available from href="http://connect.ala.org/node/ ">ala connect. on sunday, june th jenn riley, jason clark, and i presented at the alcts/lita jointly sponsored session … cataloguing for the open web: schema.org in library catalogues and websites tldr; my slides are href="http://stuff.coffeecode.net/ /understanding_schema">here, and the slides from jenn and jason are also available from href="http://connect.ala.org/node/ ">ala connect. on sunday, june th jenn riley, jason clark, and i presented at the alcts/lita jointly sponsored session … linked data interest panel, part good talk by richard wallis this morning at the ala annual conference on publishing entities on the web. many of his points map extremely closely to what i've been saying and will be saying tomorrow during my own session (albeit with ten fewer minutes). i was particularly heartened to hear … linked data interest panel, part good talk by richard wallis this morning at the ala annual conference on publishing entities on the web. many of his points map extremely closely to what i've been saying and will be saying tomorrow during my own session (albeit with ten fewer minutes). i was particularly heartened to hear … rdfa introduction and codelabs for libraries my rdfa introduction and codelab materials for the ala preconference on practical linked data with open source are now online! and now i've finished leading the rdfa + schema.org codelab that i've been stressing over and refining for about a month at the american library association annual conference practical … rdfa introduction and codelabs for libraries my rdfa introduction and codelab materials for the ala preconference on practical linked data with open source are now online! and now i've finished leading the rdfa + schema.org codelab that i've been stressing over and refining for about a month at the american library association annual conference practical … dropping back into the semantic web i've been at the extended (formerly european) semantic web conference ( eswc) in anissaras, greece for four days now. my reason for attending was to present my paper seeding structured data by default in open source library systems (presentation) (paper). it has been fantastic. as a librarian attending a conference … dropping back into the semantic web i've been at the extended (formerly european) semantic web conference ( eswc) in anissaras, greece for four days now. my reason for attending was to present my paper seeding structured data by default in open source library systems (presentation) (paper). it has been fantastic. as a librarian attending a conference … rdfa, schema.org, and open source library systems two things of note: i recently submitted the camera-ready copy for my eswc paper, seeding structured data by default via open source library systems (**preprint**). the paper focuses on the work i've done with evergreen, koha, and vufind to use emerging web standards such as rdfa lite and schema … rdfa, schema.org, and open source library systems two things of note: i recently submitted the camera-ready copy for my eswc paper, seeding structured data by default via open source library systems (**preprint**). the paper focuses on the work i've done with evergreen, koha, and vufind to use emerging web standards such as rdfa lite and schema … mapping library holdings to the product / offer mode in schema.org back in august, i mentioned that i taught evergreen, koha, and vufind how to express library holdings in schema.org via the http://schema.org/offer class. what i failed to mention was how others can do the same with their own library systems (well, okay, i linked to the … mapping library holdings to the product / offer mode in schema.org back in august, i mentioned that i taught evergreen, koha, and vufind how to express library holdings in schema.org via the http://schema.org/offer class. what i failed to mention was how others can do the same with their own library systems (well, okay, i linked to the … what would you understand if you read the entire world wide web? on tuesday, february th, i'll be participating in laurentian university's research week lightning talks. unlike most five-minute lightning talk events in which i've participated, the time limit for each talk tomorrow will be one minute. imagine different researchers getting up to summarize their research in one minute each, and … what would you understand if you read the entire world wide web? on tuesday, february th, i'll be participating in laurentian university's research week lightning talks. unlike most five-minute lightning talk events in which i've participated, the time limit for each talk tomorrow will be one minute. imagine different researchers getting up to summarize their research in one minute each, and … ups and downs tuesday was not the greatest day, but at least each setback resulted in a triumph... first, the periodical proposal for schema.org--that i have poured a good couple of months of effort into--took a step closer to reality when dan brickley announced on the public-vocabs list that he had … ups and downs tuesday was not the greatest day, but at least each setback resulted in a triumph... first, the periodical proposal for schema.org--that i have poured a good couple of months of effort into--took a step closer to reality when dan brickley announced on the public-vocabs list that he had … broadening support for linked data in marc the following is an email that i sent to the marc mailing list on january , that might be of interest to those looking to provide better support for linked data in marc (hopefully as just a transitional step): in the spirit of making it possible to express linked … broadening support for linked data in marc the following is an email that i sent to the marc mailing list on january , that might be of interest to those looking to provide better support for linked data in marc (hopefully as just a transitional step): in the spirit of making it possible to express linked … want citations? release your work! last week i was putting the finishing touches on the first serious academic paper i have written in a long time, and decided that i wanted to provide backup for some of the assertions i had made. naturally, the deadline was tight, so getting any articles via interlibrary loan was … want citations? release your work! last week i was putting the finishing touches on the first serious academic paper i have written in a long time, and decided that i wanted to provide backup for some of the assertions i had made. naturally, the deadline was tight, so getting any articles via interlibrary loan was … file_marc: . . release fixes data corruption bug i released file_marc . . yesterday after receiving a bug report from the most excellent mark jordan about a basic (but data corrupting) problem that had existed since the very early days (almost seven years ago). if you generate marc binary output from file_marc, you should upgrade immediately. in … file_marc: . . release fixes data corruption bug i released file_marc . . yesterday after receiving a bug report from the most excellent mark jordan about a basic (but data corrupting) problem that had existed since the very early days (almost seven years ago). if you generate marc binary output from file_marc, you should upgrade immediately. in … talk proposal: structuring library data on the web with schema.org: we're on it! i submitted the following proposal to the library technology conference and thought it might be of general interest. structuring library data on the web with schema.org: we're on it! abstract until recently, there has been a disappointing level of adoption of schema.org structured data in traditional core … talk proposal: structuring library data on the web with schema.org: we're on it! i submitted the following proposal to the library technology conference and thought it might be of general interest. structuring library data on the web with schema.org: we're on it! abstract until recently, there has been a disappointing level of adoption of schema.org structured data in traditional core … file_marc makes it to stable . . release (finally!) way back in , i thought "it's a shame there is no php library for parsing marc records!", and given that much of my most recent coding experience was in the php realm, i thought it would be a good way of contributing to the world of code lib. thus file_marc … file_marc makes it to stable . . release (finally!) way back in , i thought "it's a shame there is no php library for parsing marc records!", and given that much of my most recent coding experience was in the php realm, i thought it would be a good way of contributing to the world of code lib. thus file_marc … finally tangoed with reveal.js to create presentations ... and i have enjoyed the dance. yes, i know i'm way behind the times. over the past few years i was generating presentations via asciidoc, and i enjoyed its very functional approach and basic output. however, recently i used google drive to quickly create a few slightly prettier but much … finally tangoed with reveal.js to create presentations ... and i have enjoyed the dance. yes, i know i'm way behind the times. over the past few years i was generating presentations via asciidoc, and i enjoyed its very functional approach and basic output. however, recently i used google drive to quickly create a few slightly prettier but much … rdfa and schema.org all the library things tldr: the evergreen and koha integrated library systems now express their record details in the schema.org vocabulary out of the box using rdfa. individual holdings are expressed as offer instances per the w c schema bib extension community group proposal to parallel commercial sales offers. and i have published a … rdfa and schema.org all the library things tldr: the evergreen and koha integrated library systems now express their record details in the schema.org vocabulary out of the box using rdfa. individual holdings are expressed as offer instances per the w c schema bib extension community group proposal to parallel commercial sales offers. and i have published a … a flask of full-text search in postgresql update: more conventional versions of the slides are available from google docs or in on speakerdeck (pdf) . on august , , i gave the following talk at the pycon canada conference: i’m a systems librarian at laurentian university. for the past six years, my day job and research … a flask of full-text search in postgresql update: more conventional versions of the slides are available from google docs or in on speakerdeck (pdf) . on august , , i gave the following talk at the pycon canada conference: i’m a systems librarian at laurentian university. for the past six years, my day job and research … parsing the schema.org vocabulary for fun and frustration for various reasons i've spent a few hours today trying to parse the schema.org vocabulary into a nice, searchable database structure. unfortunately, for a linked data effort that's two years old now and arguably one of the most important efforts out there, it's been an exercise in frustration. owl … parsing the schema.org vocabulary for fun and frustration for various reasons i've spent a few hours today trying to parse the schema.org vocabulary into a nice, searchable database structure. unfortunately, for a linked data effort that's two years old now and arguably one of the most important efforts out there, it's been an exercise in frustration. owl … linked data irony, example one of probably many i'm currently ramping up my knowledge of the linked dataworld, and ran across the proceedings of the www workshop on linked data on the web. which are published on the web (yay!) as open access (yay!) in pdf (what?). thus, the papers from the linked data workshop at the w … linked data irony, example one of probably many i'm currently ramping up my knowledge of the linked dataworld, and ran across the proceedings of the www workshop on linked data on the web. which are published on the web (yay!) as open access (yay!) in pdf (what?). thus, the papers from the linked data workshop at the w … pycon canada - postgresql full-text search and flask on august , , i'll be giving a twenty-minute talk at pycon canada on a flask of full-text search with postgresql. i'm very excited to be talking about python, at a python conference, and to be giving the python audience a peek at postgresql's full-text search capabilities. with a twenty … pycon canada - postgresql full-text search and flask on august , , i'll be giving a twenty-minute talk at pycon canada on a flask of full-text search with postgresql. i'm very excited to be talking about python, at a python conference, and to be giving the python audience a peek at postgresql's full-text search capabilities. with a twenty … carlcore metadata application profile for institutional repositories a long time ago, in what seemed like another life, i attended the access conference as a relatively new systems librarian at laurentian university. the subject of the preconference was this totally new-to-me thing called "institutional repositories", which i eventually worked out were basically web applications oriented towards content … carlcore metadata application profile for institutional repositories a long time ago, in what seemed like another life, i attended the access conference as a relatively new systems librarian at laurentian university. the subject of the preconference was this totally new-to-me thing called "institutional repositories", which i eventually worked out were basically web applications oriented towards content … making the evergreen catalogue mobile-friendly via responsive css back in november the evergreen community was discussing the desire for a mobile catalogue, and expressed a strong opinion that the right way forward would be to teach the current catalogue to be mobile-friendly by applying principles of responsive design. in fact, i stated: almost all of this can be … making the evergreen catalogue mobile-friendly via responsive css back in november the evergreen community was discussing the desire for a mobile catalogue, and expressed a strong opinion that the right way forward would be to teach the current catalogue to be mobile-friendly by applying principles of responsive design. in fact, i stated: almost all of this can be … structured data: making metadata matter for machines update - - : now with video of the presentation, thanks to the awesome #egcon volunteers! i've been attending the evergreen conference in beautiful vancouver. this morning, i was honoured to be able to give a presentation on some of the work i've been doing on implementing linked data via schema … structured data: making metadata matter for machines update - - : now with video of the presentation, thanks to the awesome #egcon volunteers! i've been attending the evergreen conference in beautiful vancouver. this morning, i was honoured to be able to give a presentation on some of the work i've been doing on implementing linked data via schema … introducing version control & git in . hours to undergraduates our university offers a computer science degree, but the formal curriculum does not cover version control (or a number of other common tools and practices in software development). students that have worked for me in part-time jobs or summer positions have said things like: if it wasn't for that one … introducing version control & git in . hours to undergraduates our university offers a computer science degree, but the formal curriculum does not cover version control (or a number of other common tools and practices in software development). students that have worked for me in part-time jobs or summer positions have said things like: if it wasn't for that one … triumph of the tiny brain: dan vs. drupal / panels a while ago i inherited responsibility for a drupal instance and a rather out-of-date server. (you know it's not good when your production operating system is so old that it is no longer getting security updates). i'm not a drupal person. i dabbled with drupal years and years ago … triumph of the tiny brain: dan vs. drupal / panels a while ago i inherited responsibility for a drupal instance and a rather out-of-date server. (you know it's not good when your production operating system is so old that it is no longer getting security updates). i'm not a drupal person. i dabbled with drupal years and years ago … finding drm-free books on the google play store john mark ockerbloom recently said, while trying to buy a drm-free copy of john scalzi's redshirts on the google play store: “the catalog page doesn’t tell me what format it’s in, or whether it has drm; it instead just asks me to sign in to buy it.” i … finding drm-free books on the google play store john mark ockerbloom recently said, while trying to buy a drm-free copy of john scalzi's redshirts on the google play store: “the catalog page doesn’t tell me what format it’s in, or whether it has drm; it instead just asks me to sign in to buy it.” i … first go program: converting google scholar xml holdings to ebsco discovery service holdings update - - : and here's how to implement stream-oriented xml parsing many academic libraries are already generating electronic resource holdings summaries in the google scholar xml holdingsformat, and it seems to provide most of the metadata you would need to provide a discovery layer summary in a nice, granular format … first go program: converting google scholar xml holdings to ebsco discovery service holdings update - - : and here's how to implement stream-oriented xml parsing many academic libraries are already generating electronic resource holdings summaries in the google scholar xml holdingsformat, and it seems to provide most of the metadata you would need to provide a discovery layer summary in a nice, granular format … what does a system librarian do? preface: i'm talking to my daughter's kindergarten class tomorrow about my job. exciting! so i prepped a little bit; it will probably go entirely different, but here's how it's going to go in my mind... my name is dan scott. i’m amber’s dad. i’m a systems librarian … farewell, old google books apis since the announcement of the new v google books api, i've been doing a bit of work with it in python (following up on my part of the conversation). today, google announced that many of their older apis were now officially deprecated. included in that list are the google books … the new google books api and possibilities for libraries on the subject of the new google books api that was unveiled during the google io conference last week, jonathan rochkind states: once you have an api key, it can keep track of # requests for that key — it’s not clear to me if they rate limit you, and … creating a marc record from scratch in php using file_marc in the past couple of days, two people have written me email essentially saying: "dan, this file_marc library sounds great - but i can't figure out how to create a record from scratch with it! can you please help me?" yes, when you're dealing with marc, you'll quickly get all weepy … access conference in beautiful british columbia the official announcement for the canadian library association (cla) emerging technology interest group (etig)-sponsored access conference for went out back in november, announcing vancouver, british columbia, as the host. note that the schedule has changed from its original dates to october - ! i've told a number of people … troubleshooting ariel send and receive functionality i'm posting the following instructions for testing the ports required by ariel interlibrary loan software. i get requests for this information a few times a year, and at some point it will be easier to find on my blog than to dig through my email archives from over years … chilifresh-using libraries: are you violating copyright? when i was preparing my access presentation about social sharing and aggregation in library software, i came across chilifresh, a company that aggregates reviews written by library patrons from across libraries that subscribe to the company's review service. i was a bit disappointed to see that the service almost … on avoiding accusations of forking a project sometimes forking a project is necessary to reassert community control over a project that has become overly dominated by a single corporate rules: see openindiana and libreoffice for recent examples. and in the world of distributed version control systems, forking is viewed positively; it's a form of evolution, where experimental … library hackers want you to throw down the gauntlet on october th, a very special event is happening: the access hackfest. a tradition since access , the hackfest brings together library practitioners of all kinds to tackle challenges and problems from the mundane to the sublime to the ridiculous. if you can imagine a spectrum with three axes, you … file_marc . . - now offering two tasty flavours of marc-as-json output i've just released the php pear library file_marc . . . this release brings two json serialization output methods for marc to the table: tojsonhash() returns json that adheres to bill dueber's proposal for the array-oriented marc-hash json format at new interest in marc-hash json tojson() returns json that adheres … in which i perceive that gossip is not science marshall breeding published the results of his international survey of library automation a few days ago. juicy stuff, with averages, medians, and modes for the negative/positive responses on a variety of ils and vendor-related questions, and some written comments from the respondents. one would expect the library geek … pkg_check_modules syntax error near unexpected token 'deps,' the next time you bash your brains against autotools for a while wondering why your perfectly good pkg_check_modules() macro, as cut and paste directly from the recommended configure.ac entry for the package you're trying to integrate (in this case libmemcached), and you get the error message pkg_check_modules syntax error … marc library for c# coders c# isn't in my go-to list of programming languages, but i can understand why others would be interested in developing applications in c#. so it's good news to the c# community of library developers (it would be interesting to find out how many of you are out there) that there … doing useful things with the txt dump of sfx holdings, part : database there must be other people who have much more intelligent things than me with the txt dump of sfx holdings that you can generate via the web administration interface, but as i've gone through this process at least twice and rediscovered it each time, perhaps i'll save myself an hour … transparent acquisitions budgets and expenditures for academic libraries in my most recent post over at the academic matters site, after a general discussion about "new books lists" in academic libraries, i tackle one of the dirty laundry areas for academic libraries: exposing how collection development funds are allocated to departments. here's a relevant quote: for - , we decided … making skype work in a windows xp virtualbox guest instance if you, like me, install skype in a windows xp virtualbox guest instance running on an ubuntu host on a thinkpad t with an intel dual-core -bit processor, it might throw windows exceptions and generate error reports as reported in virtualbox ticket # . if you then go into your … in which my words also appear elsewhere i'm excited to announce the availability of my first post as an invited contributor to the more than bookends blog over at the revamped academic matters web site. my fellow contributors are anne fullerton and amy greenberg, and i'm delighted to be included with them in our appointed task of … presentation: libx and zotero direct link to the instructional presentation on libx and zotero at laurentian university (odt) (pdf) i had the pleasure of giving an instructional session to a class of graduate students on monday, november th. the topic i had been asked to present was an extended version of the artificially enhanced … archive of oclc worldcat policy as posted - - i noticed last night (sunday, november nd, ) that the new and much-anticipated / feared oclc worldcat policy had been posted. as far as the clarified terms went, i was willing to give them the benefit of the doubt until they were actually posted. i was first alerted to the freshly … dear dan: why is using flash for navigation a bad idea? i received the following email late last week, and took the time to reply to it tonight. i had originally been asked by a friend to help diagnose why his organization's site navigation wasn't working in some of his browsers. i noticed that the navigation bar was implemented in flash … boss me around, s'il vous plait my place of work, laurentian university, is looking for a new director of the j.n. desmarais library. the call for applications closes october th. i think our library has done some impressive work (participating in the food security project for the democratic republic of congo, building the mining environment … software freedom day - sudbury i opted to do something out of the unusual (for me) this year when i learned about software freedom day; i signed up to organize an event in sudbury. given everything that was already on my plate, it was pure foolishness to do so - but it was also important to … in which digital manifestations of myself plague the internets over the past few months, i've been fortunate enough to participate in a few events that have been recorded and made available on the 'net for your perpetual amusement. well - amusing if you're a special sort of person. following are the three latest such adventures, in chronological order: couchdb: delicious … test server strategies occasionally on the #openils-evergreen irc channel, a question comes up what kind of hardware a site should buy if they're getting serious about trying out evergreen. i had exactly the same chat with mike rylander back in december, so i thought it might be useful to share the strategy we … inspiring confidence that my problem will be solved hmm. i think i'm in trouble if the support site itself is incapable of displaying accented characters properly. corrupted characters in a problem report about corrupted characters. oh dear. my analysis of the problem is that the content in the middle is contained within a frame, and is actually encoded … couchdb: delicious sacrilege well, the talk about couchdb (an open-source document database similar in concept to lotus notes, but with a restful api and json as an interchange format) wasn't as much of a train wreck as it could have been. i learned a lot putting it together, and had some fun with … oooh... looks like i've got (even more) work cut out for me php is getting a native doubly-linked list structure. this is fabulous news; when i wrote the file_marc pear package, i ended up having to implement a linked list class in pear to support it. file_marc does its job today (even though i haven't taken it out of alpha yet), but … geek triumph what a night. i upgraded serendipity, dokuwiki, drupal, involving four different servers and three different linux distros, and shifted one application from one server to another (with seamless redirects from the old server to the new) with close to no downtime. i think this is the first time i've completed … a chance to work at laurentian university library hey folks, if you're interested in working at laurentian university, we've got a couple of tenure-track positions looking for qualified people who can stand the thought of working with me... (nothing like narrowing the field dramatically, ah well). the following position descriptions are straight out of the employment vacancies page … ariel: go back to your room, now! i've been working on automating the delivery of electronic documents to our patrons; most of the work over the summer was spent in ensuring that we had our legal and policy bases covered. i read through the documentation for ariel, our chosen ill software, to ensure that everything we wanted … "a canonical example of a next-generation opac?" ooh, yes, i remember writing that now. not about evergreen, which has book bags and format limiters and facets and whiz-bangy unapi goodness whose potential for causing mayhem has barely been scratched - but about fac-back-opac, the django-and-solr-and-jython beast that mike beccaria and i picked up from casey durfee's scraps pile … the pain: discovery layer selection i returned from a week of vacation to land solidly in the middle of a discovery layer selection process -- not for our library, yet, but from a consortial perspective clearly having some impact on possible decisions for us further on down the road. as the systems librarian, i was nominated … access draft program is online! i had been getting anxious about the lack of news on the access conference front, but just saw in my trusty rss feed that the draft program schedule is now available. i'm already looking forward to jessamyn west's opening keynote and roy tennant's closing keynote. they always bring … evergreen vmware image available for download after much iteration and minor bug-squashing in my configuration, i am pleased to announce the evergreen on gentoo vmware image is available for download. the download itself is approximately mb as a zipped image; when you unzip the image, it will require approximately gb of disk space. ( ) basic instructions … in which i make one apology, and two lengthy explanations i recently insulted richard wallis and rob styles of talis by stating on dan chudnov's blog: to me it felt like talis was in full sales mode during both richard's api talk and rob's lightning talk i must apologize for using the terms "sales mode" and "sales pitch" to describe … facbackopac: making casey durfee's code talk to unicorn for the past couple of days, i've been playing with casey durfee's code that uses solr and django to offer a faceted catalogue. my challenge? turn a dense set of code focused on dewey and horizon ils into a catalogue that speaks lc and unicorn. additionally, i want it to … lightning talk: file_marc for php i gave a lightning talk at the code lib conference today on “file_marc for php” introducing the file_marc library to anybody who hasn't already heard about it. i crammed nine slides of information into five minutes, which was hopefully enough to convince people to start using it and provide feedback on … google summer of code lib? google just announced that they will start accepting applications in march for the google summer of code (gsoc) . in , over organizations participated in the gsoc, and google expects to have a similar number participating in . there are no lack of potential open-source development projects in the … long time, no wild conjecture so here's the first of two posts based on purely wild conjecture. in a lengthening chain of trackbacks, ryan eby mentioned christina's observation that springlink has started displaying google ads, presumably to supplement their subscription and pay-per-article income. ryan goes on to wonder: will vendors continue with the subscription model … a short-term sirsidynix prediction the second of tonight's wild conjecture-based predictions. one of the things that i was thinking about as i was shovelling the snow off our driveway on monday (other than yes! finally some snow... one of these days amber is going to go rolling around in it) was the position that … reflections at the start of was a year full of change - wonderful, exhausting change. here's a month-by-month summary of the highlights of : january i did a whole lot of work on the pecl ibm_db extension, reviewed a good book on xml and php, and finally fixed up my blog a little bit. i've … oh, vista has _acquired_ sirsidynix... a little over a week ago, i made the following prediction following the extremely under-the-radar press release on december nd that vista equity partners was investing in sirsidynix: i'll go out on a limb and say that a merger or acquisition of sirsidynix in is unlikely ( % confidence), but … musing about sirsidynix's new investment partner sirsi corporation merged with dynix corporation in june . now sirsidynix has announced that vista equity partners is investing in their company. let's take a look at vista's investment philosophy: *we invest in companies that uniquely leverage technology to deliver best-of-class products or services.* i wonder if vista confused "most … save your forehead from flattening prematurely i gave up on trying to get ubuntu . (edgy eft) to run ejabberd today; it looks like there are some fundamental issues going on between the version of erlang and the version of ejabberd that get bundled together. that was a fairly serious setback to my "evergreen on … bibliocommons wireframe walk-through after the future of the ils symposium wrapped up, beth jefferson walked some of us through the current state of the bibliocommons mocked-up web ui for public library catalogs; the project grew out of a youth literacy project designed to encourage kids to read through the same sort of social … future of the ils symposium: building our community and a business case i headed down to windsor early on tuesday morning for the future of the ils symposium hosted by the leddy library at the university of windsor. it was a good thing i decided to take the hours of bus + train approach to getting there, as sudbury's airport was completely … neat-o: archimède uses apache derby a while back i mentioned on the dspace-devel mailing list that i was interested in adapting dspace to use embedded apache derby as the default database, rather than postgresql, as a means of lowering the installation and configuration barriers involved with setting up access to an external database. i haven't … pear file_marc . . alpha officially released just a short note to let y'all know that i received the thumbs-up from my fellow pear developers to add file_marc as an official pear package. what does this mean? well, assuming you have php . + and pear installed, you can now download and install file_marc and its prerequisite … belated access notes: saturday, oct. th final entry in publishing my own hastily jotted access conference notes--primarily for my own purposes, but maybe it will help you indirectly find some real content relating to your field of interest at the official podcast/presentation web site for access . contents include: consortial updates from asin, quebec … getting the goods: libraries and the last mile in my continuing series of publishing my access notes, roy tennant's keynote on finishing the task of connecting our users to the information they need is something to which every librarian should pay attention. if you don't understand something i've written, there's always the podcast of roy's talk. in … access notes: october my continuing summaries from access . thursday, october th was the first "normal" day of the conference featuring the following presentations: open access, open source, content deals: who pays? (leslie weir) our ontario: yours to recover (art rhyno, walter lewis) improving the catalogue interface using endeca (tito sierra) lightning talks … library geeks in human form so, i think i read somewhere on #code lib that dan chudnov, the most excellent host of the library geeks podcast, refused to make human-readable links to the mp files for the podcasts available in plain old html because he had bought into the stodgy old definition of podcasts (hah! "stodgy … double-barreled php releases i'm the proud parent of two new releases over the past couple of days: one official pear release for linked list fans, and another revision of the file_marc proposal for library geeks. structures_linkedlist a few days ago marked the first official pear release of the structures_linkedlist. yes, it's only at … feeling sorry for our vendor so i'm here in rainy alabama (the weather must have followed me from ottawa) taking a training course from our ils vendor. i'm getting some disturbing insights into the company that are turning my general state of disbelief at the state of the system that we're paying lots of money … backlog of access notes following on my plea for access to access presentations, i'm in the process of posting the notes i took at the carl instutitional repository pre-conference and access . i probably should have posted these to a wiki so that others (like the presenters) could go ahead and make corrections/additions … calling for access to all future access presentations it's a bit late now, but as the guy in the corner with the clicky keyboard desperately trying to take notes during the presentations (when not stifling giggles and snorts from #code lib), i would be a lot more relaxed if i was certain that the presentations were going to be … secretssss of free wifi at access the bulk of the access conference is being held at a hotel-that-shall-not-be-named-for-reasons-that-will-become-apparent-shortly in ottawa this week. i was at the carl pre-conference on institutional repositories today and a kind man (wayne johnston from the university of guelph) tipped me off that the hotel's pay-for-wifi system is a little bit … laundry list systems librarians on the always excellent techessence, dorothea salo posted hiring a systems librarian. the blog post warned against libraries who put together a “laundry-list job description” for systems librarians: sure, it'd be nice to have someone who can kick-start a printer, put together a desktop machine from scraps, re-architect a website … file_marc and structure_linked_list: new alpha releases earlier in the month i asked for feedback on the super-alpha marc package for php. most of the responses i received were along the lines of "sounds great!" but there hasn't been much in the way of real suggestions for improvement. in the mean time, i've figured out (with lukas … super-alpha marc package for php: comments requested okay, i've been working on this project (let's call it pear_marc, although it's not an official pear project yet) in my spare moments over the past month or two. it's a new php package for working with marc records. the package tries to follow the pear project standards (coding, documentation … google forms - create and analyze surveys, for free. one account. all of google. sign in to continue to forms enter your email find my account sign in with a different account create account one google account for everything google about google privacy terms help storming of the united states capitol - wikipedia storming of the united states capitol from wikipedia, the free encyclopedia jump to navigation jump to search violent incident and riot in washington, d.c. on january a request that this article title be changed to united states capitol attack is under discussion. please do not move this article until the discussion is closed. for a chronological guide to this subject, see timeline of the storming of the united states capitol. storming of the united states capitol part of the – united states election protests and attempts to overturn the united states presidential election clockwise from top: protesters gathering outside the capitol; donald trump speaking to supporters at the "save america" rally; a gallows that was erected by pro-trump protesters outside the united states capitol building; crowd is appearing to retreat from tear gas; tear gas being deployed outside the capitol building; a crowd pressing in to the capitol at the eastern entrance date january , ( - - ) location united states capitol, washington, d.c. ° ′ . ″n ° ′ . ″w / . °n . °w / . ; - . coordinates: ° ′ . ″n ° ′ . ″w / . °n . °w / . ; - . caused by opposition to the results of the united states presidential election[ ] donald trump's and his allies' false claims of presidential election fraud[ ][ ] right-wing extremism in the united states[ ] goals disrupt, delay, and overturn the electoral college vote count in favor of trump seize and destroy the certificates of ascertainment of the electoral college votes[ ] pressure congress and vice president mike pence to overturn the election of joe biden methods demonstration civil disorder: rioting,[ ] vandalism, [ ] looting,[ ] assault,[ ] arson[ ] political subversion: propaganda (big lie),[ ] conspiracy,[ ] agitation,[ ] intimidation,[ ] obstruction of justice, legislature takeover, terrorism[ ] (bombing; alleged: hostage-taking and lynching)[ ] resulted in failure of trump supporters: delay of counting electoral votes by several hours;[ ] resumption of presidential transition leading up to the inauguration of joe biden second impeachment of trump;[ ] suspensions of trump's social media accounts[ ] resignations of trump administration officials[ ][ ] elaine chao, secretary of transportation betsy devos, secretary of education tyler goodspeed, chief of the council of economic advisers stephanie grisham, chief of staff to the first lady anna cristina niceta lloyd, white house social secretary matthew pottinger, deputy national security advisor mick mulvaney, special envoy for northern ireland resignations of capitol security officials[ ] steven sund, chief of the capitol police michael c. stenger, sergeant at arms of the senate paul d. irving, sergeant at arms of the house of representatives suspensions of capitol police officers on suspicion of involvement[ ] installation of new security fencing around the capitol and heightened security measures[ ] casualties and criminal charges death(s) [ ] rioters[ ][ ][ ] police officer[ ][ ] injuries unknown number of rioters injured, at least rioters hospitalized[ ] d.c. metro police officers injured[ ] almost capitol police officers injured,[ ] including at least hospitalized[ ] damage undetermined;[ ] ransacked offices and chambers; property stolen[ ] charged + identified and investigated for domestic terrorism, seditious conspiracy, attempted assassination,[ ] and insurrection[ ][ ][ ] criminally charged[ ] , + digital media tips received[ ][ ] the storming of the united states capitol was a riot and violent attack against the th united states congress at the united states capitol on january , . part of the – united states election protests, it was carried out by a mob of supporters of donald trump, the th president of the united states, in a failed attempt to overturn the united states presidential electio in the united states presidential election.[ ] the capitol was placed under lockdown while lawmakers were evacuated. five people died from the event, while dozens more were injured.[ ] called to action by trump,[ ] thousands[ ] of his supporters gathered in washington, d.c., on january and in support of the stolen election conspiracy theory, trump's false claims that the election had been "stolen" from him,[ ][ ] and to demand that vice president mike pence and congress reject joe biden's victory.[ ] on the morning of january , at a "save america" rally on the ellipse, trump repeated false claims of election irregularities[ ] and spoke of a need to "fight", saying "if you don’t fight like hell, you’re not going to have a country anymore".[ ][ ]: : : at the president's encouragement,[ ] thousands of the protesters then walked to the capitol, where a joint session of congress was beginning the electoral college vote count to formalize biden's victory. many of the crowd at the capitol, some of whom had gathered earlier, breached police perimeters and stormed the building.[ ][ ] these rioters occupied, vandalized, and looted[ ] parts of the building for several hours.[ ] many became violent, assaulting capitol police officers and reporters, erecting a gallows on the capitol grounds, and attempting to locate lawmakers to take hostage and harm. they chanted "hang mike pence",[ ] blaming him for not rejecting the electoral college votes, although he lacked the constitutional authority to do so.[ ] the rioters targeted house speaker nancy pelosi (d–ca),[ ][ ] vandalizing and looting[ ] her offices, as well as those of other members of congress.[ ] upon security being breached, capitol police evacuated the senate and house of representatives chambers. several buildings in the capitol complex were evacuated, and all were locked down.[ ] rioters occupied and ransacked the empty senate chamber while federal law enforcement officers drew handguns to defend the evacuated house floor.[ ][ ] improvised explosive devices were found near the capitol grounds, as well as at offices of the democratic national committee, the republican national committee, and in a nearby vehicle.[ ][ ] trump initially resisted sending the d.c. national guard to quell the mob.[ ] in a twitter video, he called the rioters "very special" and told them to "go home in peace" while repeating his false election claims.[ ][ ] the capitol was cleared of rioters by mid-evening,[ ] and the counting of the electoral votes resumed and was completed in the early morning hours. pence declared president-elect biden and vice-president-elect kamala harris victors and affirmed that they would assume office on january . pressured by his administration, the threat of removal, and numerous resignations, trump later committed to an orderly transition of power in a televised statement.[ ][ ] the assault on the capitol was widely condemned by political leaders and organizations in the united states and internationally. mitch mcconnell (r–ky), senate minority leader, called the storming of the capitol a "failed insurrection"[ ] and said that the senate "will not bow to lawlessness or intimidation".[ ] several social media and technology companies suspended or banned trump's accounts from their platforms,[ ][ ] and many business organizations cut ties with him. a week after the riot, the house of representatives commenced the second impeachment of donald trump for incitement of insurrection, making him the only u.s. president to have been impeached twice.[ ] opinion polls showed that a large majority of americans disapproved of the storming of the capitol and of trump's actions leading up to and following it, although many republicans supported the attack or at least did not blame trump for it.[ ] as part of investigations into the attack, the federal bureau of investigation (fbi) opened more than subject case files and more than grand jury subpoenas and search warrants were issued.[ ] at least people have been arrested and charged with crimes.[ ] dozens of people present at the riot were later found to be listed in the fbi's terrorist screening database, most as suspected white supremacists.[ ] members of the anti-government paramilitary oath keepers and neo-fascist proud boys groups were indicted on conspiracy charges for allegedly staging planned missions in the capitol.[ ][ ][ ] contents background . planning of the rally . . funding and donations . prior intelligence and concerns of violence . january planning meeting . police preparations . national guard restricted events in washington, d.c. january , , . events before the "march to save america" . "save america" rally . . donald trump's speech . rioting in the capitol building . . pennsylvania avenue march . . capitol breach . . senate adjourned and evacuated . . house recessed and evacuated . planting of pipe bombs . participating groups . . state lawmakers . trump's conduct during the riot . congressional conduct during the riot . law enforcement and national guard response . completion of electoral vote count casualties and damage . deaths and injuries . damage, theft, and impact . . technology theft and cybersecurity concerns . . conservation of items damaged or left behind reactions . domestic reactions . international reactions . terminology used to refer to event aftermath . investigations and prosecutions . . impeachment of trump events outside washington, d.c. . state capitols and cities . international see also notes references external links background president donald trump during a campaign rally in while there had been other instances of violence at the capitol in the th and th centuries, the event was the most severe assault since the burning of washington by the british army during the war of .[ ][ ][ ] in the november , presidential election, democratic candidate joe biden defeated the incumbent republican president donald trump in both the popular vote (biden received . million votes, or . %, to trump's . million, or . %) and the electoral college vote (biden won to ).[ ] [ ][ ] the results became clear four days after election day, after the vote had been tallied.[ ] before, during, and after the counting of votes, trump and other republicans attempted to overturn the election, falsely claiming widespread voter fraud in five swing states that biden won: arizona, georgia, michigan, pennsylvania, and wisconsin.[ ] these attempts to overturn the election have been characterized by some as an attempted coup d'état[ ] and an implementation of the "big lie".[ ] after the election, trump waged a -day campaign to subvert the election, first through legal challenges and then (once those failed) through an extralegal effort.[ ] although trump's lawyers concluded within ten days after the election that legal challenges to the election results had no factual basis or legal merit,[ ] trump sought to overturn the results by filing at least lawsuits, including two brought to the supreme court, that sought to nullify election certifications and void votes cast for biden in each of the five states; these challenges were all rejected by the courts for lack of evidence or standing.[ ] trump then mounted a campaign to pressure republican governors and secretaries of state, and republican-controlled state legislatures, to nullify results, by replacing slates of biden electors with those declared to trump, or manufacturing evidence of fraud (which would likely violate election tampering statutes enacted by the states); and demanding lawmakers investigate supposed election "irregularities", such as by conducting signature matches of mail-in ballots (regardless of efforts already undertaken). trump also personally inquired about invoking martial law to "re-run" or reverse the election, which would be illegal and unconstitutional,[ ][ ] and appointing a special counsel to find incidences of fraud (even though federal and state officials have concluded that such cases were very isolated or non-existent); trump ultimately undertook neither step.[ ] congress was scheduled to meet on january , , to count the results of the electoral college vote and certify the winner, typically a ceremonial affair.[ ][ ] trump had spent previous days suggesting that vice president mike pence should stop biden from being inaugurated, which is not within pence's constitutional powers as vice president and president of the senate. trump repeated this call in his rally speech on the morning of january .[ ] the same afternoon, pence released a letter to congress in which he said he could not challenge biden's victory.[ ][ ] planning of the rally on december , four days after the electoral college voted, trump called for supporters to attend a rally before the january congressional vote count, to continue his challenge to the validity of several states' election results. trump tweeted, "big protest in d.c. on january th. be there, will be wild!"[ ][ ] the "march to save america" and rally that preceded the riots at the capitol were initially organized by women for america first, a (c)( ) organization chaired by amy kremer, co-founder of women for trump.[ ] they obtained a permit with an estimated attendance of , .[ ] in late and early , kremer organized and spoke at a series of events across the country as part of a bus tour to encourage attendance at the january rally and support trump's efforts to overturn the election result.[ ] women for america first invited its supporters to join a caravan of vehicles traveling to the event.[ ] event management was carried out by event strategies, a company founded by tim unes, who worked for trump's presidential campaign.[ ] on january , trump retweeted a post by kremer promoting the january rally, adding that he would be there. from that point, although kremer still held the permit, planning essentially passed to the white house.[ ] according to the new york times, trump discussed the speaking lineup and the music to be played at the event. although the initial plan for the rally called for people to remain at the ellipse until the counting of electoral slates was complete, the white house said they should march to the capitol, as trump repeatedly urged during his speech.[ ] play media ali alexander on december , , promoting the objection to counting electoral votes for biden on january , ali alexander, a right-wing political activist who took part in organizing the rally and expressed support for the storming as "completely peaceful", was reported as saying in december that representatives paul gosar (r–az), andy biggs (r–az) and mo brooks (r–al) were involved in the planning of "something big".[ ] "we're the four guys who came up with a january event," he said.[ ] according to alexander, "it was to build momentum and pressure and then on the day change hearts and minds of congress peoples who weren't yet decided or who saw everyone outside and said, 'i can't be on the other side of that mob.'" his remarks received more scrutiny after the events of january , causing biggs to respond with a statement denying any relationship with alexander.[ ][ ][ ] the washington post wrote that videos and posts revealed earlier connections between alexander and the three members of congress.[ ] the rioters openly planned to disrupt the counting of electoral college ballots for several weeks prior to the event, and called for violence against congress, pence, and law enforcement.[ ] plans were coordinated on "alt-tech" platforms distinct from larger social media platforms such as reddit or twitter, which had implemented bans to censor violent language and images. websites such as thedonald.win (a successor to the reddit forum r/the_donald), social networking service parler, chat app telegram, gab, and others, were used to discuss previous trump rallies and made plans for storming the capitol.[ ][ ] many of the posters planned for violence prior to the event, with some individuals discussing how to avoid police on the streets, which tools to bring to help pry open doors, and how to smuggle weapons into washington d.c.[ ] there was also discussion of their perceived need to attack law enforcement.[ ] following clashes with washington d.c. police during protests on december , , the proud boys and other far-right groups turned against supporting law enforcement.[ ] at least one group, stop the steal, posted on december , , its plans to occupy the capitol with promises to "escalate" if met with opposition from law enforcement.[ ] discussions on multiple sites talked very graphically and explicitly about "war", physically taking charge at the event, and killing politicians, including a request for opinions about which politician should be hung first (with a gif of a noose).[ ] one of the comments cited in the fbi memo declared trump supporters should go to washington and get 'violent. stop calling this a march, or rally, or a protest. go there ready for war. we get our president or we die.'...'[i]t is our duty as americans to fight, kill and die for our rights.'[ ] funding and donations rioters outside the capitol organizations that officially participated in the event include: black conservatives fund, eighty percent coalition, moms for america, peacably gather, phillis schlafly eagles, rule of law defense fund, stop the steal, turning point action, tea party patriots, women for america first, and wildprotest.com.[ ] rule of law defense fund, which is a (c)( ) arm of the republican attorneys general association, also paid for robocalls to invite people to "march to the capitol building and call on congress to stop the steal".[ ] conspiracy theorist alex jones's media company paid $ , to book the ellipse for the event,[ ][ ] of which $ , was donated by publix heiress and prominent trump donor julie jenkins fancelli.[ ] jones claimed that the trump white house asked him to lead the march to the capitol.[ ] charlie kirk said on twitter that its political action committee turning point action and students for trump had sent over buses to the capitol.[ ] roger stone recorded a video for stop the steal security project to raise funds "for the staging, the transportation and most importantly the security" of the event.[ ] other people attempted to raise funds in december via gofundme to help pay for transportation to the rally, with limited success.[ ] an investigation by buzzfeed news identified more than a dozen fundraisers to pay for travel to the planned rally. gofundme subsequently deactivated several of the campaigns after the riot, but some campaigns had already raised part or all of their fundraising goals prior to deactivation.[ ] prior intelligence and concerns of violence in the days leading up to the storming, several organizations that monitored online extremism had been issuing warnings about the event.[ ] on december , , a u.k. political consultant who studies trump-related extremism tweeted a forecast of what the planned event of january would become, including deaths.[ ] on december , , the federal bureau of investigation (fbi) issued at least one bulletin to law enforcement agencies across the country, warning of the potential of armed protesters targeting legislatures.[ ] prior to january , , the local joint terrorism task force was notified by the fbi of possible impending violence at the capitol.[ ] the washington post reported an internal fbi document on january warned of rioters preparing to travel to washington, and setting up staging areas in various regional states. the document used the term "war" to describe the rioters' motive, which mentioned specific violence references, including the blood of black lives matter and antifa members.[ ] however, the fbi decided not to distribute a formal intelligence bulletin.[ ][ ] some security specialists later reported they had been surprised that they hadn't received information from the fbi and dhs before the event. later reflections about the intelligence failures revealed that surprise that no threat assessments had been issued, with possible causes for the failure related to dhs personnel changes and law enforcement biases.[ ] the anti-defamation league published a january blog post warning about violent rhetoric being espoused by trump supporters leading up to the electoral college vote count, including calls to violently disrupt the counting process. the post said that it was not aware of any credible threats of violence, but noted that "if the past is any indication, the combination of an extremist presence at the rallies and the heated nature of the rhetoric suggests that violence is a possibility."[ ][ ] also on january , british security firm g s conducted a risk analysis, which found that there would be violent groups in washington, d.c., between january and inauguration day based on online posts advocating for violence.[ ][ ] advance democracy, inc., a nonpartisan governance watchdog, found , posts from accounts related to qanon that referenced the events of january in the six days leading up to it, including calls for violence.[ ] after the event members of the oath keepers anti-government paramilitary group were indicted on conspiracy charges for allegedly staging a planned mission in the capitol.[ ][ ][ ] january planning meeting the evening of january th, trump's closest allies had a meeting at the trump international hotel in washington d.c.[ ] tommy tuberville has since stated that he did not attend the meeting[ ] but evidence appears to show otherwise.[ ][ ] police preparations washington, d.c., mayor muriel bowser requested on december , , that district of columbia national guard troops be deployed to support local police during the anticipated demonstrations. in her request, she wrote that the guards would not be armed and that they would be primarily responsible for "crowd management" and traffic direction, allowing police to focus on security concerns. acting secretary of defense christopher c. miller approved the request on january , . the approval activated troops, with no more than to be deployed at any given time.[ ] three days before the riots, the department of defense twice offered to deploy the national guard to the capitol, but were told by the united states capitol police that it would not be necessary.[ ] three days before the storming, a -page report from the capitol police's intelligence unit described that congress would be the target on the day of the electoral college vote counting, but the report was apparently not shared widely.[ ] two days before the storming, bowser announced that the metropolitan police department of the district of columbia (mpd) would lead law enforcement for the event, and would be coordinating with the capitol police, the u.s. park police, and the secret service.[ ] (jurisdictionally, mpd is responsible for city streets of the national mall and capitol area, whereas the park police are responsible for the ellipse (the site of trump's speech and rally that day), the secret service is responsible for the vicinity of the white house, and the capitol police is responsible for the capitol complex itself).[ ] "to be clear, the district of columbia is not requesting other federal law enforcement personnel and discourages any additional deployment without immediate notification to, and consultation with, mpd if such plans are underway," bowser wrote in a letter to the united states department of justice, and reproduced this language in a tweet.[ ] also on january , capitol police chief steven sund was refused additional national guard support by the then-house sergeant-at-arms paul d. irving and then-senate sergeant-at-arms michael c. stenger.[ ][ ] the fbi spoke to over a dozen known extremists and "was able to discourage those individuals from traveling to d.c.", according to a senior fbi official. the fbi shared information with the capitol police in advance of the protest.[ ] robert contee, the acting chief of the metropolitan police department of the district of columbia, said after the event that his department had possessed no intelligence indicating the capitol would be breached.[ ] capitol police chief sund said his department had developed a plan to respond to "first amendment activities" but had not planned for the "criminal riotous behavior" they encountered.[ ] as a result, capitol police staffing levels mirrored that of a normal day and officers did not prepare riot control equipment.[ ] u.s. secretary of the army ryan d. mccarthy said law enforcement agencies' estimates of the potential size of the crowd, calculated in advance of the event, varied between , and , .[ ] on january , the national park service estimated that , people would attend the "save america" rally, based on people already in the area.[ ] national guard restricted days after the election, on november , donald trump fired defense secretary mark esper, replacing him with christopher c. miller. in a january memo, miller prohibited deploying d.c. guard members with weapons, helmets, body armor or riot control agents without his personal approval.[ ] on january , secretary of the army ryan mccarthy issued a memo placing limits on the district of columbia national guard.[ ] maj. gen. william j. walker, the commanding general of the dc national guard, later explained "all military commanders normally have immediate response authority to protect property, life, and in my case, federal functions — federal property and life. but in this instance i did not have that authority."[ ] events in washington, d.c. january , , for a detailed timeline of the events in washington d.c. and its aftermath, see timeline of the storming of the united states capitol. further information: big lie § donald trump's claim of a stolen election events before the "march to save america" on january , several events related to overturning the election took place in or around the national mall in washington, d.c. cindy chafian, who founded the eighty percent coalition, organized the "rally to revival",[ ] which was permitted to take place at freedom plaza including a "rally to save america".[ ] on the same day, the "save the republic rally" was organized by moms for america in the early afternoon at area across from the russell senate office building;[ ] and the "one nation under god" rally, which was organized by virginia women for trump, stop the steal, american phoenix project, and jericho march, took place near the united states supreme court.[ ] "the silent majority" rally was organized by james epley and permitted in the north inner gravel walkway between th and th streets within the national mall.[ ] epley's events took place on january and . at least ten people were arrested, several on weapons charges, on the night of january and into the morning of january .[ ] on january , the "wild protest" was organized by stop the steal and took place in area across from the russell senate office building.[ ] on the same day, the "freedom rally" was organized by virginia freedom keepers, latinos for trump, and united medical freedom super pac at first street ne, located across from the russell senate office building.[ ] "save america" rally protesters at washington union station in the morning on january wikisource has original text related to this article: donald trump's save america rally speech the "save america" rally (or “march to save america”) took place on january in the ellipse within the national mall.[ ] trump supporters gathered about a quarter of a mile north on the ellipse, where trump, his lawyer and adviser rudy giuliani, and others, such as chapman university school of law professor john c. eastman, gave speeches.[ ] giuliani repeated conspiracy theories that voting machines used in the election were "crooked" and called for "trial by combat".[ ][ ] representative mo brooks told the crowd, "today is the day american patriots start taking down names and kicking ass."[ ] representative madison cawthorn (r–nc) said, "this crowd has some fight."[ ] amy kremer told attendees "it is up to you and i to save this republic" and called on them to "keep up the fight".[ ] trump's sons, donald jr. and eric, also spoke, naming and verbally attacking republican congressmen and senators who were not supporting the effort to challenge the electoral college vote, and promising to campaign against them in future primary elections.[ ] donald trump's speech trump gave a speech from behind a glass barrier, declaring he would "never concede" the election, criticizing the media and calling for pence to overturn the election results, something outside pence's constitutional power.[ ][ ] his speech contained many falsehoods and misrepresentations that inflamed the crowd.[ ] trump did not overtly call on his supporters to use violence or enter the capitol,[ ] but his speech was filled with violent imagery,[ ] and trump suggested that his supporters had the power to prevent biden from taking office.[ ] stop the steal signs seen in front of the capitol trump called for his supporters to "walk down to the capitol" to "cheer on our brave senators and congressmen and women and we're probably not going to be cheering so much for some of them." he told the crowd that he'd be going with them. as to the counting of biden's electoral votes, trump said, "we can't let that happen" and suggested that biden would be an "illegitimate president."[ ][ ] referring to the day of the elections, trump said, "most people would stand there at : in the evening and say, 'i want to thank you very much,' and they go off to some other life, but i said, 'something's wrong here. something's really wrong. [it] can't have happened.' and we fight. we fight like hell and if you don’t fight like hell, you’re not going to have a country anymore."[ ]: : : he said the protesters would be "going to the capitol and we're going to try and give [republicans] the kind of pride and boldness that they need to take back our country".[ ] trump also said, "you'll never take back our country with weakness. you have to show strength and you have to be strong. we have come to demand that congress do the right thing and only count the electors who have been lawfully slated."[ ][ ] he denounced representative liz cheney (r-wy), stating that "we've got to get rid of the weak congresspeople, the ones that aren't any good, the liz cheneys of the world."[ ] he called upon his supporters to "fight much harder" against "bad people";[ ] told the crowd that "you are allowed to go by very different rules";[ ] said that his supporters were "not going to take it any longer";[ ] framed the moment as a last stand,[ ] suggested that pence and other republican officials put themselves in danger by accepting biden's victory;[ ] and told the crowd that he would march with them to the capitol (although he did not do so).[ ][ ] in addition to the twenty times he used the term "fight", trump once used the term "peacefully", saying, "i know that everyone here will soon be marching over to the capitol building to peacefully and patriotically make your voices heard."[ ] before trump had finished speaking, some supporters began moving up the national mall, telling others that they were storming the capitol.[ ] rioting in the capitol building pennsylvania avenue march proud boys marching in front of the u.s. supreme court building on january , by : a.m. and concurrent with events happening in and around the u.s. capitol, a rally of trump supporters filled the ellipse, which is located just south of the white house grounds and about . miles from the capitol. coming from the white house, trump addressed the rally, from noon to : p.m., encouraging this crowd to walk down pennsylvania avenue to the capitol; members of the crowd began walking toward the capitol "in a steady stream" before he finished speaking.[ ] at the end of his speech, trump returned to the white house. meanwhile, another crowd of trump supporters that had gathered outside the capitol began clashing with the police and pushing forward to the building. the crowd walking up pennsylvania avenue from the ellipse merged with the crowd at the capitol.[ ] a reliable estimate of the total size of the crowd cannot be ascertained, seeing that aerial photos are not permitted in washington, d.c., for reasons of security. the crowd was estimated to be in the thousands.[ ] on the capitol grounds, senator josh hawley (r-mo), a leader of the group of lawmakers who vowed to challenge the electoral college vote, greeted protesters with a raised fist as he passed by on his way to the joint session of congress in the early afternoon.[ ][ ] around : p.m. est, hundreds of trump supporters clashed with officers and pushed through barriers along the perimeter of the capitol.[ ][ ] federal officials estimate that about people entered the building.[ ] the crowd swept past barriers and officers, with some members of the mob spraying officers with chemical agents or hitting them with lead pipes.[ ][ ] although many rioters simply walked to the doors of the capitol, some resorted to ropes and makeshift ladders.[ ] representative zoe lofgren (d–ca), aware that rioters had reached the capitol steps, was unable to reach steven sund by phone; house sergeant-at-arms paul d. irving told lofgren the doors to the capitol were locked and "nobody can get in".[ ] a short time afterward, the capitol police requested reinforcements from the dc metropolitan police, who arrived within minutes.[ ] meanwhile, sund, at : p.m., called irving and stenger and asked them for an emergency declaration required to call in the national guard; they both told sund they would "run it up the chain". irving called back with formal approval an hour later.[ ] at : p.m., the on-scene capitol police commander declared a riot.[ ] at : p.m., capitol police officers removed a barricade on the northeast side of the capitol.[ ] capitol breach trump supporters crowding the steps of the capitol after displacing police shield wall preventing access just after : p.m., windows were broken through, and the mob breached the building[ ][ ] and entered the national statuary hall.[ ][ ][ ][ ] as rioters began to storm the capitol and other nearby buildings, some buildings in the complex were evacuated.[ ] outside the building, the mob constructed a gallows and tied a noose to it, punctured the tires of a police vehicle, and left a note saying "pelosi is satan" on the windshield.[ ] politico reported some rioters briefly showing their police badges or military identification to law enforcement as they approached the capitol, expecting therefore to be let inside; a capitol police officer told buzzfeed news that one rioter told him "[w]e're doing this for you" as he flashed a badge.[ ] concerned about the approaching mob, representative maxine waters (d-ca) called capitol police chief steven sund, who was not on capitol grounds but at the police department's headquarters. when asked what the capitol police were doing to stop the rioters, sund told waters, "we're doing the best we can" before the line went dead.[ ] a gallows built outside the capitol in many cases, those who stormed the capitol appeared to employ tactics, body armor and technology such as two-way radio headsets that were similar to those of the very police they were confronting.[ ] several rioters carried plastic handcuffs, possibly with the intention of using them to take hostages.[ ][ ][ ] some of the rioters carried confederate battle flags[ ][ ][ ][ ] or nazi emblems.[ ][ ][ ][ ] some rioters wore riot gear, including helmets and military-style vests. for the first time in u.s. history, a confederate battle flag was displayed inside the capitol.[ ][ ][ ][ ] christian imagery and rhetoric was prevalent. rioters carried crosses and signs saying, "jesus saves", and "jesus ". on the national mall, rioters chanted, "christ is king." one rioter who stormed into the building carried a christian flag. rioters referred to the neo-fascist proud boys as "god's warriors".[ ][ ] these were mainly neo-charismatic, prophetic christians who practice their faith outside of mainstream denominations, who believe that trump is the messiah, or that he was anointed by god to save christian americans from religious persecution.[ ] although a few evangelical leaders supported the riots,[ ] most condemned the violence and criticized trump for inciting the crowd.[ ] this criticism came from liberal christian groups such as the red-letter christians as well as evangelical groups who were generally supportive of trump.[ ][ ] this criticism did not affect evangelical support for donald trump. investigative journalist sarah posner, author of unholy: why white evangelicals worship at the altar of donald trump, argues that many white evangelical christians in the u.s. create an echo chamber whereby trump's missteps are blamed on the democratic party, leftists, or the mainstream media, the last of which being viewed as especially untrustworthy.[ ] senate adjourned and evacuated play media c-span broadcast of the senate going into recess after protesters infiltrate the capitol at the time, the joint session of congress – which had already voted to accept the nine electoral votes from alabama and three from alaska without objection – was split so that each chamber could separately consider an objection to accepting arizona's electoral votes that had been raised by representative paul gosar (r-az) and endorsed by senator ted cruz (r-tx). both chambers were roughly halfway through their two-hour debate on the motion.[ ][ ] officer eugene goodman (pictured at the inauguration of joe biden), who slowed down the mob while debate over the arizona electoral college votes continued, an armed police officer entered the senate chamber, positioned facing the back entrance of the chamber. pence handed the floor from senator kyrsten sinema (d-az) to senator james lankford (r-ok). moments later, pence was escorted out by members of the secret service. the rioters began to climb the stairs towards the senate chamber. a lone capitol police officer, eugene goodman, worked to slow the mob down as he radioed that they had reached the second floor. just steps from the still-unsealed senate chamber doors, the rioters instead followed the capitol police officer, leading them back away from the senate. banging could be heard from outside as people attempted to breach the doors. as lankford was speaking, the senate was gaveled into recess, and the doors were locked at : p.m. a minute later, the rioters reached the gallery outside the chamber.[ ][ ] a police officer carrying a semi-automatic weapon appeared on the floor and stood between senate majority leader mitch mcconnell and senate minority leader chuck schumer (d-ny).[ ] senator mitt romney (r-ut) exasperatedly threw up his hands and directly criticized several fellow republicans who were challenging president-elect biden's electoral votes, yelling to them, "this is what you've gotten, guys."[ ] several members of senate parliamentarian elizabeth macdonough's staff carried the boxes of electoral college votes and documentation out of the chamber to hidden safe rooms within the building.[ ][ ] due to security threat inside: immediately, move inside your office, take emergency equipment, lock the doors, take shelter. —capitol police alert[ ] trump tweeted that pence "didn't have the courage to do what should have been done" at : p.m. afterwards, trump followers on far-right social media called for pence to be hunted down, and the mob began chanting, "where is pence?" and "find mike pence!"[ ][ ][ ] outside, the mob chanted, "hang mike pence!",[ ] which some crowds continued to chant as they stormed the capitol;[ ][ ] at least three rioters were overheard by a reporter saying they wanted to find pence and execute him as a "traitor" by hanging him from a tree outside the building.[ ] all buildings in the complex were subsequently locked down, with no entry or exit from the buildings allowed. capitol staff were asked to move into offices and lock their doors and windows; those outside were advised to "seek cover".[ ] as the mob roamed the capitol, lawmakers, aides, and staff took shelter in offices and closets. aides to mitch mcconnell, barricaded in a room just off a hallway, heard a rioter outside the door "praying loudly", asking for "the evil of congress [to] be brought to an end".[ ] the rioters entered and ransacked the office of the senate parliamentarian.[ ] play media people inside the capitol being evacuated. staff and reporters inside the building were taken by secure elevators to the basement, and then to an underground bunker constructed following the attempted attack on the capitol. evacuees were redirected while en route after the bunker was also infiltrated by the mob.[ ] with senators still in the chamber, trump reached senator tommy tuberville (r-al) by phone and told him to do more to block the counting of biden's electoral votes. the call had to be cut off when the senate chamber was evacuated at : p.m.[ ][ ][ ][ ] after evacuation, the mob briefly took control of the chamber, with some armed and armored men carrying plastic handcuffs and some posing with raised fists on the senate dais that pence had left minutes earlier.[ ][ ] pence's wife karen pence, daughter charlotte pence bond, and brother greg pence (a member of the house; r–in) were in the capitol at the time it was attacked.[ ] as pence and his family were being escorted from the senate chamber to a nearby hideaway, they came within a minute of being visible to rioters on a staircase only feet away.[ ] sergeant-at-arms of the senate michael c. stenger accompanied a group of senators including lindsey graham (r-sc) and joe manchin (d-wv) to a secure location in a senate office building. once safe, the lawmakers were "furious" with stenger; graham asked him, "how does this happen? how does this happen?" and added that they "[are] not going to be run out by a mob."[ ] house recessed and evacuated meanwhile, in the house chamber around : p.m., while gosar was speaking, speaker pelosi was escorted out of the chamber. the house was gaveled into recess, but would resume a few minutes later.[ ][ ] amid the security concerns, representative dean phillips (d–mn) yelled, "this is because of you!" at his republican colleagues.[ ] the house resumed debate around : p.m. around : , when gosar finished speaking, the house went into recess again. the rioters had entered the house wing and were attempting to enter the speaker's lobby just outside the house chamber. lawmakers were still inside and being evacuated, with pelosi, kevin mccarthy and a few other individuals being taken to a "secure location".[ ][ ] with violence breaking out, capitol security advised the members of congress to take cover.[ ][ ] members of congress inside the house chamber were told to put on gas masks, as law enforcement had begun using tear gas within the building.[ ][ ][ ][ ][ ] play media video shot inside the house of representatives chamber with armed security blocking the doors abc news reported that shots were fired within the capitol.[ ][ ] an armed standoff took place at the front door of the chamber of the house of representatives: as the mob attempted to break in, federal law enforcement officers drew their guns inside[ ] and pointed them towards the chamber doors, which were barricaded with furniture.[ ] in a stairway, one officer fired a shot at a man coming toward him.[ ] photographer erin schaff said that, from the capitol rotunda, she ran upstairs, where rioters grabbed her press badge. police found her, and, as her press pass had been stolen, they held her at gunpoint before her colleagues intervened.[ ] panic buttons in at least two representatives' offices were found to have been torn out.[ ] staffers for representative ayanna pressley (d–ma) barricaded themselves in pressley's office. they attempted to call for help with panic buttons that had been previously installed and even used in safety drills, but "[e]very panic button in my office had been torn out — the whole unit", pressley's chief of staff told the boston globe.[ ] multiple rioters, using the cameras on their cell phones, documented themselves occupying the capitol and the offices of various representatives,[ ][ ][ ] storming the offices of speaker pelosi,[ ][ ] accessing secure computers and stealing a laptop.[ ] planting of pipe bombs two pipe bombs, planted the previous night around p.m.,[ ] were found within a few blocks of the capitol.[ ] a pipe bomb was discovered next to a building containing republican national committee (rnc) offices at around : p.m. about minutes later, another pipe bomb was found under a bush at the democratic national committee (dnc) headquarters.[ ][ ] the devices were of a similar design – about one foot in length, with end caps and wiring apparently attached to a timer, and containing an unknown powder and some metal.[ ][ ] they were safely detonated by bomb squads; police later said they were "hazardous" and could have caused "great harm".[ ] the fbi distributed a photo of the person who they believe planted the devices and offered a reward of up to $ , for information;[ ] by the end of the month, they doubled the amount of the promised reward.[ ][ ] sund told the washington post on january that he suspected the pipe bombs were intentionally placed to draw police away from the capitol;[ ] representative tim ryan (d–oh) echoed the sentiment in a virtual news conference on january , who said "we do believe there was some level of coordination ... because of the pipe bombs ... that immediately drew attention away from the breach that was happening".[ ][ ] participating groups among the many flags flown by rioters were the gadsden flag and women for trump. the mob that stormed the capitol consisted of some of trump's longtime and most fervent supporters, coming from across the united states.[ ] the mob included republican party officials and political donors, far-right militants, and white supremacists.[ ] some individuals came heavily armed. included in the group were some convicted criminals, including a man who had been released from a florida prison after serving a sentence for attempted murder.[ ] supporters of the boogaloo movement, the traditionalist worker party, the three percenters, the proud boys, the oath keepers, qanon, the groyper army, and national-anarchism, as well as neo-confederates and holocaust deniers, among others, were present during the riot, with some wearing emblematic gear. neo-nazi and völkisch-inspired neopagan apparel was also worn by some participants during the riots, including a shirt emblazoned with references to the auschwitz–birkenau concentration camp and its motto, arbeit macht frei (german for "work makes you free").[ ] after the storming of the capitol, two white nationalists known for racist and anti-semitic rhetoric streamed to their online followers a video posted on social media showing a man harassing an israeli journalist seeking to conduct a live report outside the building.[ ] some participants wore shirts bearing the abbreviation mwe, standing for " million wasn't enough", a reference to the number of jewish people who were killed in the holocaust.[ ] according to the fbi, the majority of individual participants in the riot who appeared on its terrorist watchlist "are suspected white supremacists."[ ] following the event, members of the nationalist social club, a neo-nazi street gang, detailed their participation in the storming and claimed the acts were the "beginning of the start of white revolution in the united states".[ ] an academic analysis reported in the atlantic found that percent of the people who were arrested for invading the capitol had no clear public connection to established far-right militias, known white-nationalist gangs, or any other known militant organizations. "the overwhelming reason for action, cited again and again in court documents, was that arrestees were following trump’s orders to keep congress from certifying joe biden as the presidential-election winner."[ ] they were older than participants in previous far-right violent demonstrations and more likely to be employed, with % being business owners. the researchers concluded that these "middle-aged, middle-class insurrectionists" represented "a new force in american politics—not merely a mix of right-wing organizations, but a broader mass political movement that has violence at its core and draws strength even from places where trump supporters are in the minority."[ ] the associated press reviewed public and online records of more than participants after the storming and found that many of them shared conspiracy theories about the presidential election on social media and had also believed other qanon and "deep state" conspiracy theories. additionally, several had threatened democratic and republican politicians before the storming.[ ] the event was described as "extremely online", with "pro-trump internet personalities" and fans streaming live footage while taking selfies.[ ][ ] some military personnel participated in the riot;[ ] the department of defense is investigating members on active and reserve duty who may have been involved in the riot.[ ][ ] an analysis by national public radio showed that nearly % of defendants charged in relation to the attack served in the military; in the general population, % of all american adults are veterans.[ ] police officers and a police chief from departments in multiple states are under investigation for their alleged involvement in the riot.[ ] as of january , at least law enforcement officers are suspected of participating in trump's pre-riot rally, or joining the capitol riots, or both.[ ] two capitol police officers were suspended, one for directing rioters inside the building while wearing a make america great again hat, and the other for taking a selfie with a rioter.[ ][ ] court charges filed by federal prosecutors against members of the oath keepers militia who stormed the capital indicated that the militiamen were updated via facebook messages on the location of lawmakers as they were evacuated, and relayed communications such as "we have about - of us. we are sticking together and sticking to the plan" and "all members are in the tunnels under capital seal them in. turn on gas".[ ][ ][ ] anti-vaccine activists and conspiracy theorists were also present at the rally.[ ] most notably, members of the right-wing tea party patriots-backed group america's frontline doctors, including founder simone gold and communications director john strand, were arrested in connection with the assault on the capitol.[ ][ ] some trump loyalists falsely claimed that antifa had staged the incident as a false flag operation to implicate trump supporters.[ ][ ][ ][ ] state lawmakers at least eighteen republican current and former state legislators were present at the event, including west virginia state senator mike azinger, nevada state assemblywoman annie black, virginia state senator amanda chase, maryland delegate daniel l. cox, alaska state representative david eastman, west virginia delegate derrick evans, colorado state representative-elect ron hanks, missouri state representative justin hill, arizona state representative mark finchem, virginia state delegate dave larock, michigan state representative matt maddock, pennsylvania state senator doug mastriano, illinois state representative chris miller,[ ] rhode island state representative justin k. price,[ ] and tennessee representative terri lynn weaver, as well as outgoing georgia state representative vernon jones (a former democrat who announced at the rally that he had joined the republican party), outgoing arizona state representative anthony kern, and former pennsylvania state representative rick saccone. weaver claimed to have been "in the thick of it" and evans filmed himself entering the capitol alongside rioters. all denied participating in acts of violence.[ ][ ][ ][ ] evans was charged by federal authorities on january with entering a restricted area;[ ] he resigned from the house of delegates the next day.[ ] amanda chase was censured by the virginia state senate for her actions surrounding the event;[ ] in response she filed a federal lawsuit against the senate.[ ] trump's conduct during the riot play media donald trump's statement during the conflict, two hours after the building was breached. the video was originally posted on twitter at : p.m. and shared on other social media before being removed from all platforms for violating various policies. trump, who had spent previous weeks promoting the "save america" rally,[ ] was "initially pleased" when his supporters breached the capitol and refused to intercede,[ ] but also "expressed disgust on aesthetic grounds" over the "low class" appearance of the supporters involved in the rioting.[ ] senator ben sasse (r-ne) said that senior white house officials told him that trump was "delighted" to hear that rioters were entering the capitol.[ ] staffers reported that trump had been "impossible to talk to throughout the day", and that his inability to deal with his election loss had, according to one staffer, made trump "out of his mind."[ ] concerned that trump may have committed treason through his actions, white house counsel pat cipollone reportedly advised administration officials to avoid contact with trump and ignore any illegal orders that could further incite the storming to limit their prosecutorial liability under the sedition act of .[ ] shortly after : p.m. est, as the riot was going on and after senators had been evacuated from the senate floor, trump phoned senators mike lee (r-ut) and tommy tuberville (r-al), asking them to make more objections to the counting of the electoral votes to try to overturn the election.[ ] at : p.m., as his supporters violently clashed with police at the capitol, trump tweeted, "please support our capitol police and law enforcement. they are truly on the side of our country. stay peaceful!"[ ] the washington post later reported that trump did not want to include the words "stay peaceful".[ ] trump was in the west wing of the white house at the time. a close adviser to trump said the president was not taking many phone calls. when trump watches television, the adviser explained, he will pause a recorded program to take a phone call, but "if it's live tv, he watches it, and he was just watching it all unfold."[ ] by : p.m., pressure was building on trump to condemn supporters engaged in the riots; trump's former communications director, alyssa farah, called upon him to "condemn this now" and wrote "you are the only one they will listen to."[ ] by : p.m., trump tweeted "i am asking for everyone at the u.s. capitol to remain peaceful. no violence! remember, we are the party of law & order – respect the law and our great men and women in blue," but did not call upon the crowd to disperse.[ ] by : p.m., a number of congressional republicans called upon trump to more specifically condemn violence and to call on his supporters to end the occupation of the capitol: house minority leader kevin mccarthy (r–ca) said that he had spoken to trump to ask him to "calm individuals down"; senator marco rubio (r-fl) issued a tweet telling trump that "it is crucial you help restore order by sending resources to assist the police and ask those doing this to stand down"; and representative mike gallagher (r–wi), in a video message, told trump to "call it off".[ ] in contrast to trump, who only called upon his supporters to "remain peaceful", pence called for the occupation of the capitol to end immediately.[ ] lindsey graham later told the washington post that "it took [trump] awhile to appreciate the gravity of the situation ... [he] saw these people as allies in his journey and sympathetic to the idea that the election was stolen".[ ] by : p.m., white house press secretary kayleigh mcenany said that the national guard and "other federal protective services" had been deployed.[ ] at : p.m. on national television, president-elect biden called for president trump to end the riot. at : p.m., trump issued a video message on social media that was later taken down by twitter, facebook and youtube. in it, he praised his supporters and repeated his claims of electoral fraud, saying: "this was a fraudulent election, but we can't play into the hands of these people. we have to have peace. so go home. we love you. you're very special. you've seen what happens. you see the way others are treated that are so bad and so evil. i know how you feel. but go home and go home in peace."[ ][ ] at : p.m., trump tweeted: "these are the things and events that happen when a sacred landslide election victory is so unceremoniously & viciously stripped away from great patriots who have been badly & unfairly treated for so long" and then issued a call: "go home with love & in peace. remember this day forever!"[ ][ ][ ] at : p.m., rudy giuliani placed a second call to lee's number and left a voicemail intended for tuberville urging him to make more objections to the electoral votes as part of a bid "to try to just slow it down". giuliani said: "i know they're reconvening at tonight, but it ... the only strategy we can follow is to object to numerous states and raise issues so that we get ourselves into tomorrow – ideally until the end of tomorrow."[ ] congressional conduct during the riot during the riots representative lauren boebert (r-co) posted on twitter some information about the police response and the location of members, including the fact that speaker pelosi had been taken out of the chamber, for which she has faced calls to resign for endangering members.[ ][ ] boebert responded that she was not sharing private information, since pelosi's removal was also being broadcast on tv.[ ] representative ayanna pressley left the congressional safe room for fear of other members there "who incited the mob in the first place".[ ] while sheltering for hours in the "safe room" – a cramped, windowless room where people sat within arms' length of each other – some republican congress members refused to wear facemasks, even when their democratic colleagues begged them to. during the following week, three democratic members tested positive for the coronavirus. an environmental health expert described the situation as a "superspreader" event.[ ] law enforcement and national guard response play media armed guards walking through the halls of congress after they were ransacked sund joined a conference call with d.c. government and pentagon officials at : p.m. where he "[made] an urgent, urgent immediate request for national guard assistance", telling them he needed "boots on the ground". however, lt. gen. walter e. piatt, director of the army staff, said he could not recommend that army secretary ryan mccarthy approve the request, telling sund and others "i don't like the visual of the national guard standing a police line with the capitol in the background".[ ] piatt later told the washington post that he "did not make the statement or any comments similar to what was attributed to me by chief sund".[ ] lt. gen. charles a. flynn, brother of retired lt. gen. michael flynn, was also on the phone call. (the army initially denied charles flynn's participation but confirmed it on january , when flynn himself told the washington post that he "entered the room after the call began and departed prior to the call ending.")[ ] about : p.m. on january , d.c. mayor muriel bowser ordered a : p.m. curfew to go into effect that night.[ ] virginia governor ralph northam also issued a curfew for nearby alexandria and arlington county in northern virginia.[ ][ ] armed dhs agents were on standby near the capitol in case of unrest, but were not deployed until after the violence had subsided.[ ] pentagon officials reportedly restricted d.c. guard troops from being deployed except as a measure of last resort, and from receiving ammunition and riot gear; troops were also instructed to engage with protesters only in situations warranting self-defense and could not share equipment with local police or use surveillance equipment without prior approval from acting defense secretary christopher c. miller.[ ][ ] mccarthy and miller decided to deploy the entire , -strong force of d.c. national guard to quell violence.[ ][ ] about : p.m., miller spoke with pence, pelosi, mcconnell and schumer, and directed the national guard and other "additional support" to respond to the riot.[ ][ ][ ] the order to send in the national guard, which trump initially resisted, was approved by pence.[ ][ ] this bypassing of the chain of command has not been explained.[ ] around : p.m., northam said that he was working with bowser and congress leaders to respond and that he was sending members of the virginia national guard and virginia state troopers to support d.c. law enforcement, at the mayor's request.[ ] at : p.m., stenger told sund he would ask mitch mcconnell for help expediting the national guard authorization.[ ] protesters gathered outside the united states capitol it took over three hours for police to retake control of the capitol, using riot gear, shields, and batons,[ ] and up to eight hours to fully clear the capitol and its grounds.[ ] capitol police were assisted by the local d.c. metropolitan police department,[ ] which sent officers (over a quarter of the total force) to the capitol during the insurrection, along with an additional officers to the capitol grounds.[ ] smoke grenades were deployed on the senate side of the capitol by capitol police working to clear rioters from the building.[ ] black officers employed with capitol police reported being subjected to racial epithets (including repeated uses of "nigger") by some of the rioters.[ ] capitol police chief steven sund said his officers' slow response to the rioting was due to their being preoccupied by the improvised explosive devices found near the capitol.[ ] fbi and department of homeland security agents wearing riot gear entered the dirksen senate office building around : p.m.[ ] new jersey governor phil murphy announced at : p.m. that elements of the new jersey state police were being deployed to the district of columbia at the request of d.c. officials, and that the new jersey national guard was prepared for deployment if necessary.[ ] shortly before : p.m., congressional leaders were reportedly being evacuated from the capitol complex to fort mcnair, a nearby army base.[ ] around : p.m., maryland governor larry hogan announced that he would send the maryland state police and maryland national guard, after speaking to the secretary of the army.[ ][ ] hogan's requests of the defense department to authorize national guard troops to be deployed at the capitol initially were denied in multiple instances.[ ] at around : p.m., the senate sergeant at arms announced that the capitol had been secured.[ ] riot police and protesters outside the capitol in the evening as police continued to try to push rioters away from the capitol, protests continued, with some moving out of the capitol hill area. some verbal and physical attacks on reporters were reported, with attackers denigrating media outlets as providing "fake news".[ ] one rioter told a cnn crew as they were being harassed by others, "there's more of us than you. we could absolutely fucking destroy you!"[ ] a video on social media recorded a man harassing an israeli journalist covering the events live.[ ] by : p.m., police had arrested at least thirteen people and seized five firearms.[ ] although bowser had ordered a : p.m. curfew, it went largely ignored by the pro-trump rioters, hundreds of whom remained in the capitol hill area two hours after the curfew went into effect.[ ] new york governor andrew cuomo pledged to deploy a thousand members of the new york national guard to d.c., in addition to the resources promised by other states.[ ] on the night of january , bowser issued an order extending the public emergency in washington, d.c., for days, writing in the order that she expected some people would "continue their violent protests through the inauguration".[ ][ ] the following day, secretary of the army ryan d. mccarthy announced that a fence would be built around the capitol, and remain in place for at least days; construction of the fence began that same day. mccarthy also said new jersey national guard troops would be mobilized, as would troops from the delaware, new york, and pennsylvania national guards.[ ] by the end of the day, police had arrested people for "unrest-related" offenses, with about half of these arrests occurring on the capitol grounds.[ ] a vehicle containing a semi-automatic rifle and a cooler full of eleven molotov cocktails was also found nearby.[ ][ ] the driver was subsequently arrested.[ ] he also had three handguns in his possession at the time of his arrest.[ ] d.c. metro police incurred significant costs, preliminarily estimated to be $ . million, responding to the attack on the capitol and securing downtown d.c. the week after.[ ] completion of electoral vote count main article: united states electoral college vote count play media the senate resumes the debate on the challenge to arizona's electoral college after p.m. congress reconvened in the evening of january after the capitol was cleared of trespassers. senate majority leader mitch mcconnell reopened the senate's session around : p.m. est, saying that the senate refused to be intimidated, and that it would count the electors and declare the president "tonight", after two hours of debate on the objection to the arizona electors. he called the vote the most consequential in his -plus years of congressional service. at : p.m., the senate rejected the objection – , with only six republicans voting in favor: ted cruz (tx), josh hawley (mo), cindy hyde-smith (ms), john neely kennedy (la), roger marshall (ks), and tommy tuberville (al).[ ] congressional staffers removed the electoral college certificates from the capitol as it was evacuated. at : p.m., the house of representatives rejected a similar motion to dispute the arizona vote by a margin of – .[ ] all of the "yeas" came from republicans while the "nays" were from republicans and democrats.[ ] a planned objection to the georgia slate of electors was rejected after co-signing senator, kelly loeffler (r-ga), withdrew her support in light of the day's events.[ ] another objection was raised by hawley and representative scott perry (r–pa) to the pennsylvania slate of electors, triggering another two-hour split in the joint session to debate the objection.[ ] at : a.m. on january , the senate rejected this objection by a – vote, with the same people voting the same way as before with the exceptions of senators cynthia lummis (r-wy) and rick scott (r-fl) voting in favor and john n. kennedy voting against.[ ] at : a.m., the house of representatives similarly rejected the motion to sustain the objection by a margin of – . again, all of the votes in favor were republican, while this time, only republicans voted against and democrats voted against.[ ] representative peter meijer (r–mi) said that several of his republican colleagues in the house would have voted to certify the votes, but did not out of fear for the safety of their families,[ ] and that at least one specifically voted to overturn biden's victory against their conscience because they were shaken by the mob attack that day.[ ] at : a.m., congress confirmed the outcome of the electoral college vote, biden's votes to trump's , with pence declaring that biden and harris would take office on january .[ ][ ][ ] casualties and damage deaths and injuries capitol police officer brian d. sicknick died from injuries sustained in the riot.[ ][ ] five people died or were fatally injured during the event: one was a capitol police officer, and four were among those who stormed or protested at the capitol.[ ] nearly capitol police officers were injured,[ ] of whom at least were hospitalized and one was in critical condition;[ ] all had been released from the hospital by january .[ ] additionally, rioters injured more than d.c. metro police officers during the attack, including one who remained hospitalized five days after the attack.[ ] some rioters beat officers with pipes,[ ] and some officers sustained head injuries from being struck with lead pipes.[ ] rioters injured other police officers by using chemical irritants and stun guns, hitting them with fists, sticks, poles and clubs; trampling and stampeding them; pushing them down stairs or against statues; and shining laser pointers in their eyes. many police officers were attacked with bear spray, a highly concentrated form of pepper spray stronger than the tear gas typically carried by officers.[ ] one d.c. metro officer was hit six times with a stun gun and suffered an apparent heart attack.[ ] according to the chairman of the capitol police officers' union, multiple officers sustained traumatic brain injuries; one officer suffered two cracked ribs and two smashed spinal discs; and another officer would lose an eye.[ ] one of the most intense of the many violent incidents occurred shortly after p.m., as a swarm of rioters attempted to breach a door on the west front of the capitol. there, rioters dragged three d.c. metro police officers out of formation and down a set of stairs, trapped them in a crowd, and assaulted them with improvised weapons (including hockey sticks, crutches, flags, poles, sticks, and stolen police shields) as the mob chanted "police stand down!" and "usa!"[ ] at least one of the officers was also stomped upon.[ ] president joe biden and first lady jill biden attend the viewing for officer sicknick's remains at the u.s. capitol on february , capitol police officer brian d. sicknick, , a -year veteran of the force, died as a result of injuries sustained while engaging with protesters. the exact cause of his injuries has not been made public, with investigators lacking evidence that someone caused his death.[ ] reuters reported that sicknick suffered a thromboembolic stroke[ ] and collapsed after returning to his division office. he was later placed on life support,[ ] but died the following day.[ ][ ] sicknick will lie in honor in the capitol rotunda prior to his burial at arlington national cemetery.[ ][ ] his death is being investigated by the metropolitan police department's homicide branch, the uscp, and federal authorities.[ ] during the riot, ashli elizabeth babbitt, a -year-old air force veteran from san diego,[ ][ ][ ] was fatally shot by capitol police as she attempted to climb through a shattered window in a barricaded door leading into the speaker's lobby, which has direct access to the house floor.[ ][ ][ ] the incident was recorded on several cameras.[ ][ ] babbitt was unarmed when she was shot and killed; however, officers were aware that many rioters and intruders could be carrying concealed weapons.[ ] the d.c. metropolitan police department launched an investigation into the death,[ ] and the officer who shot her was placed on administrative leave pending the investigation.[ ] babbitt was a follower of qanon, and had tweeted the previous day "the storm is here", a reference to a qanon conspiracy theory.[ ][ ][ ] since the event, babbitt has been described as a martyr by some far-right extremists who view her as a freedom fighter.[ ] on the contrary, some qanon supporters, including lawyer l. lin wood, have claimed that babbitt is still alive and that her apparent death was a "false flag" operation.[ ] three others also died: rosanne boyland, , of kennesaw, georgia; kevin greeson, , from athens, alabama; and benjamin philips, , of bloomsburg, pennsylvania.[ ][ ][ ] boyland was trampled to death by people rushing to breach a tunnel entrance on the west side of the capitol;[ ] she was a radicalized follower of qanon whose family had begged her not to attend.[ ][ ] greeson had a heart attack outdoors on the capitol grounds, and was declared dead at : p.m., shortly before the breach of the capitol.[ ][ ] his family said he was "not there to participate in violence or rioting, nor did he condone such actions".[ ] philips died of a stroke.[ ] the philadelphia inquirer reported that there was no indication that philips participated in the raid.[ ] philips had started the social media site "trumparoo", intended for trump supporters.[ ] a family member of boyland said that "the president's words incited a riot that killed four of his biggest fans last night".[ ] morale among the capitol police plummeted after the riots. the department responded to "a couple of incidents" where officers threatened to harm themselves; one officer turned in her weapon because she feared what she would do with it.[ ] one capitol police officer who was on duty at the capitol during the riot died by suicide three days later.[ ][ ] separately, a dc police officer was reported to have committed suicide in the days following the riot.[ ] damage, theft, and impact a damaged window in the capitol rioters stormed the offices of nancy pelosi, flipping tables and ripping photos from walls;[ ][ ] the office of the senate parliamentarian was ransacked;[ ] art was looted;[ ] and feces was tracked into several hallways.[ ][ ] windows were smashed throughout the building, leaving the floor littered with glass and debris.[ ][ ] some items of furniture were damaged, turned over, or stolen.[ ] one door had "murder the media" scrawled onto it.[ ] rioters damaged associated press recording and broadcasting equipment outside the capitol after chasing away reporters.[ ] rioters also destroyed a display honoring the life of congressman and civil rights leader john lewis.[ ][ ] a photo of representative andy kim (d–nj) cleaning up the litter in the rotunda after midnight went viral.[ ] multiple sources noted that federal prison industries, as a "mandatory source" for government agencies, would receive priority when the government begins purchasing goods fpi manufactures such as office furniture to replace what was damaged in the riots.[ ][ ][ ] the rioters caused extensive physical damage.[ ][ ] the office of the architect of the capitol (aoc), which maintains the capitol and preserves its art and architecture, released an initial damage assessment on january . the aoc reported interior damage from the riot (specifically broken glass, broken doors, and graffiti), and also reported that some statues, paintings, and historic benches "displayed the residue of various pepper sprays, tear gas and fire extinguishers deployed by both rioters and law enforcement personnel."[ ] items, including portraits of john quincy adams and james madison, as well as a marble statue of thomas jefferson, were covered in "corrosive gas agent residue"; these were sent to the smithsonian for assessment and restoration.[ ] a th-century marble bust of president zachary taylor was defaced with what seemed to be blood, but the most important works in the capitol collection, such as the john trumbull paintings, were unharmed.[ ][ ] on the capitol's exterior, two th-century bronze light fixtures designed by frederick law olmsted were damaged.[ ] because the capitol is not insured against loss, taxpayers will pay for damage inflicted by the siege.[ ] abc news reported that the fbi and the bureau of alcohol, tobacco and firearms (atf) had recovered several improvised explosive devices intended to cause serious harm, and were looking at those in the mob that were trained perhaps in the military and more intent on causing serious harm, including harming vice president pence. abc analyst and retired cia officer mick mulroy said the fbi would likely be conducting a full counterintelligence sweep on all those who participated in the assault to determine possible foreign intelligence ties, as they may have taken sensitive information from the congressional offices.[ ][ ] the presence of several military veterans who took part in the assault has created growing concern among former military members.[ ] technology theft and cybersecurity concerns a laptop owned by senator jeff merkley (d-or) was stolen.[ ] a laptop taken from speaker pelosi's office was "[a] laptop from a conference room ... that was only used for presentations", according to pelosi's deputy chief of staff.[ ] representative ruben gallego (d–az) said "we have to do a full review of what was taken, or copied, or even left behind in terms of bugs and listening devices."[ ] military news website sofrep reported that "several" secret‑level laptops were stolen, some of which had been abandoned while still logged in to siprnet, causing authorities to temporarily shut down siprnet for a security update on january and leading the united states army special operations command to re-authorize all siprnet-connected computers on january .[ ][ ] representative anna eshoo (d–ca) said in a statement that "[i]mages on social media and in the press of vigilantes accessing congressional computers are worrying" and that she had asked the chief administrative officer of the house (cao) "to conduct a full assessment of threats based on what transpired".[ ] the cao said it was "providing support and guidance to house offices as needed".[ ] one protester was arrested on charges of theft. it was alleged that she had stolen a laptop or hard drive from pelosi's office with the intention of sending it to a friend in russia for sale to the country's foreign intelligence service.[ ] conservation of items damaged or left behind signs, flags, stickers, pelosi's damaged nameplate, and other items left behind from the riot will be preserved as historical artifacts in the collections of the house and senate and those of national museums.[ ] anthea m. hartig, director of the smithsonian's national museum of american history, said that the smithsonian would seek to collect and preserve "objects and stories that help future generations remember and contextualize jan. and its aftermath", a statement echoed by jane campbell, president of the capitol historical society.[ ][ ] reactions domestic reactions main article: domestic reactions to the storming of the united states capitol pro-trump protesters around the capitol on the evening of january after drawing widespread condemnation from congress, members of his administration, and the media, trump released a video-taped statement on january to stop the resignations of his staff and the threats of impeachment or removal from office. in the statement, he condemned the violence at the capitol, saying that "a new administration will be inaugurated", which was widely seen as a concession, and that his "focus now turns to ensuring a smooth, orderly, and seamless transition of power" to the biden administration.[ ][ ] vanity fair reported that trump was at least partially convinced to make the statement by senator lindsey graham (r-sc), who told trump that a sufficient number of senate republicans would support removing him from office unless he conceded.[ ] white house press secretary kayleigh mcenany had attempted to distance the administration from the rioters' behavior in a televised statement earlier in the day.[ ] on january , the new york times reported that trump had told white house aides that he regretted committing to an orderly transition of power and would never resign from office.[ ] in another unusual move, chairman of the joint chiefs of staff mark milley and all of the other joint chiefs[a] issued a statement on january condemning the storming of the capitol, reminding personnel of their sworn obligation to support and defend the constitution, that biden was about to become their commander-in-chief and stating that "the rights of freedom of speech and assembly do not give anyone the right to resort to violence, sedition and insurrection."[ ] they said, "as we have done throughout our history, the u.s. military will obey lawful orders from civilian leadership, support civilian authorities to protect lives and property, ensure public safety in accordance with the law, and remain fully committed to protecting and defending the constitution of the united states against all enemies, foreign and domestic."[ ][ ] on january senate majority leader mitch mcconnell said "the mob was fed lies" and "they were provoked by the president and other powerful people."[ ] the u.s. flag at the capitol at half-staff in honor of brian sicknick, january house speaker pelosi had the flags at the capitol lowered to half-staff in sicknick's honor.[ ][ ] trump initially declined to lower flags at the white house or other federal buildings under his control, before changing his mind four days later.[ ][ ][ ] biden, pence, and pelosi offered condolences to sicknick's family; trump did not.[ ][ ] after sicknick's death, senator ted cruz (r-tx) received backlash for previous speeches that were perceived as calls for violence.[ ] international reactions main article: international reactions to the storming of the united states capitol over other countries and international organizations expressed their concerns over the protests and condemned the violence, with some specifically condemning trump's own role in inciting the attack.[ ][ ] multiple world leaders have made a call for peace, describing the riots as "an attack on democracy".[ ] the leaders of some countries, including brazil, poland, hungary, and russia, declined to condemn the situation, and described it as an internal u.s. affair.[ ] several nato intelligence agencies outside the united states also briefed their governments that it was an attempted coup by president trump which may have had help from federal law-enforcement officials.[ ] terminology used to refer to event the events have variously been described as treason,[ ] insurrection, sedition, domestic terrorism,[ ] an attempt by trump to carry out a coup d'état[ ][ ] or self-coup,[ ] and the biggest challenge to american democracy since the civil war.[ ] some newspapers described the storming as "anarchy".[ ][ ] the phrase "storm the capitol" was mentioned , times in the days preceding the events; many of these were in viral tweet threads which included details on how to enter the building.[ ] the term storm has particular resonance in qanon discourse; adherents have often referred to a coming storm in which the cabal which allegedly controls the u.s. will be destroyed.[ ] the washington post spoke to political scientists who said the event was a self-coup, where the head of government attempts to strongarm the other branches of government to entrench power.[ ] fiona hill, former national security adviser under trump, also analyzed the events and described trump's actions going back several months or more as all constituting a failed self-coup.[ ] on january , the storming was formally classified as a coup by the coup d’état project, an initiative of the university of illinois's cline center that catalogs attempted coups.[ ][ ] aftermath main article: aftermath of the storming of the united states capitol national guardsmen at the capitol building on january , , in preparation for biden's inauguration secretary of defense lloyd austin speaks with national guardsmen during a visit to the capitol, jan. , because trump continued to claim that the presidential election was rigged without any credible evidence and continued to incite violence, twitter suspended trump's main account first for hours and then permanently.[ ] following this, trump attempted to access alternate accounts on the platform to continue with the tirade but all tweets were subsequently deleted and the accounts either suspended or banned.[ ] furthermore, trump was banned from other major social media outlets including facebook, youtube, and snapchat.[ ] law enforcement's failure to prevent the mob from breaching the capitol attracted scrutiny to the capitol police and other police agencies involved.[ ][ ][ ] several members of the trump administration resigned,[ ] as did three top security officials for congress.[ ] in the wake of the capitol attack and members of congress being increasingly harassed at airports, additional security was assigned to them for air travel. through the inauguration of biden on january , capitol police were to be stationed at d.c.-area airports (reagan national, baltimore-washington, and dulles),[ ] and the transportation security administration (tsa) increased its screening of d.c.-bound air passengers.[ ] security was also put on high alert at the capitol itself; a "non-scalable" security fence was placed around the capitol,[ ] and up to , national guard members were deployed to secure washington, d.c., in advance of inauguration day.[ ] a new security perimeter was created for the inauguration, blocking off large portions of the city near capitol hill.[ ] the washington metro announced it would be closing to subway stations from january to , and re-routing buses around the security zone to discourage people from travelling to the area.[ ][ ][ ] many motels in and around d.c. ceased taking reservations and canceled preexisting ones in the days leading up to inauguration.[ ] public health experts have said that the storming of the capitol was a potential covid- superspreader event.[ ] questions have been raised in the media regarding the discrepancy in police response to black lives matter and white supremacist protesters, including the rioters who stormed the us capitol.[ ][ ][ ] according to an analysis by the guardian of statistics collected by the us crisis monitor, "police in the united states are three times more likely to use force against leftwing protesters than rightwing protesters", regardless of whether the protest is peaceful or violent.[ ][ ] investigations and prosecutions fbi poster seeking information on violence at the capitol on january , michael r. sherwin, the interim united states attorney for the district of columbia, said rioters could be charged with seditious conspiracy or insurrection.[ ] regarding calls for the president to be prosecuted for inciting the violence that led to the five deaths,[ ][ ] sherwin suggested that trump could be investigated for comments he made to his supporters before they stormed the capitol. he also said that others, including any capitol police officers, who "assisted or facilitated or played some ancillary role" in the events could also be investigated.[ ][ ] the federal bureau of investigation (fbi) opened more than subject case files in connection with the attack, and more than grand jury subpoenas and search warrants were issued.[ ] by february , , at least people have been arrested and charged with crimes.[ ] some criminal indictments are under seal.[ ] the majority of cases are in federal court, while others are in d.c. superior court.[ ] the fbi has received more than , digital media tips from the public.[ ][ ] notable arrests include: west virginia state lawmaker derrick evans, who resigned from office; klete keller, a former u.s. olympic swimmer;[ ] the leader of a proud boys group in hawaii;[ ] jake angeli, also known as the "qanon shaman";[ ] far-right activist tim "baked alaska" gionet;[ ] and the -year-old son of a brooklyn judge.[ ][ ] by january , , the number of people arrested and charged with federal crimes amounted to less than a quarter of those involved in the attack on the capitol.[ ] of those arrested and charged with federal crimes as of that date, approximately were charged with trespassing or disrupting congress only; were charged with interference with law enforcement, property crimes, weapons offenses, or making threats, but were not charged with conspiracy or assault; and were charged with the most serious crimes: conspiracy or assault.[ ] as of that date: at least of those charged had "expressed allegiance to or affinity for militant groups" including the proud boys, oath keepers, three percenters, and patriot front; at least of those charged claimed to be current or former members of the military; and at least had clearly expressed a belief in the qanon conspiracy theory.[ ] impeachment of trump main article: second impeachment of donald trump on january , , representatives david cicilline (d-ri), jamie raskin (d-md), and ted lieu (d-ca) introduced to the house a single article of impeachment against trump, which they had written, for "incitement of insurrection" in urging his supporters to march on the capitol building.[ ][ ][ ][ ] nancy pelosi named impeachment managers, led by raskin and followed in seniority by diana degette, cicilline, joaquin castro, eric swalwell, lieu, stacey plaskett, madeleine dean, and joe neguse.[ ] trump was impeached for the second time on january . he is the only federal official in united states history to have ever been impeached twice.[ ][ ] events outside washington, d.c. state capitols and cities see also: united states inauguration week protests trump supporters and police at the texas state capitol on january in an internal "situational information report" dated december , , the fbi's minneapolis field office warned of armed protests at every state capitol, orchestrated by the far-right boogaloo movement, before biden's inauguration.[ ][ ] a number of states experienced demonstrations and armed protests at the state capitols or in the streets on january , numbering in dozens or hundreds of participants. the pro-trump events remained without incident in indiana,[ ] minnesota,[ ] nevada,[ ] nebraska,[ ] ohio,[ ] arizona, colorado, kansas, michigan, pennsylvania, tennessee, texas, wisconsin, and wyoming.[ ] precautionary measures, such as closures of state capitols and evacuation of members and staff, were taken in several of the states in response to the events in washington d.c.[ ][ ] in some states the events were marked by incidents or particular security concerns. in california, eleven people were arrested for illegal possession of pepper spray at a demonstration near the state capitol in sacramento. there was at least one reported assault. several roads were closed in downtown sacramento and some bus lines were stopped, with over police assigned to the demonstration. some members of the crowd wore t-shirts supporting the far-right proud boys.[ ] there were also protests in the los angeles area, including at the los angeles police department headquarters downtown; as well as in beverly hills and in newport beach. an incident was reported of a protester spraying a counter-protester with a chemical irritant.[ ] during the los angeles protests, a mob of to trump supporters physically assaulted a black woman who was walking down the street, shouting racial slurs and chanting "all lives matter" while shoving, striking, spraying with pepper spray, and ripping off her hair extensions.[ ][ ] in georgia, about pro-trump activists gathered outside the state capitol in atlanta,[ ] including armed militia movement members.[ ] a courthouse complex and two other government buildings were closed as a precaution.[ ] chester doles, a former ku klux klan member who leads the far-right group american patriots usa, attempted to enter the state capitol to deliver a "redress of grievances" about the election to georgia secretary of state brad raffensperger;[ ] however, raffensperger and his staff evacuated early as a precaution.[ ][ ][ ] in oklahoma, at the state capitol in oklahoma city, one arrest was made on charges of attempted arson as well as assault and battery for attempting to light other people's flags on fire.[ ] the protest numbered in the hundreds and was otherwise peaceful.[ ] in oregon, arrest were made after hundreds gathered outside the oregon state capitol in salem.[ ] in washington, pro-trump activists in, some of whom were armed, broke through the gates at the washington governor's mansion at the state capitol campus in olympia, and occupied the front lawn, prompting a standoff with the state patrol.[ ][ ] protests were again being held at state capitols in the week before inauguration – mostly on january , which is a day of symbolic importance to the adherents of the qanon conspiracy, "q" being the th letter of the alphabet.[ ] international internationally, trump's allegations of a "stolen" election found a small audience among conspiracy theorists and fringe groups.[ ] in canada, a few dozen people rallied in support of trump in toronto, vancouver, and calgary.[ ] at the vancouver rally, cbc photojournalist ben nelms was assaulted by one of the demonstrators.[ ] in japan, a few hundred people rallied in support of trump in tokyo, with several people carrying the u.s. flag and the rising sun flag, a controversial symbol in east asia because of its association with japanese imperialism. the gathering in tokyo was backed by happy science, a new religious movement that has been described as a cult, and took place several hours before the rally in washington, d.c.[ ] in new zealand, about participants attending a "freedom rally" outside the new zealand parliament in wellington organized by conspiracy theorist and new zealand public party leader billy te kahika waved pro-trump banners and flags.[ ][ ] see also list of attacks on legislatures list of incidents of political violence in washington, d.c. republican reactions to donald trump's claims of election fraud timeline of protests against donald trump list of incidents of grave disorder in the british house of commons parliament house riot notes ^ in addition to milley, the others who signed the statement were general john e. hyten of the air force, the vice chairman of the joint chiefs of staff; general james c. mcconville, the army chief of staff; general david h. berger, the commandant of the marine corps; admiral michael m. gilday, the chief of naval operations; charles q. brown jr., the air force chief of staff; general john w. raymond, the chief of space operations; and daniel r. hokanson, the chief of the national guard bureau.[ ] references ^ a b c d e barry, dan; mcintire, mike; rosenberg, matthew (january , ). "'our president wants us here': the mob that stormed the capitol". the new york times. archived from the original on january , . retrieved january , . ^ a b c d reeves, jay; mascaro, lisa; woodward, calvin (january , ). "capitol assault a more sinister attack than first appeared". associated press. retrieved january , . under battle flags bearing donald trump's name, the capitol's attackers pinned a bloodied police officer in a doorway, his twisted face and screams captured on video. they mortally wounded another officer with a blunt weapon and body-slammed a third over a railing into the crowd. 'hang mike pence!' the insurrectionists chanted as they pressed inside, beating police with pipes. they demanded house speaker nancy pelosi's whereabouts, too. they hunted any and all lawmakers: 'where are they?' outside, makeshift gallows stood, complete with sturdy wooden steps and the noose. guns and pipe bombs had been stashed in the vicinity. ... the mob got stirring encouragement from trump and more explicit marching orders from the president's men. 'fight like hell,' trump exhorted his partisans at the staging rally. 'let's have trial by combat,' implored his lawyer, rudy giuliani, whose attempt to throw out election results in trial by courtroom failed. it's time to 'start taking down names and kicking ass,' said republican rep. mo brooks of alabama. criminals pardoned by trump, among them roger stone and michael flynn, came forward at rallies on the eve of the attack to tell the crowds they were fighting a battle between good and evil. ^ multiple sources: chapman, steve (january , ). "the capitol riot shows the growing danger of right-wing extremism". chicago tribune. retrieved january , . hesson, ted; parker, ned; cooke, kristina; harte, julia (january , ). "u.s. capitol siege emboldens motley crew of extremists". reuters. retrieved january , . nguyen, tina; scott, mark (january , ). "right-wing extremist chatter spreads on new platforms as threat of political violence ramps up". politico. retrieved january , . coleman, alistair (january , ). "capitol riots: what are far-right trump supporters saying?". bbc. retrieved january , . goldman, adam; benner, katie; feuer, alan (january , ). "investigators eye right-wing militias at capitol riot". the new york times. retrieved january , . williams, katie bo; peniston, bradley (january , ). "right-wing extremists storm us capitol". defenseone.com. retrieved january , . zaru, deena (january , ). "the symbols of hate and far-right extremism on display in pro-trump capitol siege". abc news. retrieved january , . ^ a b pramuk, jacob (january , ). "senate salvages electoral college ballots before rioters break into the chamber". cnbc. retrieved january , . electoral college ballots rescued from the senate floor. if our capable floor staff hadn't grabbed them, they would have been burned by the mob. ^ a b c d e f g h i j k l paybarah, azi; lewis, brent (january , ). "stunning images as a mob storms the u.s. capitol". the new york times. archived from the original on january , . retrieved january , . ^ a b lakritz, talia (january , ). "shocking photos show pro-trump rioters in the capitol stealing memorabilia and breaking into the desks of lawmakers". yahoo news. retrieved january , . ^ a b benner, katie; levenson, michael (january , ). "a capitol police officer who was seriously injured wednesday remains on life support". the new york times. archived from the original on january , . retrieved january , . ^ multiple sources: becker, maki (january , ). "cheektowaga man set fire outside, entered capitol during washington insurrection". buffalo news. archived from the original on january , . retrieved january , . brown, melissa (january , ). "a visible gun led dc cops to the 'homemade napalm' in lonnie coffman's truck, affidavit shows". montgomery adviser. ^ a b multiple sources: snyder, timothy (january , ). "the american abyss" – via nytimes.com. ben-ghiat, ruth. "opinion: trump's big lie wouldn't have worked without his thousands of little lies". cnn. heavey, jan wolfe, susan (january , ). "trump lawyer giuliani faces $ . billion lawsuit over 'big lie' election fraud claims" – via www.reuters.com. "can the forces unleashed by trump's big election lie be undone?". npr.org. "donald trump is gone but his big lie is a rallying call for rightwing extremists". the guardian. january , . ^ a b savage, charlie; goldman, adam; macfarquhar, neil (january , ). "'this kettle is set to boil': new evidence points to riot conspiracy" – via nytimes.com. ^ a b carless, will (january , ). "nation's capital braces for violence as extremist groups converge to protest trump's election loss". usa today. archived from the original on january , . retrieved january , . ^ a b lee, arluther (january , ). "evidence suggests pence was in grave danger during capitol riot". the atlanta journal-constitution. archived from the original on january , . retrieved january , . ^ a b c date, jack; barr, luke (january , ). "'hazardous' suspected explosive devices found outside rnc and dnc". abc news. archived from the original on january , . retrieved january , . ^ a b woodward, alex (january , ). "what happened in washington dc yesterday? a timeline of insurrection". the independent. retrieved january , . ^ wabc (january , ). "rep alexandria ocasio-cortez: 'we came close to half of the house nearly dying' during riots". abc new york. retrieved january , . ^ "twitter permanently suspends trump from its platform, citing 'risk of further incitement of violence'". kwwl. associated press. january , . archived from the original on january , . retrieved january , . ^ "top white house officials resign after capitol hill riots, including melania trump's chief of staff". australian broadcasting corporation. reuters/ap. january , . retrieved january , . ^ a b "the trump administration officials who resigned over the violence in the capitol". the new york times. january , . retrieved january , . ^ a b everett, burgess; caygle, heather (january , ). "top dems sack capitol security officials after deadly riot". politico. archived from the original on january , . retrieved january , . ^ "capitol police officer eugene goodman hailed as 'a hero'". bbc news. january , . retrieved january , . ^ a b c katkov, mark (january , ). "army sec. says a 'non-scalable' -foot fence is going up around u.s. capitol". npr. archived from the original on january , . retrieved january , . ^ kutlu, ovunc (january , ). "death toll rises to at capitol riots in us capital". anadolu agency. ^ kates, graham; albert, victoria; freiman, jordan; carissimo, justin; jones, zoe christen (january , ). "capitol secured after assault from trump supporters". cbs news. retrieved january , . ^ multiple sources: fracassa, dominic; li, roland; beamish, rita (january , ). "live updates: twitter says users discussing second attack on u.s. capitol on jan. ". san francisco chronicle. retrieved january , . levenson, eric; vera, amir; kallingal, mallika (january , ). "what we know about the deaths in the pro-trump mob that stormed the capitol". cnn. mcevoy, jemima (january , ). "these are the four people who died amid the capitol riot". forbes. retrieved january , . ^ a b terruso, julia (january , ). "he organized a bus of trump supporters from pa. for 'the first day of the rest of our lives.' he died in d.c." philadelphia inquirer. archived from the original on january , . retrieved january , . ^ a b dartunorro, clark; thorp v, frank (january , ). "capitol police officer dies from injuries after clashing with pro-trump mob". nbc news. archived from the original on january , . retrieved january , . ^ a b kaplan, michael (january , ). "morale deteriorates among capitol police after assault on capitol". cbs news. retrieved january , . ^ melendez, pilar; bredderman, william; montgomery, blake (january , ). "woman shot dead as mob overran capitol id'ed as air force vet". the daily beast. archived from the original on january , . retrieved january , . ^ a b bruce leshan (january , ). "new video shows rioters dragging a dc police officer and beating him with an american flag". wusa. archived from the original on january , . retrieved january , . ^ a b c broadwater, luke (january , ). "the capitol police union says nearly officers were injured during the riot". the new york times. ^ a b c arluther lee (january , ). " capitol hill police officers suspended over riot". atlanta journal-constitution. archived from the original on january . ^ miller, maggie (january , ). "laptop stolen from pelosi's office during capitol riots". the hill. retrieved january , . ^ cohen, marshall (january , ). "capitol rioter charged with threatening to 'assassinate' rep. ocasio-cortez". cnn. retrieved january , . ^ multiple sources: perez, evan; herb, jeremy; polantz, katelyn; scannell, kara; carrega, christina (january , ). "prosecutors 'looking at all actors,' including trump, as charges are filed against capitol rioters". cnn. archived from the original on january , . retrieved january , . leblanc, paul (january , ). "at least domestic terrorism cases opened after us capitol breach". cnn. retrieved january , . leblanc, paul (january , ). "key takeaways from the stunning justice department news conference on the us capitol breach". cnn. retrieved january , . ^ a b "acting attorney general jeffrey a. rosen attends security briefing at fbi's strategic information and operations center on inauguration planning and recent capitol attack" (press release). office of public affairs, the united states department of justice. january , . retrieved january , . ^ a b mallin, alexander; barr, luke (january , ). "doj has identified suspects, charged in capitol riot". abc news. archived from the original on january , . retrieved january , . ^ a b c axon, rachel; pulver, dinah; stassen-berger, rachel; fraser, jayme; salman, josh; penzenstadler, nicholas; wedell, katie; hines, morgan; baratz, david (february , ). "capitol riot arrests: see who's been charged across the u.s." usa today. ^ evelyn, kenya (january , ). "capitol attack: the five people who died". the guardian. ^ barry, dan; frenkel, sheera (january , ). "'be there. will be wild!': trump all but circled the date". the new york times. retrieved january , . ^ doig, steve. "it is difficult, if not impossible, to estimate the size of the crowd that stormed capitol hill". the conversation. retrieved january , . ^ higgins, andrew (january , ). "the art of the lie? the bigger the better – lying as a political tool is hardly new. but a readiness, even enthusiasm, to be deceived has become a driving force in politics around the world, most recently in the united states". the new york times. retrieved january , . ^ a b c d jim rutenberg, jo becker, eric lipton, maggie haberman, jonathan martin, matthew rosenberg and michael s. schmidt (february , ). " days: trump's campaign to subvert the election". new york times. trump's ... effort to reverse his loss turned into ... an extralegal campaign to subvert the election, rooted in a lie so convincing to some of his most devoted followers that it made the deadly jan. assault on the capitol almost inevitable ... with each passing day the lie grew, finally managing to do what the political process and the courts would not: upend the peaceful transfer of power that for years had been the bedrock of american democracy.cs maint: uses authors parameter (link) ^ multiple sources: amenabar, teddy; zauzmer, julie; davies, emily; brice-saddler, michael; ruane, michael e.; et al. (january , ). "live updates: hundreds storm capitol barricades; two nearby buildings briefly evacuated; trump falsely tells thousands he won". the washington post. archived from the original on january , . retrieved january , . peñaloza, marisa (january , ). "trump supporters clash with capitol police at protest". npr. archived from the original on january , . retrieved january , . rodd, scott; hooks, kris (january , ). "trump supporters, proud boys converge on california's capitol to protest electoral college count". kxjz. archived from the original on january , . retrieved january , – via capitalradio. ^ multiple sources: blake, aaron (january , ). "'let's have trial by combat': how trump and allies egged on the violent scenes wednesday". the washington post. retrieved january , . president donald trump, abc news prime: call for trump's removal, capitol hill security failure, global reaction to riots on youtube, abc news, january , , minutes : – : . haberman, maggie (january , ). "trump told crowd 'you will never take back our country with weakness'". the new york times. retrieved january , . da silva, chantal (november , ). "'reckless' and 'stupid': trump jr calls for 'total war' over election results". the independent. retrieved january , . ^ woodward, calvin. "ap fact check: trump's call to action distorted in debate". associated press. retrieved february , . ^ a b "trump's / / speech". january , . archived from the original on january , . retrieved january , . ^ jacobo, julia. "this is what trump told supporters before many stormed capitol hill". abc news. retrieved january , . we have come to demand that congress do the right thing and only count the electors who have been lawfully slated. lawfully slated. i know that everyone here will soon be marching over to the capitol building to peacefully and patriotically make your voices heard. today, we will see whether republicans stand strong for integrity of our elections. but whether or not they stand strong for our country, our country. our country has been under siege for a long time. far longer than this four year period. ^ bennet, dalton; brown, emma; cahlan, sarah; sohuyn lee, joyce; kelly, meg; samuels, elyse; swaine, jon (january , ). " minutes of fear: a video timeline from inside the capitol siege". the washington post. retrieved january , . ^ a b barrett, ted; raju, manu; nickeas, peter (january , ). "pro-trump mob storms us capitol as armed standoff takes place outside house chamber". cnn. archived from the original on january , . retrieved january , . ^ "vandalized": "trump supporters storm capitol; dc national guard activated; woman fatally shot". the washington post. january , . pallini, thomas (january , ). "photos show the aftermath of an unprecedented and destructive siege on the us capitol that left rioters dead". business insider. "looted": daly, matthew; balsamo, michael (january , ). "deadly siege focuses attention on capitol police". associated press. retrieved january , . ^ multiple sources: landale, james (january , ). "capitol siege: trump's words 'directly led' to violence, patel says". bbc news. archived from the original on january , . retrieved january , . dozier, kimberly; bergengruen, vera (january , ). "incited by the president, trump supporters violently storm the capitol". time. archived from the original on january , . retrieved january , . ^ evon, dan. "was 'hang mike pence' chanted at capitol riot?". snopes. retrieved january , . ^ polantz, katelyn (january , ). "us says capitol rioters intended to 'capture and assassinate' elected officials". cnn. retrieved january , . ^ feis, aaron (january , ). "capitol riot seen in new shocking footage like never before". new york post. retrieved january , . ^ betz, bradford (january , ). "new video shows maga mob rifling through paperwork in senate chamber as they hunt down nancy pelosi". fox news. retrieved january , . ^ rosenberg, matthew (january , ). "he looted speaker pelosi's office, and then bragged about it" – via nytimes.com. ^ multiple sources: conradis, brandon (january , ). "pelosi's office vandalized after pro-trump rioters storm capitol". the hill. retrieved january , . swaine, jon (january , ). "man who posed at pelosi desk said in facebook post that he is prepared for violent death". the washington post. retrieved january , . rambaran, vandana (january , ). "oregon sen. jeff merkley says laptop stolen from office during capitol riots". fox news. retrieved january , . colvin, jill (january , ). "capitol mob built gallows and chanted 'hang mike pence'". associated press. archived from the original on january , . "us capitol rioters chanted 'hang mike pence' – video footage". the times of israel. turner-cohen, alex (january , ). "us capitol riot mob wanted to kill mike pence, run pelosi over with a car". news.com.au. retrieved january , . ^ a b "watch live: protesters swarm us capitol steps as congress counts electoral votes". nbc washington. january , . archived from the original on january , . retrieved january , . ^ macias, amanda; mangan, dan (january , ). "u.s. capitol secured hours after pro-trump rioters invade congress". cnbc. archived from the original on january , . retrieved january , . ^ a b mcevoy, jemima (january , ). "dc protests live coverage: entire capitol now on lockdown as protesters enter the building". forbes. archived from the original on january , . retrieved january , . ^ higgins, tucker (january , ). "dc protests: fbi says suspicious devices were rendered safe". cnbc politics. ^ benner, katie (january , ). "pipe bomb found and destroyed at r.n.c.; d.n.c. is evacuated – the new york times". the new york times. ^ collins, kaitlan; cohen, zachary; starr, barbara; hansler, jennifer (january , ). "pence took lead as trump initially resisted sending national guard to capitol". cnn. retrieved january , . ^ a b zilbermints, regina (january , ). "trump tells rioters 'go home,' repeats claims that election 'fraudulent'". the hill. archived from the original on january , . retrieved january , . ^ a b durkee, alison (january , ). "trump justifies supporters storming capitol: 'these are the things and events that happen'". forbes. archived from the original on january , . retrieved january , . ^ soldow, david s. (january , ). "memorandum for record: timeline for december , — january , " (pdf). defense.gov. office of the secretary of defense. archived from the original (pdf) on january , . retrieved january , . ^ liptak, kevin; stracqualursi, veronica; malloy, allie (january , ). "isolated trump reluctantly pledges 'orderly' transition after inciting mob". cnn. archived from the original on january , . ^ fordham, evie (january , ). "trump promises 'orderly transition' on jan. after electoral college results certified". fox news. archived from the original on january , . retrieved january , . ^ a b "'mob was fed lies': mitch mcconnell accuses trump of 'provoking' riot". smh. retrieved january , . ^ "senate debate on arizona electoral college vote challenge, part ". c-span. january , . retrieved january , . ^ karni, annie; haberman, maggie (january , ). "trump openly condones supporters who violently stormed the capitol, prompting twitter to lock his account". the new york times. archived from the original on january , . retrieved january , . ^ brian fung (january , ). "twitter bans president trump permanently". cnn. retrieved january , . ^ fandos, nicholas (january , ). "the house impeaches trump for 'incitement of insurrection,' setting up a senate trial". the new york times. retrieved january , . ^ multiple sources: castronuovo, celine (january , ). "poll: majority of republicans blame biden for mob storming the capitol". the hill. smith, matthew; ballard, jamie; sanders, linley (january , ). "most voters say the events at the us capitol are a threat to democracy". yougov. archived from the original on january , . retrieved january , . castronuovo, celine (january , ). "poll: percent of republicans support capitol riots". the hill. retrieved january , . kahn, chris (january , ). "majority of americans want trump removed immediately after u.s. capitol violence – reuters/ipsos poll". reuters. retrieved january , . ^ a b c "sedition investigations could 'bear fruit soon' in capitol riot cases, fbi says". wusa. january , . ^ barrett, devlin (january , ). "dozens of people on fbi terrorist watch list came to d.c. the day of capitol riot". the washington post. ^ cnn, david shortell, katelyn polantz, evan perez and zachary cohen. "members of extremist oath keepers group planned attack on us capitol, prosecutors say". cnn. ^ perez, evan (february , ). "new charges allege proud boys prepped for capitol insurrection". cnn. ^ holpuch, amanda (january , ). "us capitol's last breach was more than years ago". the guardian. archived from the original on january , . retrieved january , . for the first time on wednesday, it was the site of an armed insurrection incited by the sitting president....not since has the building been breached. then, it was by british troops who set fire to the building during a broader attack on washington in the war of . ^ puckett, jason; spry jr., terry (january , ). "has the us capitol ever been attacked before?". tegna inc. verify. archived from the original on january , . retrieved january , – via wxia-tv. while this is the first large-scale occupation of the u.s. capitol since , there have been several other instances of violence at the u.s. capitol, particularly in the th century. ^ fisher, marc; flynn, meagan; contrera, jessica; loennig, carol d. (january , ). "the four-hour insurrection: how a trump mob halted american democracy". the washington post. archived from the original on january , . retrieved january , . the attack, which some historians called the most severe assault on the capitol since the british sacked the building in ^ "presidential election results". the new york times. december , . retrieved december , . ^ " us presidential election results: live map". abc news. december , . retrieved december , . ^ holder, josh; gabriel, trip; paz, isabella grullón (december , ). "biden's electoral college votes make his victory official". the new york times. ^ blod, michael r.; riccardi, nicholas (december , ). "biden officially secures enough electors to become president". ap news. archived from the original on december , . retrieved january , . ^ a b c d multiple sources: miller, zeke; long, colleen; eggert, david (november , ). "trump tries to leverage power of office to subvert biden win". associated press. archived from the original on november , . retrieved december , . haberman, maggie; rutenberg, jim; corasaniti, nick; epstein, reid j. (november , ). "trump targets michigan in his ploy to subvert the election". the new york times. archived from the original on december , . retrieved december , . "trump presses georgia governor to help subvert election". marketwatch. associated press. december , . archived from the original on december , . retrieved december , . gardner, amy; dawsey, josh; bade, rachael (december , ). "trump asks pennsylvania house speaker for help overturning election results, personally intervening in a third state". the washington post. archived from the original on december , . retrieved december , . hale, lee; et al. (november , ). "the growing backlash against trump's efforts to subvert the election : consider this from npr". npr. archived from the original on december , . retrieved december , . raju, manu; herb, jeremy (december , ). "house conservatives urge trump not to concede and press for floor fight over election loss". cnn. archived from the original on december , . retrieved december , . ^ multiple sources: graham, david a. (january , ). "this is a coup". the atlantic. archived from the original on january , . retrieved january , . musgrave, paul (january , ). "this is a coup. why were experts so reluctant to see it coming?". foreign policy. archived from the original on january , . retrieved january , . solnit, rebecca (january , ). "call it what it was: a coup attempt". the guardian. archived from the original on january , . retrieved january , . coleman, justine (january , ). "gop lawmaker on violence at capitol: 'this is a coup attempt'". thehill. archived from the original on january , . retrieved january , . jacobson, louis (january , ). "politifact – is this a coup? here's some history and context to help you decide". politifact. retrieved january , . a good case can be made that the storming of the capitol qualifies as a coup. it’s especially so because the rioters entered at precisely the moment when the incumbent’s loss was to be formally sealed, and they succeeded in stopping the count. ^ claire o. finkelstein and richard painter (december , ). "invoking martial law to reverse the election could be criminal sedition". just security.cs maint: uses authors parameter (link) ^ u.s.c. § ^ "congress' electoral count to resume after violent protests halt process". npr.org. archived from the original on january , . retrieved january , . ^ a b c nickeas, peter (january , ). "pro-trump supporters have flooded dc to protest president's election loss". cnn. archived from the original on january , . retrieved january , . ^ glantz, aaron; the center for investigative reporting (january , ). "read pence's full letter saying he can't claim "unilateral authority" to reject electoral votes". pbs news hour. associated press. archived from the original on january , . retrieved january , . ^ a b c green, jordan (january , ). "'we're gonna kill congress': trump's far-right supporters promise violence at today's dc protests". the raw story. archived from the original on january , . retrieved january , . ^ a b c schwartz, brian (january , ). "pro-trump dark money groups organized the rally that led to deadly capitol hill riot". cnbc. archived from the original on january , . retrieved january , . ^ beaujon, andrew (january , ). "here's what we know about the pro-trump rallies that have permits". washingtonian. retrieved february , . ^ a b tanfani, joseph; berens, michael; parker, ned (january , ). "how trump's pied pipers rallied a faithful mob to the capitol". reuters. archived from the original on january , . retrieved january , . ^ tanfani, joseph; berens, michael; parker, ned (january , ). "how trump's pied pipers rallied a faithful mob to the capitol". reuters. retrieved february , . ^ montini, ej (january , ). "arizona reps. andy biggs and paul gosar implicated by activist in capitol insurrection". the arizona republic. archived from the original on january , . retrieved january , . ^ grim, ryan; chávez, aída (january , ). "freedom caucus chair andy biggs helped plan january event, lead organizer says". the intercept. archived from the original on january , . retrieved january , . ^ kuznia, rob; devine, curt; bronstein, scott; ortega, bob (january , ). "extremists intensify calls for violence ahead of inauguration day". cnn. archived from the original on january , . retrieved january , . ^ sommer, will (january , ). "'stop the steal' organizer in hiding after denying blame for riot". archived from the original on january , . retrieved january , – via www.thedailybeast.com. ^ anglen, robert; hansen, ronald j. (january , ). "reps. paul gosar, andy biggs credited in video with organizing trump crowd in dc on day of riot". the arizona republic. archived from the original on january , . retrieved january , . ^ "gop reps. deny giving 'reconnaissance tours' to capitol rioters". news.yahoo.com. archived from the original on january , . retrieved january , . ^ a b graff, garrett m. (january , ). "behind the strategic failure of the capitol police". politico. archived from the original on january , . retrieved january , . ^ a b lytvyenko, jane; hensley-clancy, molly (january , ). "the rioters who took over the capitol have been planning online in the open for weeks". buzzfeed news. archived from the original on january , . retrieved january , . ^ a b frenkel, sheera (january , ). "the storming of capitol hill was organized on social media". the new york times. archived from the original on january , . retrieved january , . ^ mcevoy, jemima (january , ). "capitol attack was planned openly online for weeks – police still weren't ready". forbes. archived from the original on january , . retrieved january , . ^ barrett, devlin; hsu, spencer; davis, aaron (january , ). "'be ready to fight': fbi probe of u.s. capitol riot finds evidence detailing coordination of an assault". the washington post. retrieved january , . ^ "march to save america". archived from the original on january , . ^ corey, jamie / (january , ). "republican attorneys general dark money group organized protest preceding capitol mob attack". documented. retrieved january , . ^ a b graziosi, graig (january , ). "alex jones says he paid $ , for rally that led to capitol riot". the independent. archived from the original on january , . retrieved january , . ^ a b shalini ramachandran, alexandra berzon and rebecca ballhaus (february , ). "jan. rally funded by top trump donor, helped by alex jones, organizers say". wall street journal.cs maint: uses authors parameter (link) ^ palmer, ewan (january , ). "charlie kirk says not everyone in capitol mob was an insurrectionist". newsweek. archived from the original on january , . retrieved january , . ^ "stop the steal security project". archived from the original on january , . ^ wong, venessa (january , ). "trump supporters used gofundme to "build an army" in dc". buzzfeed news. archived from the original on january , . retrieved january , . ^ a b c d kuznia, rob; devine, curt; bronstein, scott; ortega, bob (january , ). "extremists intensify calls for violence ahead of inauguration day". cnn. archived from the original on january , . retrieved january , . ^ "the man who saw yesterday's coup attempt coming is only surprised it wasn't much worse". gq. condé nast. archived from the original on january , . retrieved january , . ^ long, colleen; balsamo, michael; kunzelman, michael (january , ). "fbi warns of plans for nationwide armed protests next week". associated press. archived from the original on january , . retrieved january , . ^ a b dilanian, ken (january , ). "fbi has opened case files for crimes committed during capitol riot". nbc news. archived from the original on january , . retrieved january , . ^ barrett, devlin; zapotosky, matt (january , ). "fbi report warned of 'war' at capitol, contradicting claims there was no indication of looming violence". the washington post. archived from the original on january , . retrieved january , . ^ dilanian, ken; ainsley, julia (january , ). "worried about free speech, fbi never sent bulletin on capitol threats". nbc news. archived from the original on january , . retrieved january , . ^ temple-raston, dina (january , ). "why didn't the fbi and dhs produce a threat report ahead of the capitol insurrection?". npr. national public radio. archived from the original on january , . retrieved january , . ^ "extremists and mainstream trump supporters plan to protest congressional certification of biden's victory". anti-defamation league. january , . archived from the original on january , . retrieved january , . ^ "january washington, d.c., security outlook: intelligence report, part three". g s. january , . archived from the original on january , . retrieved january , . ^ cnn, david shortell, katelyn polantz, evan perez and zachary cohen. "members of extremist oath keepers group planned attack on us capitol, prosecutors say". cnn. archived from the original on january , . retrieved january , . ^ savage, charlie; goldman, adam; macfarquhar, neil (january , ). "'this kettle is set to boil': new evidence points to riot conspiracy". archived from the original on january , . retrieved january , – via nytimes.com. ^ "united states v. caldwell, crowl and watkins". united states district court for the district of columbia. january , . ^ abramson, seth. "january meeting at trump international hotel could hold the key to the january insurrection". sethabramson.substack.com. retrieved february , . ^ burkhalter, eddie (january , ). "trump appointee says tuberville met with trump family, advisers on eve of capitol attack". retrieved february , . ^ pierce, charles p. (january , ). "tommy tuberville is a formidable bag of hammers". esquire. retrieved february , . ^ burkhalter, eddie (january , ). "photos put tuberville in trump's hotel on jan. despite denying meeting". retrieved february , . ^ marquardt, alex; starr, barbara; main, alison; cole, devan (january , ). "pentagon approves dc mayor's request to deploy national guard for upcoming demonstrations". cnn. archived from the original on january , . retrieved january , . ^ a b long, colleen; baldor, lolita; balsamo, michael; merchant, nomaan (january , ). "capitol police rejected offers of federal help to quell mob". associated press. archived from the original on january , . retrieved january , . ^ leonnig, carol (january , ). "capitol police intelligence report warned three days before attack that 'congress itself' could be targeted". the washington post. archived from the original on january , . retrieved january , . ^ a b c myre, greg (january , ). "where was security when a pro-trump mob stormed the capitol?". npr. archived from the original on january , . retrieved january , . ^ walsh, deirdre (january , ). "former capitol police chief steven sund defends agency's role in jan. attack". npr. archived from the original on january , . retrieved january , . ^ beaumont, peter (january , ). "ex-head of capitol police: officials reluctant to call in national guard". the guardian. archived from the original on january , . retrieved january , . ^ dilanian, ken; tom winter, jonathan dienst and andrew blankstein (january , ). "fbi, nypd told capitol police about risk of violence, officials say". nbc news. archived from the original on january , . retrieved january , . ^ a b c d bertrand, natasha (january , ). "justice department warns of national security fallout from capitol hill insurrection". politico. archived from the original on january , . retrieved january , . ^ long, colleen; balsamo, michael; mascaro, lisa (january , ). "capitol police were overrun, 'left naked' against rioters". associated press. archived from the original on january , . retrieved january , . ^ shabad, rebecca (january , ). "trump to address d.c. rally where as many as , people are expected". nbc news. archived from the original on january , . retrieved january , . ^ a b c sonne, paul. "pentagon restricted commander of d.c. guard ahead of capitol riot". retrieved february , – via www.washingtonpost.com. ^ a b c beaujon, andrew (january , ). "here's what we know about the pro-trump rallies that have permits". washingtonian. archived from the original on january , . retrieved january , . rally to revival ^ "eight percent coalition". eight percent coalition. retrieved january , . fight to save america ^ "save the republic rally". moms for america. archived from the original on january , . retrieved january , . ^ o'gorek, elizabeth (january , ). "where are wednesday's pro-trump protests?". hill rag. archived from the original on january , . retrieved january , . ^ ramirez, stephanie (january , ). "several arrested on gun charges as pro-trump rallies begin in dc". fox dc. archived from the original on january , . retrieved january , . ^ "wildprotest.com". archived from the original on january , . ^ "freedom rally ". freedom rally . archived from the original on january , . retrieved january , . ^ hayes, rob (january , ). "faculty call for firing of chapman university professor who spoke at pro-trump rally". abc los angeles. archived from the original on january , . retrieved january , . ^ palma, bethania (january , ). "did rudy giuliani call for 'trial by combat' before trump mob broke into capitol?". snopes. archived from the original on january , . retrieved january , . ^ flaherty, colleen (january , ). "chapman faces pressure to fire professor who spoke at trump rally". inside higher ed. archived from the original on january , . retrieved january , . ^ gattis, paul (january , ). "mo brooks: today patriots start 'kicking ass' in fighting vote results". al.com. archived from the original on january , . retrieved january , . ^ pellicer, lauren (january , ). "nc congressional delegates react to violence as pro-trump mob storms us capitol". blue ridge public radio. ^ graziosi, graig (january , ). "trump's sons declare war on gop". the independent. archived from the original on january , . retrieved january , . ^ haberman, maggie (january , ). "trump, speaking to protesters, declares 'we will never concede.'". the new york times. archived from the original on january , . retrieved january , . ^ multiple sources: liptak, kevin; stracqualursi, veronica; malloy, allie (january , ). "trump publicly acknowledges he won't serve a second term a day after inciting mob". cnn. archived from the original on january , . retrieved january , . morris, david z. (january , ). "'we will never concede': how donald trump incited an attack on america". fortune. archived from the original on january , . retrieved january , . rucker, philip (january , ). "trump's presidency finishes in 'american carnage' as rioters storm the capitol". the washington post. archived from the original on january , . retrieved january , . kiely, eugene; farley, robert; gore, d'angelo; robertson, lori (january , ). "trump's falsehood-filled 'save america' rally". factcheck.org. annenberg public policy center of the university of pennsylvania. archived from the original on january , . retrieved january , . tan, shelly; shin, youjin; rindler, danielle (january , ). "how one of america's ugliest days unraveled inside and outside the capitol". the washington post. archived from the original on january , . retrieved january , . ^ a b c d e f g blake, aaron (january , ). "what trump said before his supporters stormed the capitol, annotated". the washington post. archived from the original on january , . retrieved january , . ^ a b c d e f g h savage, charlie (january , ). "incitement to riot? what trump told supporters before mob stormed capitol". the new york times. archived from the original on january , . retrieved january , . ^ andersen, travis (january , ). "before mob stormed us capitol, trump told them to 'fight like hell' –". the boston globe. archived from the original on january , . retrieved january , . ^ kranish, michael (january , ). "before riot, trump said 'we got to get rid' of rep. liz cheney". the washington post. archived from the original on january , . retrieved january , . ^ mogelson, luke. "among the insurrectionists". the new yorker (january , ). archived from the original on january , . retrieved january , . ^ a b "how a presidential rally turned into a capitol rampage". the new york times. january , . archived from the original on january , . retrieved january , . ^ doig, steve (january , ). "it is difficult, if not impossible, to estimate the size of the crowd that stormed capitol hill". the conversation. archived from the original on january , . retrieved january , . ^ bernard, katie (january , ). "a photographer and a fist pump. the story behind the image that will haunt josh hawley". the kansas city star. archived from the original on january , . retrieved january , . ^ regnier, chris (january , ). "senator hawley criticized for acknowledging capitol protesters with fist pump". ktvi. archived from the original on january , . retrieved january , . ^ lee, jessica (january , ). "did trump watch capitol riots from a private party nearby?". snopes. archived from the original on january , . retrieved january , . ^ barrett, devlin; hsu, spencer s. (january , ). "justice department, fbi debate not charging some of the capitol rioters". the washington post. retrieved february , . ^ greenberg, jon; kim, noah y. (january , ). "black lives matter protests and the capitol assault: comparing the police response". politifact. archived from the original on january , . retrieved january , . ^ smith, harrison; olivo, antonio (january , ). "rioters use ropes, makeshift ladders to invade capitol". the washington post. archived from the original on january , . ^ a b c d e f demirjian, karoun; leonnig, carol d.; kane, paul; davis, aaron c. (january , ). "inside the capitol siege: how barricaded lawmakers and aides sounded urgent pleas for help as police lost control". the washington post. archived from the original on january , . retrieved january , . ^ a b hermann, peter (january , ). "'we got to hold this door'". the washington post. retrieved january , . ^ a b c d leonnig, carol d.; davis, aaron c.; hermann, peter; demirjian, karoun (january , ). "outgoing capitol police chief: house, senate security officials hamstrung efforts to call in national guard". the washington post. archived from the original on january , . retrieved january , . ^ leatherby, lauren; ray, arielle; singhvi, anjali; triebert, christiaan; watkins, derek; willis, haley (january , ). "how a presidential rally turned into a capitol rampage". the new york times. archived from the original on january , . retrieved january , . ^ "knowingly entering or remaining in any restricted building in any restricted building or grounds without lawful authority". title of the united states code. p. § . archived from the original on january , . retrieved january , – via law.cornell.edu. ^ "parading, demonstrating or picketing in the capitol buildings". title of the united states code. p. § unlawful activities. archived from the original on january , . retrieved january , . ^ lang, brent; littleton, cynthia (january , ). "u.s. capitol on lockdown, pro-trump protestors breach police lines". variety. archived from the original on january , . retrieved january , . ^ a b balsamo, michael; long, colleen (january , ). "the latest: schumer says jan. , , will live in infamy". associated press. archived from the original on january , . ^ "woman dies after shooting in u.s. capitol; d.c. national guard activated after mob breaches building". the washington post. january , . archived from the original on january , . a mob was able to breach security and successfully enter the building ^ biesecker, michael; bleiberg, jake; laporta, james (january , ). "capitol rioters included highly trained ex-military and cops". ap news. archived from the original on january , . retrieved january , . ^ smith, jamil (january , ). "white entitlement, on parade". rolling stone. archived from the original on january , . retrieved january , . ^ a b landay, jonathan; zengerle, patricia; morgan, david (january , ). "'failure at the top:' after u.s. capitol stormed, security chiefs out". reuters. archived from the original on january , . retrieved january , . ^ a b mclaughlin, eliott c. (january , ). "before wednesday, insurgents waving confederate flags hadn't been within miles of the us capitol". cnn. archived from the original on january , . retrieved january , . ^ contreras, russell (january , ). "descendant of robert e. lee says confederate flag at capitol was "treasonous"". axios. archived from the original on january , . retrieved january , . ^ anderson, javonte (january , ). "capitol riot images showing confederate flag a reminder of country's darkest past". usa today. archived from the original on january , . retrieved january , . ^ tumlin, remy (january , ). "evening briefing special edition: a pro-trump mob storms the capitol". the new york times. archived from the original on january , . retrieved january , . ^ fitz-gibbon, jorge (january , ). "neo-nazis among protesters who stormed us capitol". archived from the original on january , . retrieved january , . ^ eddy, melissa (january , ). "amid the rampage at the u.s. capitol, a sweatshirt stirs troubling memories". the new york times. archived from the original on january , . retrieved january , . ^ devine, curt; bronstein, scott (january , ). "man in 'camp auschwitz' sweatshirt during capitol riot identified". cnn. archived from the original on january , . retrieved january , . ^ cramer, maria (january , ). "confederate battle flag in the capitol: a 'jarring' first in u.s. history". the new york times. archived from the original on january , . retrieved january , . ^ blumenthal, sidney (january , ). "trump's maga insurrectionists were perverse us civil war re-enactors". the guardian. archived from the original on january , . retrieved january , . ^ smith, clint (january , ). "the whole story in a single photo". the atlantic. archived from the original on january , . retrieved january , . during the civil war, the confederate army never reached the capitol. the rebel flag, to my knowledge, had never been flown inside the halls of congress until wednesday. two days ago, a man walked through the halls of government bearing the flag of a group of people who had seceded from the united states and gone to war against it. ^ a b c olmstead, molly (january , ). "how prominent evangelicals reacted to the capitol riot". slate magazine. archived from the original on january , . retrieved january , . ^ jenkins, jack (january , ). "as chaos hits capitol, two forms of faith on display". the salt lake tribune. religion news service. archived from the original on january , . retrieved january , . ^ boorstein, michelle (january , ). "for some christians, the capitol riot doesn't change the prophecy: trump will be president". washington post. archived from the original on january , . retrieved january , . ^ mccarthy, bryn (january , ). "capitol riots: religious leaders condemn violence, urge peace even in political disagreement". fox news. archived from the original on january , . retrieved january , . ^ watts, craig m. (january , ). "the cost of believing liars". red letter christians. archived from the original on january , . retrieved january , . ^ jervis, rick; ramirez, marc; ruiz-goiriena, romina (january , ). "'no regrets': evangelicals and other faith leaders still support trump after deadly us capitol attack". usa today. archived from the original on january , . retrieved january , . ^ "senate rejects challenge to biden arizona win". pbs. associated press. january , . archived from the original on january , . retrieved january , . ^ "live coverage: congress certifies biden win after pennsylvania, arizona challenges fail". the hill. capitol hill publishing corp. january , . archived from the original on january , . retrieved january , . ^ "sen. lankford interrupted while speaking before senate as rioters enter capitol". tulsa world. january , . archived from the original on january . retrieved january , . ^ williams, michael; mishanec, nora; li, roland; fracassa, dominic (january , ). "live updates: twitter says users discussing second attack on u.s. capitol on jan. ". laredo morning times. archived from the original on january , . retrieved january , . ^ a b fandos, nicholas; schaff, erin; cochrane, emily (january , ). "'senate being locked down': inside a harrowing day at the capitol". the new york times. archived from the original on january , . retrieved january , . ^ harvey, olivia (january , ). "the photos of these women saving the ballot boxes belong in history books". yahoo finance. archived from the original on january , . retrieved january , . ^ frenkel, sheera (january , ). "the storming of capitol hill was organized on social media". the new york times. archived from the original on january , . retrieved january , . ^ fisher, marc; flynn, meagan; contrera, jessica; leonnig, carol d. (january , ). "the four-hour insurrection". the washington post. archived from the original on january , . retrieved january , . ^ barbaro, michael (january , ). "how they stormed congress". the daily (new york times podcast). archived from the original on january , . retrieved january , . ^ a b colvin, jill (january , ). "hurt feelings, anger linger after pence, trump clash". abc news. archived from the original on january , . retrieved january , . ^ "'hang mike pence!' trump supporters intended to execute vice president during capitol riots". the raw story. january , . archived from the original on january , . retrieved january , . ^ sheth, sonam (january , ). "a reuters photographer says he overheard pro-trump insurrectionists saying they wanted to hang mike pence at the capitol". business insider. archived from the original on january , . retrieved january , . ^ a b morris, seren (january , ). "video of senate office ransacked in capitol chaos viewed over million times". newsweek. archived from the original on january , . retrieved january , . ^ a b kane, paul (january , ). "inside the assault on the capitol: evacuating the senate". the washington post. archived from the original on january , . retrieved january , . ^ romboy, dennis (january , ). "how president trump misdialed utah sen. mike lee while the capitol was under siege". deseret news. archived from the original on january , . retrieved january , . ^ a b c serfaty, sunlen; cole, devan; rogers, alex (january , ). "as riot raged at capitol, trump tried to call senators to overturn election". cnn. archived from the original on january , . retrieved january , . ^ "associated press timeline of events at the capitol, dead". my sun coast. associated press. archived from the original on january , . retrieved january , . ^ kois, dan (january , ). "they were out for blood". slate. archived from the original on january , . retrieved january , . ^ acosta, jim; brown, pamela (january , ). "trump pressured pence to engineer a coup, then put the vp in danger, source says". cnn. archived from the original on january , . retrieved january , . ^ parker, ashley; leonnig, carol d.; kane, paul; brown, emma (january , ). "how the rioters who stormed the capitol came dangerously close to pence". the washington post. archived from the original on january , . retrieved january , . ^ a b wire, sarah d. (january , ). "reporter inside capitol describes being in a roomful of people 'panicked that i might inadvertently give away their location'". the seattle times. los angeles times. archived from the original on january , . retrieved january , . ^ "watch the moment house floor forced into recess as protesters enter capitol". msn. the washington post. january , . archived from the original on january , . retrieved january , . ^ hensley-clancy, molly; nashrulla, tasneem; baird, addy; jamieson, amber; namako, tom; hall, ellie; mcleod, paul; baer, stephanie k. (january , ). "trump launched a deadly attempted coup, encouraging a mob to breach the us capitol building because he lost the presidential election". buzzfeed news. archived from the original on january , . retrieved january , . ^ a b c d swaine, jon; bennett, dalton; sohyun lee, joyce; kelly, meg (january , ). "video shows fatal shooting of ashli babbitt in the capitol". the washington post. archived from the original on january , . retrieved january , . ^ caygle, heather; ferris, sarah (january , ). "inside pelosi's push to impeach trump: this time it's personal". politico. archived from the original on january , . retrieved january , . ^ fandos, nicholas; cochrane, emily; sullivan, eileen; thrush, glenn; kanno-youngs, zolan (january , ). "pence and lawmakers evacuated as protesters storm the capitol, halting congress's counting of electoral votes". the new york times. archived from the original on january , . retrieved january , . ^ a b kanno-youngs, zolan; tavernise, sabrina; cochrane, emily (january , ). "as house was breached, a fear 'we'd have to fight' to get out". the new york times. archived from the original on january , . retrieved january , . ^ a b berge, clint (january , ). "happening now: armed standoff inside us capitol, shots fired". wqow. archived from the original on january , . retrieved january , . ^ a b "violent pro-trump mob storms us capitol". bbc news. archived from the original on january , . retrieved january , . ^ chowdhury, maureen (january , ). "lawmaker says staff were able to remove electoral ballots before rioters breached senate floor". cnn. archived from the original on january , . ^ winsor, morgan; pereira, ivan; mansell, william (january , ). "updates: capitol breached by protesters, shots reported fired inside". abc news. archived from the original on january , . retrieved january , . ^ "police draw guns inside the capitol". the new york times. january , . archived from the original on january , . retrieved january , . ^ a b "trump supporters storm u.s. capitol, with one woman killed and tear gas fired". the washington post. january , . archived from the original on january , . retrieved january , . ^ a b c kavi, aishvarya (january , ). "lawmakers recount feeling unsafe because of colleagues' behavior during capitol siege". the new york times. archived from the original on january , . retrieved january , . ^ ulloa, jasmine; annear, steve (january , ). "'it was like looking at evil': the capitol attack through the eyes of the massachusetts delegation". the boston globe. archived from the original on january , . retrieved january , . ^ "nancy pelosi's office occupied by pro-trump rioters amid chaos at u.s. capitol". people. archived from the original on january , . retrieved january , . ^ lee, timothy b. (january , ). "pro-trump reporter gloats over access to fleeing hill staffer's computer". ars technica. archived from the original on january , . retrieved january , . ^ "arkansas man admits to storming capitol, sitting at nancy pelosi's desk". thv .com. archived from the original on january , . retrieved january , . ^ a b benner, katie; haberman, maggie; schmidt, michael s. (january , ). "live updates: pro-trump mob breaches capitol, halting vote certification". the new york times. archived from the original on january , . retrieved january , . ^ a b schaff, erin; tavernise, sabrina (january , ). "marauding protesters vandalize speaker pelosi's office". the new york times. archived from the original on january , . retrieved january , . ^ brewster, thomas. "capitol hill mob accessed congressional computers—'consider them all compromised'". forbes. archived from the original on january , . retrieved january , . ^ a b shortell, david (january , ). "pipe bombs found near capitol on january were placed the night before, fbi says". cnn. retrieved january , . ^ benner, katie; haberman, maggie; schmidt, michael s. (january , ). "an explosive device is found at the r.n.c., and the d.n.c. is evacuated". the new york times. archived from the original on january , . retrieved january , . ^ a b balsamo, michael (january , ). "discovery of pipe bombs in dc obscured by riot at capitol". the seattle times. archived from the original on january , . retrieved january , . ^ a b sanchez, rosa (january , ). "fbi posts photo of person who placed suspected pipe bombs outside dnc, rnc". abc news. archived from the original on january , . retrieved january , . ^ fitzsimons, tim (january , ). "fbi ups reward to $ , for suspect who placed pipe bombs during capitol riot". nbc news. archived from the original on january , . retrieved january , . ^ "representative tim ryan news conference on u.s. capitol attack". c-span. january , . archived from the original on january , . retrieved january , . ^ benner, katie; goldman, adam (january , ). "justice dept. pursues at least suspects in capitol riot". the new york times. archived from the original on january , . retrieved january , . ^ a b c d e biesecker, michael; kunzelman, michael; flaccus, gillian; mustian, jim (january , ). "records show fervent trump fans fueled us capitol takeover". associated press. archived from the original on january , . retrieved january , . the insurrectionist mob that showed up at the president's behest and stormed the u.s. capitol was overwhelmingly made up of longtime trump supporters, including republican party officials, gop political donors, far-right militants, white supremacists, members of the military and adherents of the qanon myth that the government is secretly controlled by a cabal of satan-worshiping pedophile cannibals. records show that some were heavily armed and included convicted criminals, such as a florida man recently released from prison for attempted murder. ^ multiple sources: yousef, odette (january , ). "the bizarre outfits at the capitol weren't just costumes. they were a message". wbez. archived from the original on january , . retrieved january , . neilson, susie; mcfall-johnsen, morgan (january , ). "several groups of extremists stormed the capitol on wednesday. here are some of the most notable individuals, symbols, and groups". the business insider. archived from the original on january , . retrieved january , . adkins, laura e.; burack, emily (january , ). "hate on display: a guide to the symbols and signs on display at the capitol insurrection". j. the jewish news of northern california. jewish telegraphic agency. archived from the original on january , . retrieved january , . "'second revolution begins': armed right-wing groups celebrate capitol attack". yahoo! news. archived from the original on january , . retrieved january , . read, richard; hennessy-fisk, molly; chabria, anita (january , ). "far-right extremists want to attack more capitols, but are divided after d.c. riot". los angeles times. archived from the original on january , . retrieved january , . yancey-bragg, n'dea; bacon, john; carless, will; miller, ryan w. "pro-trump rioters breach capitol, forcing lockdown; one person shot; pence evacuated, senate chamber cleared out". usa today. archived from the original on january , . retrieved january , . sales, ben. "fears as trump supporters, including anti-semitic groups, rally in washington". www.timesofisrael.com. archived from the original on january , . retrieved january , . "was the confederate flag raised at the u.s. capitol in washington, d.c.?". snopes.com. archived from the original on january , . retrieved january , . ^ press, associated (january , ). "anti-semitism seen in capitol insurrection raises alarms". daily herald. archived from the original on january , . retrieved january , . ^ "man in 'camp auschwitz' sweatshirt at capitol riot arrested". washington examiner. january , . archived from the original on january , . retrieved january , . ^ barrett, devlin; hsu, spencer s.; lang, marissa j. "dozens of people on fbi terrorist watch list came to d.c. the day of capitol riot". washington post. archived from the original on january , . retrieved january , . ^ makuch, ben (january , ). "neo-nazis boast about participation in capitol hill invasion". vice. archived from the original on january , . retrieved january , . ^ a b pape, robert a.; ruby, keven (february ). "the capitol rioters aren't like other extremists". the atlantic. retrieved february , . ^ "cnn's elle reeve: 'donald trump plus the internet brings extremism to the masses'". www.weny.com. archived from the original on january , . retrieved january , . ^ penzenstadler, nick. "internet detectives swarmed the effort to id capitol riot mob, with mixed results". usa today. archived from the original on january , . retrieved january , . ^ myers, meghann; shane iii, leo; south, todd; rempfer, kyle (january , ). "how many troops were involved in the capitol riot? figuring that out won't be easy". military times. archived from the original on january , . retrieved january , . ^ vanden brook, tom (january , ). "in wake of capitol riot, active-duty army officer under investigation". usa today. archived from the original on january , . retrieved january , . ^ driesbach, tom; anderson, meg (january , ). "nearly in defendants in capitol riot cases served in the military". npr.org. archived from the original on january , . retrieved january , . ^ bellware, kim (january , ). "police departments across the u.s. open probes into whether their own members took part in the capitol riot". the washington post. archived from the original on january , . retrieved january , . ^ ben-menachem, jonathan (january , ). "the cops at the capitol". the appeal. retrieved february , . ^ stelloh, tim; moe, alex; talbot, haley (january , ). " capitol police officers suspended, possibly arrested over pro-trump riot, congressman says". nbc news. archived from the original on january , . retrieved january , . ^ nickeas, peter; grayer, annie; nobles, ryan (january , ). " capitol police officers suspended and at least more under investigation for alleged roles in riot". cnn. archived from the original on january , . retrieved january , . ^ "u.s. v thomas edward caldwell, donovan ray crowl and jessica marie watkins". the washington post. archived from the original on january , . retrieved january , . ^ shortell, david. "us capitol riots: members of extremist oath keepers group planned attack, prosecutors say - cnnpolitics". cnn.com. archived from the original on january , . retrieved january , . ^ "paramilitary group was intent on capturing capitol, bragged about operation". azcentral.com. january , . retrieved january , . ^ devine, curt; griffin, drew (february , ). "leaders of the anti-vaccine movement used 'stop the steal' crusade to advance their own conspiracy theories". cnn. retrieved february , . ^ satija, neena (january , ). "'i do regret being there': simone gold, noted hydroxychloroquine advocate, was inside the capitol during the riot". washington post. issn - . archived from the original on january , . retrieved january , . ^ dazio, stefanie (january , ). "capitol photos, videos lead to california doctor's arrest". abcnews.go.com. abc news. archived from the original on january , . retrieved january , . ^ zadrozny, brandy (january , ). "trump loyalists push evidence-free claims that antifa activists fueled mob". nbc news. ^ bajak, aleszu (january , ). "how the antifa conspiracy theory traveled from the fringe to the floor of congress". usa today. ^ o'rourke, clara (january , ). "no, the fbi didn't confirm antifa activists breached the capitol". politifact. ^ alba, davey (january , ). "f.b.i. says there is no evidence antifa participated in storming the capitol". the new york times. ^ a b spinelli, dan. "a list of all the lawmakers who joined pro-trump crowds on the day of the capitol riot". retrieved february , . ^ dil, cuneyt (january , ). "local lawmakers from several states joined, observed riot at us capitol". wwl-tv. charleston, w.va. associated press. archived from the original on january , . retrieved january , . ^ haas, greg (january , ). "democrats call for nevada republican annie black to resign assembly seat". news now. archived from the original on january , . retrieved january , . ^ "after 'traitor' tweet, maryland state del. cox denounces 'mob violence' at capitol". wtop. january , . archived from the original on january , . retrieved january , . ^ "over a dozen lawmakers joined crowds on day of capitol riot". wtop. january , . archived from the original on january , . retrieved january , . ^ buchanan, joe. "w.va. delegate derrick evans has been federally charged". wdtv. archived from the original on january , . retrieved january , . ^ mcelhinny, brad (january , ). "derrick evans resigns w.va. house after entering u.s. capitol with mob". west virginia metronews. archived from the original on january , . retrieved january , . ^ "'a badge of shame': virginia senate votes to censure amanda chase". january , . retrieved february , . ^ smith, samantha (february , ). "sen. amanda chase files federal lawsuit against virginia senate after censure". wsls. retrieved february , . ^ edmondson, catie (january , ). "live updates: pro-trump protesters storm the capitol". the new york times. archived from the original on january , . retrieved january , . ^ baker, peter; haberman, maggie (january , ). "capitol attack leads democrats to demand that trump leave office". the new york times. archived from the original on january , . retrieved january , . ^ nuzzi, olivia (january , ). "senior trump official: we were wrong, he's a 'fascist'". new york. archived from the original on january , . retrieved january , . ^ lonas, lexi (january , ). "sasse says trump was 'delighted' and 'excited' by reports of capitol riot". the hill. archived from the original on january , . retrieved january , . ^ acosta, jim (january , ). "senior white house correspondant". cnn (the presidential election congress counts the vote). internet archive. retrieved january , . ^ sherman, gabriel (january , ). ""they're being told to stay away from trump": after a day of violence and th amendment chatter, trump's allies are jumping ship". vanity fair. archived from the original on january , . retrieved january , . ^ a b c d e f g h "associated press timeline of events at the capitol, dead". wwsb. associated press. january , . archived from the original on january , . retrieved january , . ^ a b parker, ashley; dawsey, josh; rucker, philip (january , ). "six hours of paralysis: inside trump's failure to act after a mob stormed the capitol". archived from the original on january , . retrieved january , . ^ parker, ashley; dawsey, josh; rucker, philip (january , ). "six hours of paralysis: inside trump's failure to act after a mob stormed the capitol". the washington post. archived from the original on january , . retrieved january , . ^ gearan, anne; dawsey, josh (january , ). "trump issued a call to arms. then he urged his followers 'to remember this day forever!'". the washington post. archived from the original on january , . retrieved january , . ^ graziosi, graig (january , ). "'qanon congresswoman' lauren boebert faces calls to resign after tweeting information about nancy pelosi during capitol riot". independent. archived from the original on january , . retrieved january , . ^ rogers, katie (january , ). "a republican lawmaker for whom the spectacle is the point". the new york times. archived from the original on january , . retrieved january , . ^ itkowitz, colby (january , ). "the lockdown room was a safe space for lawmakers under siege. now some say maskless republicans made it a coronavirus hot spot". the washington post. archived from the original on january , . retrieved january , . ^ a b lamothe, dan; sonne, paul; leonnig, carol d.; davis, aaron c. (january , ). "army falsely denied flynn's brother was involved in key part of military response to capitol riot". the washington post. retrieved january , . ^ "dc mayor issues p.m. curfew following protests wednesday". cbs baltimore. january , . archived from the original on january , . retrieved january , . ^ "the latest: armed police clear out rioters, capitol complex 'secured'". news. january , . archived from the original on january , . retrieved january , . ^ "gov. northam issues state of emergency, institutes curfew in alexandria, arlington". wfxr-tv. january , . archived from the original on january , . ^ ainsley, julia (january , ). "armed dhs agents were on standby near capitol riot, were sent too late". nbc news. archived from the original on january , . retrieved january , . ^ sonne, paul; hermann, peter; ryan, missy (january , ). "pentagon placed limits on d.c. guard ahead of pro-trump protests due to narrow mission". the washington post. archived from the original on january , . retrieved january , . ^ witte, brian (january , ). "gov. hogan describes delayed permission to send maryland national guard". wrc-tv. archived from the original on january , . retrieved january , . ^ a b c cooper, helene (january , ). "army activates d.c. national guard to deploy troops to the capitol". the new york times. archived from the original on january , . retrieved january , . ^ schmitt, eric (january , ). "the entire d.c. national guard has been mobilized". the new york times. archived from the original on january , . retrieved january , . ^ mccarthy, tom; ho, vivian; greve, joan e (january , ). "trump supporters storm capitol as mcconnell warns of democracy 'death spiral'". the guardian. archived from the original on january , . retrieved january , . ^ julia, ainsley; de luce, dan; gains, mosheh (january , ). "pentagon, d.c. officials point fingers at each other over capitol riot response". nbc news. archived from the original on january , . retrieved january , . ^ walsh, joe (january , ). "reports: trump resisted sending national guard to quell violent mob at u.s. capitol". forbes. archived from the original on january , . retrieved january , . ^ haberman, maggie; cooper, helene (january , ). "trump rebuffed initial requests to deploy the national guard to the capitol. pence gave the go-ahead". the new york times. archived from the original on january , . retrieved january , . ^ williamson, jeff (january , ). "gov. northam sending virginia national guard and state troopers to washington, dc". wsls. archived from the original on january , . retrieved january , . ^ a b c hermann, peter (january , ). "d.c. police sent officers to the capitol during insurrection, spent $ . million during week of jan. ". the washington post. ^ a b c wilson, kristin; barrett, ted; raju, manu; zaslav, ali; fortinsky, sarah (january , ). "smoke grenades being deployed on senate side of the us capitol". cnn. archived from the original on january , . ^ felton, emmanuel (january , ). "these black capitol police officers describe fighting off "racist ass terrorists"". buzzfeed news. archived from the original on january , . retrieved january , . ^ a b c borger, julian (january , ). "democratic leaders call for trump's removal from office". the guardian. archived from the original on january , . retrieved january , . ^ barnes, julian e.; mazzetti, mark (january , ). "f.b.i. and homeland security make a show of force in washington". the new york times. archived from the original on january , . ^ "new jersey gov. phil murphy deploys state police to washington, d.c. after violent protesters storm u.s. capitol". kyw-tv. philadelphia. january , . archived from the original on january , . retrieved january , . ^ perez, evan (january , ). "congressional leaders are being evacuated from capitol complex". cnn. archived from the original on january , . ^ pramuk, jacob (january , ). "national guard will head to the capitol to tamp down pro-trump insurrection". cnbc. archived from the original on january , . retrieved january , . ^ "maryland troopers heading to dc to help quell unrest at us capitol, hogan says". cbs local. january , . archived from the original on january , . retrieved january , . ^ solender, andrew (january , ). "maryland governor says pentagon 'repeatedly denied' approval to send national guard to capitol". forbes. archived from the original on january , . retrieved january , . ^ wilson, kristin (january , ). "us capitol building is now secure, sergeant-at-arms says". cnn. archived from the original on january , . ^ benveniste, alexis (january , ). "capitol rioter to cnn: we could absolutely f***ing destroy you". cnn. archived from the original on january , . retrieved january , . ^ schor, elana (january , ). "anti-semitism seen in capitol insurrection raises alarms". associated press. archived from the original on january , . retrieved january , . ^ steinhauer, jennifer (january , ). "police in washington seize guns and arrest at least during violent capitol protest". the new york times. archived from the original on january , . retrieved january , . ^ romo, vanessa (january , ). "after chaos, insurrection and death, pro-trump rioters defy d.c. curfew". npr. archived from the original on january , . retrieved january , . ^ "gov. cuomo: capital riots a 'failed attempt at a coup,' ny national guard sent to dc". rochesterfirst. albany, n.y. january , . archived from the original on january , . retrieved january , . ^ levenson, michael (january , ). "washington, d.c. mayor issues order extending emergency for days". the new york times. archived from the original on january , . retrieved january , . ^ bowser, muriel (january , ), mayor's order - (pdf), archived (pdf) from the original on january , , retrieved january , ^ mclaughlin, eliott c. (january , ). "nine days police in dc arrested more people than they did during the capitol siege". cnn. archived from the original on january , . retrieved january , . ^ "dc police chief: two pipe bombs, cooler with molotov cocktails found on capitol grounds". kmgh. january , . archived from the original on january , . retrieved january , . ^ diaz, jaclyn; chappell, bill; moore, elena (january , ). "police confirm death of officer injured during attack on capitol". npr. archived from the original on january , . retrieved january , . ^ porter, tom (january , ). "trump supporter arrested at capitol had molotov cocktails: feds". business insider. archived from the original on january , . retrieved january , . ^ "alabama man arrested during capitol riots had molotov cocktails in mason jars, police say". wvtm . january , . archived from the original on january , . retrieved january , . ^ "roll call vote th congress – st session: on the objection (shall the objection submitted by the gentleman from arizona, mr. gosar, and the senator from texas, mr. cruz, and others be sustained? )". united states senate. january , . archived from the original on january , . retrieved january , . ^ "house rejects challenge to biden's electoral votes in arizona". pbs newshour. january , . archived from the original on january , . retrieved january , . ^ "final vote results for roll call ". clerk.house.gov. library of congress. january , . archived from the original on january , . retrieved january , . ^ mitchell, tia (january , ). "loeffler withdraws support for challenge, congress accepts biden win in georgia". atlanta journal constitution. archived from the original on january . retrieved january , . ^ "roll call vote th congress – st session: on the objection (shall the objection submitted by the gentleman from pennsylvania, mr. perry, and the senator from missouri, mr. hawley, be sustained? )". united states senate. january , . archived from the original on january , . retrieved january , . ^ "final vote results for roll call ". clerk.house.gov. library of congress. january , . archived from the original on january , . retrieved january , . ^ welch, matt (january , ). "amash's successor peter meijer: trump's deceptions are 'rankly unfit' – reason.com". reason. archived from the original on january , . retrieved january , . ^ meijer, peter (january , ). "rep. meijer: i experienced the heinous assault on capitol; now, time to face reality". detroit news. archived from the original on january , . retrieved january , . ^ "capitol attack: congress certifies joe biden's victory after deadly violence". bbc news. january , . archived from the original on january , . retrieved january , . ^ king, ledyard; groppe, maureen; wu, nicholas; jansen, bart; subramanian, courtney; garrison, joey (january , ). "pence confirms biden as winner, officially ending electoral count after day of violence at capitol". usa today. archived from the original on january , . retrieved january , . ^ wagner, meg; et al. (january , ). "congress finalizes biden's win after riot disrupts capitol". cnn. archived from the original on january , . retrieved january , . ^ a b c "loss of uscp officer brian d. sicknick" (press release). washington, d.c.: united states capitol police. january , . archived from the original on january , . retrieved january , . ^ safdar, khadeeja; ailworth, erin; seetharaman, deepa (january , ). "police identify five dead after capitol riot". the wall street journal. archived from the original on january , . retrieved january , . ^ a b c d peter hermann & julie zauzmer (january , ). "beaten, sprayed with mace and hit with stun guns: police describe injuries to dozens of officers during assault on u.s. capitol". the washington post. archived from the original on january , . retrieved january , . ^ evan hill, arielle ray and dahlia kozlowsky (january , ). "'they got a officer!': how a mob dragged and beat police at the capitol". new york times. archived from the original on january , . retrieved january , . ^ perez, evan; shortell, david; wild, whitney (february , ). "investigators struggle to build murder case in death of us capitol police officer brian sicknick". cnn. retrieved february , . ^ "trump to blame for death of woman trampled in capitol riot, family member says". reuters. january , . retrieved january , . ^ casiano, louis (january , ). "capitol police officer brian sicknick will lie in honor in rotunda: pelosi, schumer". fox news. retrieved january , . ^ cnn, peter nickeas, ryan nobles and ali zaslav. "pelosi and schumer say us capitol police officer killed during insurrection will lie in honor at capitol". cnn. retrieved january , . ^ a b jouvenal, justin; leonnig, carol (january , ). "ashli babbitt was shot during chaotic moments in the capitol". the washington post. archived from the original on january , . retrieved january , . ^ wagner, dennis; daniels, melissa; hauck, grace (january , ). "california woman killed during capitol riot was a military veteran and staunch trump supporter". usa today. archived from the original on january , . retrieved january , . babbitt served in the air force under the married name of ashli elizabeth mcentee ... she had been a staunch trump supporter ^ a b "capitol riots: a visual guide to the storming of congress". bbc news. january , . archived from the original on january , . retrieved january , . ^ "kusi news confirms identity of woman shot and killed inside us capitol". kusi-tv. january , . archived from the original on january , . retrieved january , . ^ kates, graham (january , ). "ashli babbitt identified as woman killed by police during u.s. capitol riots". cbs news. archived from the original on january , . retrieved january , . ^ barry, ellen; bogel-burroughs, nicholas; philipps, dave (january , ). "woman killed in capitol embraced trump and qanon". the new york times. archived from the original on january , . retrieved january , . ^ porter, tom (january , ). "qanon supporters believed marching on the capitol could trigger 'the storm,' an event where they hope trump's foes will be punished in mass executions". business insider. archived from the original on january , . retrieved january , . ^ zadrozny, brandy; gains, mosheh (january , ). "woman killed in capitol was trump supporter who embraced conspiracy theories". nbc news. archived from the original on january , . retrieved january , . ^ goldman, adam; dewan, shaila (january , ). "inside the deadly capitol shooting". the new york times. archived from the original on january , . retrieved january , . since ms. babbitt's death, far-right extremists and white supremacists have claimed her as a martyr and a "freedom fighter," even reproducing her image on flags and with anti-semitic imagery. many have demanded the release of the name of the officer who shot her. ^ palmer, ewan (january , ). "qanon supporters think capitol shooting victim ashli babbitt is still alive". newsweek. retrieved february , . ^ tan, rebecca; thompson, steve; olivo, antonio (january , ). "few details so far about deaths of 'medical emergency' victims in rioting at capitol". the washington post. archived from the original on january , . retrieved january , . ^ mascaro, lisa; tucker, eric; jalonick, mary clare; taylor, andrew (january , ). " dead as trump supporters stormed us capitol". wesh. associated press. archived from the original on january , . retrieved january , . ^ freiman, jordan (january , ). " dead after trump supporters storm u.s. capitol". cbs news. archived from the original on january , . retrieved january , . ^ hill, evan; ray, arielle; kozlowsky, dahlia (january , ). "videos show how rioter was trampled in stampede at capitol". the new york times. archived from the original on january , . retrieved january , . ^ thanawala, sudhin; dazio, stefanie; martin, jeff (january , ). "family: trump supporter who died followed qanon conspiracy". associated press. archived from the original on january , . retrieved january , . ^ dastagir, alia e. (january , ). "if someone you care about has been radicalized, here's what to know". usa today. archived from the original on january , . retrieved january , . ^ "the radicalization of kevin greeson". propublica. ^ a b kesslen, ben (january , ). "trump supporters who died during capitol riot left online presence". nbc news. archived from the original on january , . retrieved january , . ^ "ashli babbitt: the us veteran shot dead breaking into the capitol". bbc news. january , . archived from the original on january , . retrieved january , . ^ "founder of site trumparoo among dead at capitol". wpix. associated press. january , . archived from the original on january , . retrieved january , . ^ watkins, mea; packer, meghan; youngbauer, rhiannon (january , ). "kennesaw woman among people who died at d.c. riot". cbs news. archived from the original on january , . retrieved january , . ^ "a capitol police officer on duty during the coup attempt has died by suicide". buzzfeed. january , . a us capitol police officer on duty during wednesday's coup attempt by trump supporters died by suicide on saturday, his family has announced. ^ klein, allison; tan, rebecca (january , ). "capitol police officer who was on duty during the riot has died by suicide, his family says". the washington post. retrieved january , . ^ emma, caitlin; ferris, sarah (january , ). "second police officer died by suicide following capitol attack". politico. retrieved january , . ^ * sommerfeldt, chris (january , ). "pro-trump rioters smeared poop in u.s. capitol hallways during belligerent attack". daily news. new york. archived from the original on january , . retrieved january , . cineas, fabiola (january , ). "whiteness is at the core of the insurrection". vox. archived from the original on january , . retrieved january , . ^ a b c bahr, sarah (january , ). "curators scour capitol for damage to the building or its art". the new york times. archived from the original on january , . retrieved january , . ^ a b barlyn,suzanne (january , ). "u.s. taxpayers to pay capitol siege tab as government shuns insurance". reuters. archived from the original on january , . retrieved january , . ^ "mob swarms media outside capitol, damages equipment". nbc news. archived from the original on january , . retrieved january , . ^ burdyk, zack (january , ). "hoyer says rioters destroyed display commemorating john lewis". the hill. archived from the original on january , . retrieved january , . ^ "house majority leader says rioters stole tribute to late rep. john lewis". wfla. january , . archived from the original on january , . ^ wang, claire (january , ). "behind the viral photo of rep. andy kim cleaning up at midnight after riots". nbc news. archived from the original on january , . retrieved january , . ^ valentine, brittany (january , ). "prison labor is what will rebuild the u.s. capitol after its siege". al dÍa news. archived from the original on january , . retrieved january , . ^ kim, whizy. "white supremacists attacked the capitol. now, prison labor will clean up the mess". www.refinery .com. archived from the original on january , . retrieved january , . ^ "the furniture damaged in the capitol riot will almost certainly be rebuilt by incarcerated people for pennies". jezebel. archived from the original on january , . retrieved january , . ^ a b c bahr, sarah (january , ). "first inventory of damage to u.s. capitol building released". the new york times. archived from the original on january , . retrieved january , . ^ a b castronuovo, celine (january , ). "flags, signs and other items left behind in capitol riot to be preserved as historical artifacts". the hill. archived from the original on january , . retrieved january , . ^ mosk, matthew; margolin, josh; meek, james gordon; mallin, alexander (january , ). "investigators combing through massive us capitol crime scene in wake of pro-trump riot". abc news. archived from the original on january , . retrieved january , . ^ venhuizen, harm (january , ). "insurrectionist 'zip-tie guy' identified as retired air force lieutenant colonel". military times. archived from the original on january , . retrieved january , . ^ schogol, jeff (january , ). "the capitol hill insurrection reveals veterans are at war against themselves: veterans vs. veterans". task and purpose. archived from the original on january , . retrieved january , . ^ wagner, meg; macaya, melissa (january , ). "clyburn's spokesperson tells cnn they found his ipad, was not taken by rioters". cnn. archived from the original on january , . retrieved january , . ^ a b miller, maggie (january , ). "laptop stolen from pelosi's office during capitol riots". the hill. archived from the original on january , . retrieved january , . ^ black, john; sotak, j. w.; spoonts, sean (january , ). "computers with access to classified material stolen from capitol". sofrep. archived from the original on january , . retrieved january , . ^ "news & notices". www.aoc.gov. architect of the capitol. archived from the original on january , . retrieved january , . ^ krazit, tom (january , ). "don't worry about the cybersecurity fallout of the capitol breach". protocol. archived from the original on january , . retrieved january , . ^ "woman charged in theft of pelosi's laptop released from jail". npr.org. archived from the original on january , . retrieved january , . ^ "statement from the elizabeth macmillan director anthea m. hartig". national museum of american history. january , . archived from the original on january , . retrieved january , . ^ cascone, sarah (january , ). "the smithsonian is rushing to collect flags, protest signs, and other ephemera from the pro-trump insurrection in the us capitol". artnet news. archived from the original on january , . retrieved january , . ^ jack durschlag (january , ). "trump calls for healing, smooth transition after 'heinous attack' on capitol". fox news. archived from the original on january , . ^ choi, matthew (january , ). "trump condemns violence in capitol riots, more than hours later". politico. archived from the original on january , . retrieved january , . ^ sherman, gabriel (january , ). "'he knew he f – ked up': facing legal and political peril, trump is turning on even his most devoted allies". vanity fair. retrieved january , . ^ choi, matthew (january , ). "mcenany tries to distance administration from capitol riots". politico. archived from the original on january , . retrieved january , . ^ broadwater, luke; fandos, nicholas; haberman, maggie (january , ). "democrats ready impeachment charge against trump for inciting capitol mob". the new york times. retrieved january , . ^ a b "read: military joint chiefs statement condemning 'sedition and insurrection' at us capitol". cnn. january , . ^ macias, amanda (january , ). "top military leaders condemn 'sedition and insurrection' at capitol, acknowledge biden win". cnbc. ^ choi, david (january , ). "top military officers condemn the capitol hill riots and tell troops to 'stay ready'". business insider. retrieved january , . ^ a b haberman, maggie; schmidt, michael s. (january , ). "trump has not lowered flags in honor of an officer who died from injuries sustained amid the riot". the new york times. retrieved january , . while the flags at the capitol have been lowered, mr. trump has not issued a similar order for federal buildings under his control. ... mr. trump has not reached out to mr. sicknick's family, although vice president mike pence called to offer condolences, an aide to mr. pence said. ^ elis, niv (january , ). "pelosi orders flags at half-staff for capitol officer who died". the hill. retrieved january , . ^ haberman, maggie (january , ). "after refusing to do so, trump orders flags to be flown at half-staff". the new york times. retrieved january , . despite widespread criticism, mr. trump had refused to lower the flags, but relented on january . ^ diamond, jeremy; leblanc, paul (january , ). "white house orders flags lowered to honor late police officers who responded to us capitol breach". cnn. retrieved january , . ^ "white house advisers: trump has no intention of resigning". kobi-tv. january , . retrieved january , . ^ "father of slain capitol officer hopes son's death brings end to 'lunacy'". nbc news. ^ rolli, bryan (january , ). "after capitol police officer dies, #tedcruzkilledacop trends over cruz inciting rioters". the daily dot. retrieved january , . ^ "capitol riots: boris johnson condemns donald trump for sparking events". bbc news. january , . retrieved january , . ^ ward, james (january , ). "taoiseach and foreign affairs minister react to washington protests". echo live. archived from the original on january , . retrieved january , . ^ "us capitol riots: world leaders react to 'horrifying' scenes in washington". bbc news. january , . archived from the original on january , . retrieved january , . ^ "the populists finally breaking with trump". the atlantic. january , . ^ prothero, mitch (january , ). "some among america's military allies believe trump deliberately attempted a coup and may have had help from federal law-enforcement officials". business insider. archived from the original on january , . retrieved january , . ^ * milibank, dana (january , ). "president trump has committed treason". the washington post. retrieved january , . ^ multiple sources: rutz, david (january , ). "conservative media members erupt with anger over protesters storming capitol: 'this is domestic terrorism'". fox news. archived from the original on january , . retrieved january , . dreier, natalie (january , ). "capitol violence: what is sedition?". cox media group national content desk. archived from the original on january , . retrieved january , – via kiro. trepany, charles (january , ). "'this is domestic terrorism': meghan mccain, cardi b and more react to capitol riot". usa today. archived from the original on january , . retrieved january , . ^ sources that refer to the event as a coup include: brewer, ray (january , ). "trump hardliners, including in nevada, paint themselves as victims, vow protests aren't over". las vegas sun. retrieved january , . graham, david a. (january , ). "this is a coup". the atlantic. archived from the original on january , . retrieved january , . musgrave, paul (january , ). "this is a coup. why were experts so reluctant to see it coming?". foreign policy. archived from the original on january , . retrieved january , . solnit, rebecca (january , ). "call it what it was: a coup attempt". the guardian. archived from the original on january , . retrieved january , . jacobson, louis (january , ). "politifact – is this a coup? here's some history and context to help you decide". politifact. retrieved january , . a good case can be made that the storming of the capitol qualifies as a coup. it's especially so because the rioters entered precisely when the incumbent's loss was to be formally sealed, and they succeeded in stopping the count trouvé, matthieu. "washington : un coup d'etat populiste". libération (in french). archived from the original on january , . retrieved january , . rotili, lavinia. "insurrection à washington : le terme "coup d'etat" peut-il être utilisé ?". rtbf (in french). archived from the original on january , . retrieved january , . lönnaeus, olle. "usa-experten: trump uppviglade till en "mjuk statskupp"". helsingborgs dagblad (in swedish). archived from the original on january , . retrieved january , . pitzke, marc (january , ). "ein amerikanisches verbrechen". spiegel online. archived from the original on january , . retrieved january , . der putschversuch der wahlverlierer ist das grotesk anmutende, aber erschütternde finale eines vierjährigen, autokratischen fiebertraums, dessen unheimlicher regisseur donald trump war, der selbst ernannte law-and-order-präsident, assistiert von der zusehends radikalisierten republikanischen partei und rechtsextremen propagandamedien. [the attempted coup by the election losers is the grotesque but harrowing finale to a four-year autocratic fever dream whose sinister director was donald trump, the self-styled law-and-order president, assisted by the visibly radicalized republican party and far-right propaganda media.] burman, tony (january , ). "despite coup disgrace, trump probably isn't done yet". toronto star. archived from the original on january , . retrieved january , . ^ coleman, justine (january , ). "gop lawmaker on violence at capitol: 'this is a coup attempt'". the hill. archived from the original on january , . retrieved january , . ^ call, charles t. (january , ). "no, it's not a coup – it's a failed 'self-coup' that will undermine us leadership and democracy worldwide". brookings institution. retrieved january , . trump's measures to overturn the elections since november constitute a 'coup', as they involve illegal usurpation of state power, even when it may not involve the use of force. yet it is a 'self'-coup because it is perpetrated by the head of government rather than military officers or others against that chief executive. ^ borger, julian (january , ). "maga mob's capitol invasion makes trump's assault on democracy literal". the guardian. retrieved january , . ^ ellyatt, holly (january , ). "'anarchy in the usa': global media reacts to capitol chaos". wnbc. retrieved january , . ^ "us capitol riots: how the world's media reacted to trump, his supporters and the storming of congress". the west australian. january , . retrieved january , . ^ ingraham, christopher (january , ). "how experts define the deadly mob attack at the u.s. capitol". the washington post. retrieved january , . ^ hill, fiona (january , ). "yes, it was a coup attempt. here's why". politico. retrieved january , . ^ mclaughlin, eliott c. "with coup label, capitol rioters join communist party in plotting against usa, university project says". cnn. retrieved february , . ^ "coup d'état project (cdp) | cline center". clinecenter.illinois.edu. retrieved february , . ^ twitter, inc. (january , ). "permanent suspension of @realdonaldtrump". blog.twitter.com. retrieved january , . ^ conger, kate; isaac, mike (january , ). "twitter permanently bans trump, capping online revolt". the new york times. issn - . retrieved january , . ^ denham, hannah (january , ). "these are the platforms that have banned trump and his allies". washington post. issn - . retrieved january , . ^ raju, manu; barrett, ted (january , ). "facing criticism, us capitol police details response to mob, suspects arrested and officers injured". cnn. archived from the original on january , . retrieved january , . ^ thrush, glenn; dewan, shaila; eligon, john; macfarquhar, neil (january , ). "questions mount over law enforcement's failure to protect the capitol". the new york times. ^ leonnig, carol d.; davis, aaron c.; lamothe, dan; fahrenthold, david a. (january , ). "capitol breach prompts urgent questions about security failures". the washington post. archived from the original on january , . retrieved january , . ^ raju, manu (january , ). "members of congress to get increased security while traveling through airports". cnn. archived from the original on january , . retrieved january , . ^ wagner, meg; macaya, melissa; hayes, mike; mahtani, melissa (january , ). "tsa increases security measures on dc flights, including some second id checks". cnn. archived from the original on january , . retrieved january , . ^ "the latest: pelosi taps retired army general for review". star tribune. retrieved january , . ^ sophia barnes; staff (january , ). "inauguration perimeter to restrict access to downtown dc amid extended security clampdown". nbc washington. retrieved january , . ^ "metro to close stations for a week amid threats of inauguration violence". archived from the original on january , . retrieved january , . ^ "metro announces inauguration service plans, station closures | wmata". www.wmata.com. retrieved january , . ^ "temporary detour: presidential inauguration, january - | wmata". www.wmata.com. retrieved january , . ^ "dc hotels won't close during inauguration week after airbnb canceled all area reservations amid worries of renewed violence". business insider. ^ villegas, paulina; chason, rachel; knowles, hannah (january , ). "storming of capitol was textbook potential coronavirus superspreader, experts say". the washington post. retrieved january , . ^ jones, roxanne (january , ). "this is what it looks like when toxic white privilege is left unchecked". cnn. retrieved january , . ^ kendi, ibram x. (january , ). "police response at the capitol brings claims of 'white privilege' | pbs newshour" (interview). interviewed by amna nawaz. ^ mcgirt, ellen; jenkins, aric (january , ). "don't talk about the capitol siege without mentioning white privilege". fortune. ^ beckett, lois (january , ). "us police three times as likely to use force against leftwing protesters, data finds". guardian. retrieved january , . ^ "us crisis monitor | acled". july , . ^ ruger, todd (january , ). "calls for impeachment, prosecution of president after pro-trump mob storms capitol". archived from the original on january , . retrieved january , . ^ doherty, ben (january , ). "woman shot and killed in storming of us capitol named as ashli babbitt". the guardian. archived from the original on january , . ^ a b c mallin, alexander; barr, luke (january , ). "doj has identified suspects, charged in capitol riot". abc news. retrieved january , . ^ harris, craig (january , ). "former arizona olympian klete keller, charged in u.s. capitol riot, turns himself in to feds". the arizona republic. retrieved january , . ^ "'proud boys hawaii' leader to appear in federal court for alleged involvement in us capitol riot". khon . january , . archived from the original on january , . retrieved january , . ^ a b fazio, marie (january , ). "notable arrests after the riot at the capitol". the new york times. retrieved january , . ^ balsamo, michael (january , ). "far-right personality 'baked alaska' arrested in riot probe". associated press. archived from the original on january , . retrieved january , . ^ donaghue, erin (january , ). "son of brooklyn judge arrested in capitol attack". cbs news. retrieved january , . ^ a b c jennifer valentino-devries, grace ashford, denise lu, eleanor lutz, alex leeds matthews & karen yourish, arrested in capitol riot: organized militants and a horde of radicals, new york times (february , ). ^ multiple sources: "h.res. – impeaching donald john trump, president of the united states, for high crimes and misdemeanors". th united states congress. weiland, noah (january , ). "impeachment briefing: 'incitement of insurrection'". the new york times. moe, alex; shabad, rebecca; gregorian, dareh (january , ). "lawmakers called back to d.c. to vote on trump's impeachment, removal under th amendment". nbc news. ^ moe, alex; shabad, rebecca (january , ). "'he threatened the integrity of the democratic system': house introduces one article of impeachment against trump". nbc news. retrieved january , . ^ "read the house article of impeachment against president trump". los angeles times. january , . retrieved january , . ^ stolberg, sheryl gay (january , ). "grieving son's death, maryland lawmaker fights to impeach trump". the new york times. issn - . retrieved january , . ^ pelosi, nancy (january , ). "pelosi names impeachment managers". speaker of the united states house of representatives. retrieved january , . ^ wagner, meg; macaya, melissa; hayes, mike; mahtani, melissa; alfonso, fernando, iii; rocha, veronica (january , ). "trump impeachment vote: live coverage from the house of representatives". cnn. retrieved january , . ^ honig, elie (december , ). "the trump administration is hiding something". cnn digital. retrieved january , . ^ cohen, zachary; wild, whitney (january , ). "fbi warns 'armed protests' being planned at all state capitols and in washington dc". cnn. archived from the original on january , . retrieved january , . ^ dickson, caitlin (january , ). "exclusive: fbi warns of potential boogaloo violence during jan. rallies". news.yahoo.com. archived from the original on january , . retrieved january , . ^ mack, justin (january , ). "about trump supporters, proud boys rally against election results at indiana capitol". indianapolis star. archived from the original on january , . retrieved january , . ^ bornhoft, william (january , ). "trump supporters hold 'storm the capitol' rally in st. paul". mendota heights, mn patch. archived from the original on january , . retrieved january , . ^ "carson city protest peaceful, filled with pro-trump messages". klas. associated press. january , . archived from the original on january , . retrieved january , . ^ dunker, chris (january , ). "hundreds gather at state capitol to support trump's efforts to overturn election". lincoln journal star. archived from the original on january , . retrieved january , . ^ ferenchik, mark (january , ). "tempers short, emotions high at protest over election results at statehouse in columbus". the columbus dispatch. archived from the original on january , . retrieved january , . ^ mccausland, phil; burke, minyvonne; kaplan, ezra; lozano, alicia victoria (january , ). "protesters gather outside state capitols nationwide as chaos sweeps congress". nbc news. archived from the original on january , . retrieved january , . ^ "georgia secretary of state evacuates georgia capitol for safety, office closes". wxia-tv. archived from the original on january , . retrieved january , . ^ reding, shawna m. (january , ). "texas capitol grounds in austin closed as trump supporters storm us capitol". kvue. archived from the original on january , . retrieved january , . ^ cremen, alanea (january , ). " arrested for having pepper spray at capitol demonstrations | updates". abc .com. archived from the original on january , . retrieved january , . ^ wisckol, martin; bermont, bradley; rasmussen, emily (january , ). "protesters clash in downtown l.a., while trump rallies unfold elsewhere in southern california". the mercury news. archived from the original on january , . retrieved january , . ^ "'it's disgraceful': black woman attacked during pro-trump demonstration in dtla". kcal. january , . archived from the original on january , . retrieved january , . ^ "lapd 'aware of disturbing images' of black woman grabbed during protest". knbc. january , . archived from the original on january , . retrieved january , . ^ a b c "fact check: protestors did not 'storm' georgia and kansas statehouses". reuters. january , . archived from the original on january , . retrieved january , . ^ a b c michael sasso (january , ). "georgia election official avoids ex-klansman amid safety concern". retrieved january , . ^ "georgia secretary of state evacuates georgia capitol for safety, office closes". alive.com. archived from the original on january , . retrieved january , . ^ "one arrested as trump supporters gather at oklahoma capitol". ap news. january , . archived from the original on january , . retrieved january , . ^ jones, perris (january , ). "hundreds peacefully protest at oklahoma capitol in support of president donald trump". koco. archived from the original on january , . retrieved january , . ^ swindler, samantha; killen, dave; nakamura, beth (january , ). "unlawful assembly declared at oregon capitol as pro-trump mob storms u.s. congress". the oregonian. associated press. archived from the original on january , . retrieved january , . ^ bernton, hal (january , ). "trump supporters open gate at governor's mansion in olympia, occupy front lawn". the seattle times. archived from the original on january , . retrieved january , . ^ horcher, gary (january , ). "protesters break through gates at governor's mansion in olympia, storm to front door". kiro-tv. seattle. archived from the original on january , . retrieved january , . ^ barton, april (january , ). "state prepares for protests in montpelier over weekend and for inauguration day". the burlington free press. archived from the original on january , . retrieved january , . ^ a b taylor, adam (january , ). "trump's 'stop the steal' message finds an international audience among conspiracy theorists and suspected cults". the washington post. archived from the original on january , . retrieved january , . ^ little, simon (january , ). "small pro-trump rallies break out in canada amid chaos at u.s. capitol". global news. archived from the original on january , . retrieved january , . ^ daflos, penny (january , ). "photographers attacked at pro-trump rally in downtown vancouver". ctv news. archived from the original on january , . retrieved january , . ^ walls, jason (january , ). "police keeping a close eye on controversial billy tk 'freedom rally' outside beehive". the new zealand herald. archived from the original on january , . retrieved january , . ^ "billy te kahika spreads covid- misinformation at parliament rally". radio new zealand. january , . archived from the original on january , . retrieved january , . external links storming of the united states capitolat wikipedia's sister projects media from wikimedia commons news from wikinews quotations from wikiquote wikisource has original text related to this article: trump's rally speech capitol riot arrests: see who's been charged across the u.s. — tracker database created and updated by usa today what parler saw during the attack on the capitol (video archive from propublica) leatherby, lauren; singhvi, anjali (january , ). "critical moments in the capitol siege". the new york times. archived from the original on january , . (detailed timeline) bennett, dalton; brown, emma; cahlan, sarah; lee, joyce sohyun; kelly, meg; samuels, elyse; swaine, jon (january , ). " minutes of fear: a video timeline from inside the capitol siege". the washington post. archived from the original on january , . (video timeline) us capitol stormed, collected news and commentary. bbc news online. save america rally speeches (video) fbi seeking information related to violent activity at the u.s capitol building – fbi h. res. – impeaching donald john trump, president of the united states, for high crimes and misdemeanors (article of impeachment adopted by the house on january , ) h.res. – condemning and censuring representative mo brooks of alabama (censure resolution introduced on january , , by representative tom malinowski, with cosponsors) the full text of article of impeachment against donald j. trump ( ) at wikisource v t e storming of the united states capitol timeline background capitol building united states capitol united states capitol police eugene goodman brian sicknick steven sund election – presidential election protests electoral college vote count attempts to overturn the u.s. presidential election republican reactions to donald trump's claims of election fraud alt-tech apps used in the planning gab parler patriots.win participants jake angeli joe biggs derrick evans tim "baked alaska" gionet klete keller matt maddock rick saccone jon schaffer john earle sullivan‎‎ terri lynn weaver aftermath u.s. inauguration week protests companies that halted political contributions reactions domestic international second impeachment of donald trump trial related alex jones ali alexander covid- pandemic in the united states donald trump jericho march qanon rudy giuliani sedition caucus social media use by donald trump trumpism v t e ( ←) united states presidential election (→ ) joe biden, kamala harris (d), electoral votes; donald trump, mike pence (r), electoral votes conduct and aftermath background and conduct united states elections polls national state timeline after election day general election debates postal voting social media fundraising international reactions presidential electors electoral college vote count presidential transition of joe biden disputes attempts to overturn protests storming of the united states capitol international reactions second impeachment of donald trump trial lawsuits filed before election day filed during or after election day arizona georgia michigan nevada pennsylvania wisconsin texas v. pennsylvania participants parties and candidates democratic party independent party working families party primaries candidates debates forums results convention polls national state political positions endorsements automatic delegates vp candidate selection biden-sanders unity task forces candidates nominee joe biden campaign endorsements primary representative positions vp nominee: kamala harris positions withdrawn during primaries michael bennet michael bloomberg campaign endorsements positions pete buttigieg campaign endorsements positions tulsi gabbard campaign positions amy klobuchar campaign endorsements positions deval patrick bernie sanders campaign endorsements positions media coverage tom steyer elizabeth warren campaign endorsements positions andrew yang campaign endorsements withdrawn before primaries cory booker campaign endorsements positions steve bullock julian castro bill de blasio john delaney campaign positions kirsten gillibrand positions mike gravel campaign kamala harris campaign endorsements positions john hickenlooper ami horowitz jay inslee campaign wayne messam seth moulton richard ojeda beto o'rourke campaign tim ryan joe sestak eric swalwell marianne williamson campaign republican party conservative party right to life party primaries results debates convention polls endorsements candidates nominee donald trump campaign endorsements political non-political positions republican opposition republican reactions to trump's election fraud claims vp nominee: mike pence withdrawn during primaries rocky de la fuente joe walsh campaign bill weld campaign withdrawn before primaries mark sanford campaign libertarian party primaries convention candidates nominee jo jorgensen campaign endorsements vp nominee: spike cohen eliminated in balloting jim gray jacob hornberger adam kokesh john mcafee campaign john monds vermin supreme campaign arvin vohra withdrawn during primaries max abramson justin amash ken armstrong lincoln chafee mark whitney withdrawn before primaries zoltan istvan green party legal marijuana now party socialist alternative socialist party usa primaries convention results candidates nominee howie hawkins campaign endorsements vp nominee: angela walker withdrawn during primaries dario hunter david rolde alaska green party nominee jesse ventura vp nominee: cynthia mckinney constitution party primaries candidates nominee don blankenship vp nominee william mohr independent candidates candidates declared pete accetturo mark charles brock pierce jade simmons joe schriner withdrawn perry caravello jeremy gable ronnie kroell other third-party candidates alliance party american independent party reform party nominee rocky de la fuente vp nominee darcy richardson american independent party vp nominee kanye west other candidates max abramson phil collins american solidarity party nominee brian t. carroll other candidates joe schriner birthday party nominee kanye west campaign vp nominee michelle tidball bread and roses nominee jerome segal party for socialism and liberation liberty union party peace & freedom party nominee gloria la riva withdrawn vp nominee leonard peltier other candidates howie hawkins progressive party nominee dario hunter prohibition party nominee phil collins socialist action nominee jeff mackler socialist equality party nominee joseph kishore socialist workers party nominee alyson kennedy v t e presidency of donald trump campaigns presidential campaign presidential campaign make america great again rallies republican primaries endorsements debates convention general election endorsements debates reactions assassination attempt never trump movement mitt romney's anti-trump speech republican opposition in republican opposition in sexual misconduct allegations access hollywood recording wiretapping allegations presidential campaign endorsements political non-political rallies republican reactions to trump's election fraud claims tenure transition inauguration speech opinion polling social media midterm elections government shutdowns january – executive actions proclamations executive clemency pardon of joe arpaio travel bans reactions legal challenges protests replacement family separation policy migrant detentions operation faithful patriot stormy daniels scandal anonymous senior official op-ed relations with israel jerusalem recognition golan heights recognition peace plan abraham accords uae bahrain sudan morocco kosovo–serbia agreement north korea crisis singapore summit hanoi summit dmz summit – persian gulf crisis trump–ukraine scandal relations with russia helsinki summit russian bounty program saudi arabia arms deal syria strikes taiwan call withdrawal from the iran deal death of abu bakr al-baghdadi assassination of qasem soleimani covid- pandemic communication federal government data breach timeline first days q q q q q q q q q q q q q q q q january trips – international riyadh summit singapore summit helsinki summit hanoi summit dmz summit policies economy tax cuts tariffs china trade war environment paris agreement withdrawal foreign policy as candidate as president america first immigration racial views social issues cannabis personnel cabinet formation federal judges gorsuch kavanaugh barrett supreme court candidates controversies ambassadors attorneys economic advisors withdrawn appointees short-tenure appointees dismissals and resignations attorneys yates comey tillerson mcmaster pruitt inspectors general notable events business projects in russia links with russian officials trump tower meeting steele dossier crossfire hurricane classified information disclosures russian election interference timeline until july july –election day transition period special counsel investigation mueller report barr letter timeline of investigations transition period january–june july–december january–june july–december – impeachments efforts impeachment inquiry first impeachment first trial second impeachment second trial photo op at st. john's church white house covid- outbreak four seasons total landscaping press conference – u.s. election protests storming of the u.s. capitol timeline aftermath continued protests reactions domestic international opposition lawsuits protests timeline women's march march for science impeachment march family separation policy protests v t e coups, self-coups, and attempted coups since list of coups and coup attempts by country since s mali ( )c lesotho ( )c thailand ( )c soviet union ( ) haiti ( )c georgia ( – )c venezuela ( ) peru ( )‡c sierra leone ( )c algeria ( ) guatemala ( )‡ azerbaijan ( )c russia ( )‡c libya ( ) burundi ( ) nigeria ( )c gambia ( )c lesotho ( )‡ azerbaijan ( ) qatar ( )c são tomé and príncipe ( ) iraq ( ) burundi ( )c niger ( )c qatar ( ) cambodia ( )c turkey ( )c zambia ( ) lesotho ( ) republic of the congo ( ) niger ( )c pakistan ( )c côte d'ivoire ( )c s fiji ( )c ecuador ( )c burundi ( ) central african republic ( ) venezuela ( ) central african republic ( ) mauritania ( ) philippines ( ) guinea-bissau ( ) são tomé and príncipe ( ) chad ( ) haiti ( )c equatorial guinea ( ) mauritania ( )c philippines ( ) chad ( ) thailand ( )c madagascar ( ) fiji ( )c laos ( ) philippines ( ) turkey ( ) east timor ( ) mauritania ( )c guinea ( )c madagascar ( )c honduras ( )c s niger ( )c guinea-bissau ( )c ecuador ( ) madagascar ( ) democratic republic of the congo ( ) guinea-bissau ( ) bangladesh ( ) papua new guinea ( ) mali ( )c malawi ( ) guinea-bissau ( )c côte d'ivoire ( ) sudan ( ) central african republic ( – ) benin ( ) libya (april ) comoros ( ) chad ( ) egypt ( )c libya (october ) south sudan ( ) libya ( ) thailand ( )c lesotho ( ) gambia ( ) yemen ( – )c sri lanka ( ) burundi ( ) burkina faso ( ) turkey ( ) burkina faso ( ) montenegro ( ) libya ( ) gambia ( – )‡ zimbabwe ( )c equatorial guinea ( ) yemen ( )c sri lanka ( )‡ gabon ( ) sudan ( )c ethiopia ( ) s mali ( )c myanmar ( )c c successful coup ‡ self-coup no sign for attempted coup v t e coups, self-coups, and attempted coups in the americas since s paraguay ( )c haiti ( )c paraguay ( )c paraguay ( )c paraguay ( )c venezuela ( )c haiti ( )c s paraguay ( )c haiti ( )c mexico ( )c paraguay (jan )c paraguay (feb )c paraguay (mar )c mexico ( )c haiti (jan )c peru ( )c haiti (nov )c haiti ( )c costa rica ( )c peru ( )c s bolivia ( )c mexico ( )c chile ( )c chile ( )c ecuador ( )c s bolivia ( )c peru ( )c argentina ( )c brazil ( )c panama ( )c el salvador ( )c chile (jul )c chile (sep ) chile (dec ) chile (jun )c chile (sep )c uruguay ( )‡ cuba ( )c bolivia ( )c ecuador ( ) paraguay ( )c bolivia ( )c ecuador ( ) bolivia ( )c brazil ( )‡ paraguay ( )c chile ( ) chile ( ) s paraguay ( )‡ panama ( )c uruguay ( )‡ argentina ( )c bolivia ( )c el salvador (apr ) el salvador (may )c el salvador (oct )c brazil ( )c venezuela ( )c bolivia ( )c haiti ( )c peru ( )c venezuela ( )c el salvador ( )c paraguay ( )c paraguay (jan )c paraguay (feb )c paraguay (sep )c s haiti ( )c argentina ( ) panama ( )c bolivia ( )‡ cuba ( )c bolivia ( )c colombia ( )c paraguay ( )c guatemala ( )c argentina ( )c brazil ( ) brazil ( ) honduras ( )c haiti ( ) venezuela ( )c brazil ( ) s el salvador ( )c el salvador ( )c ecuador ( )c argentina ( )c peru ( )c guatemala ( )c ecuador ( )c dominican republic ( )c honduras ( )c bolivia ( )c brazil ( )c argentina ( )c panama ( )c peru ( )c bolivia ( )c brazil ( )c s bolivia ( )c bolivia ( )c honduras ( )c ecuador ( )c el salvador ( ) uruguay ( )c‡ chile (jun ) chile (sep )c bolivia ( )‡ peru ( )c honduras ( )c ecuador ( ) argentina ( )c ecuador ( )c honduras ( )c bolivia (jul )c bolivia (nov )c el salvador ( )c bolivia ( )c s suriname ( )c bolivia ( )c guatemala ( )c guatemala ( )c bolivia ( ) haiti (jun )c haiti (sep )c haiti ( ) panama ( ) paraguay ( )c s suriname ( )c haiti ( )c venezuela ( ) peru ( )‡ guatemala ( )‡ s ecuador ( ) venezuela ( ) haiti ( ) c honduras ( ) c s ecuador ( ) s el salvador ( ) united states ( ) c: successful coup ‡ self-coup no sign for attempted coup v t e covid- pandemic in the united states timeline data by state chart locations states and d.c. alabama alaska arizona arkansas california timeline s.f. bay area colorado connecticut delaware florida georgia hawaii idaho illinois indiana iowa kansas kentucky louisiana maine maryland massachusetts timeline boston timeline michigan minnesota mississippi missouri montana nebraska nevada new hampshire new jersey new mexico new york new york city north carolina north dakota ohio columbus oklahoma oregon portland pennsylvania philadelphia rhode island south carolina south dakota tennessee texas timeline austin utah vermont virginia washington washington, d.c. white house west virginia wisconsin statistics wyoming other areas american samoa guantanamo bay naval base immigration detention guam navajo nation northern mariana islands puerto rico u.s. virgin islands ships grand princess uss theodore roosevelt responses government response pre-pandemic exercises crimson contagion event federal government response trump administration communication operation warp speed biden administration policy state and local government responses eastern states multi-state council western states pact midwest governors regional pact house select coronavirus crisis subcommittee white house coronavirus task force covid- advisory board great american economic revival industry groups hospital ships usns mercy usns comfort coronavirus preparedness and response supplemental appropriations act families first coronavirus response act oval office address cares act paycheck protection program pandemic response accountability committee special inspector general for pandemic recovery covid- congressional oversight commission paycheck protection program and health care enhancement act heroes act heals act private response covid act now covid tracking project musicares covid- relief fund trump death clock media rock: a one-time special acting for a cause digital drag fest home movie: the princess bride iheart living room concert for america make it work! saturday night seder saturday night live at home sesame street: elmo's playdate the disney family singalong volume ii take me to the world: a sondheim th celebration together at home saving our selves a parks and recreation special rise up new york! "u.s. deaths near , , an incalculable loss" dear class of graduation graduate together: america honors the high school class of we are one: a global film festival global goal: unite for our future a killer party the disney holiday singalong ratatouille the musical th tony awards impacts social presidential election postal voting abortion african-american communities congressional insider trading scandal native american communities the walt disney company television programs affected foster care protests open the states george floyd election storming of the united states capitol face masks suspension of the – nba season zoom video communications economic stock market crash recession meat industry restaurant industry strikes alabama aluminum plant strike bath shipbuilders strike essential workers strike hunts point produce market strike michigan grad students strike new orleans sanitation strike notable people federal jerome adams scott atlas alex azar joe biden deborah birx rick bright anthony fauci pete gaynor brett giroir stephen hahn nancy messonnier peter navarro mike pence gustave f. perna robert r. redfield todd t. semonite moncef slaoui donald trump scott turner state greg abbott amy acton kate brown andrew cuomo ron desantis mike dewine larry hogan jay inslee brian kemp ned lamont phil murphy gavin newsom gretchen whitmer tom wolf researchers carl bergstrom natalie dean eric feigl-ding scott gottlieb akiko iwasaki marc lipsitch syra madad angela rasmussen caitlin rivers devi sridhar modern history portal politics portal society portal united states portal retrieved from "https://en.wikipedia.org/w/index.php?title= _storming_of_the_united_states_capitol&oldid= " categories: storming of the united states capitol th united states congress s coups d'état and coup attempts controversies in the united states in american politics in washington, d.c. protests attacks on buildings and structures in the s attacks on buildings and structures in the united states attacks on legislatures attempted coups d'état attacks in controversies of the united states presidential election coups d'état and coup attempts in the united states crimes against police officers in the united states deaths by firearm in washington, d.c. electoral violence far-right politics in the united states fascism in the united states january crimes january events in the united states occupations (protest) political riots in the united states presidency of donald trump riots and civil disorder in washington, d.c. trump administration controversies united states capitol united states capitol police riots hidden categories: cs maint: uses authors parameter cs french-language sources (fr) cs swedish-language sources (sv) articles with short description short description is different from wikidata wikipedia indefinitely move-protected pages wikipedia pages semi-protected against vandalism use american english from january all wikipedia articles written in american english use mdy dates from january pages using multiple image with auto scaled images coordinates on wikidata pages using collapsible list without both background and text-align in titlestyle pages using infobox civil conflict with unknown parameters articles containing video clips npov disputes from january navigation menu personal tools not logged in talk contributions create account log in namespaces article talk variants views read view source view history more search navigation main page contents current events random article about wikipedia contact us donate contribute help learn to edit community portal recent changes upload file tools what links here related changes upload file special pages permanent link page information cite this page wikidata item print/export download as pdf printable version in other projects wikimedia commons wikiquote languages afrikaans العربية azərbaycanca Беларуская bosanski català Čeština cymraeg dansk deutsch eesti Ελληνικά emiliàn e rumagnòl español esperanto euskara فارسی français gaeilge galego 한국어 Հայերեն hrvatski bahasa indonesia interlingua Íslenska italiano עברית ქართული magyar bahasa melayu မြန်မာဘာသာ nederlands 日本語 norsk bokmål polski português română Русский shqip simple english slovenščina Ślůnski کوردی suomi svenska ၽႃႇသႃႇတႆး ไทย türkçe Українська tiếng việt walon ייִדיש 粵語中文 edit links this page was last edited on february , at : (utc). text is available under the creative commons attribution-sharealike license; additional terms may apply. by using this site, you agree to the terms of use and privacy policy. wikipedia® is a registered trademark of the wikimedia foundation, inc., a non-profit organization. privacy policy about wikipedia disclaimers contact wikipedia mobile view developers statistics cookie statement the open library blog | a web page for every book the open library blog a web page for every book skip to content about « older posts introducing the open library explorer by mek | published: december , try it here! if you like it, share it. bringing years of librarian-knowledge to life by nick norman with drini cami & mek at the library leaders forum (demo), open library unveiled the beta for what it’s calling the library explorer: an immersive interface which powerfully recreates and enhances the experience of navigating a physical library. if the tagline doesn’t grab your attention, wait until you see it in action: drini showcasing library explorer at the library leaders forum get ready to explore in this article, we’ll give you a tour of the open library explorer and teach you how one may take full advantage of its features. you’ll also get a crash course on the + years of library history which led to its innovation and an opportunity to test-drive it for yourself. so let’s get started! what better way to set the stage than by taking a trip down memory lane to the last time you were able to visit your local public library. as you pass the front desk, a friendly librarian scribbles some numbers on a piece of paper which they hand to you and points you towards a relevant section. with the list of library call numbers in your hand as your compass, you eagerly make your way through waves of towering bookshelves. suddenly, you depart from reality and find yourself navigating through a sea of books, discovering treasures you didn’t even know existed. library photo courtesy of pixy.org/ / before you know it, one book gets stuffed under one arm, two more books go under your other arm, and a few more books get positioned securely between your knees. you’re doing the math to see how close you are to your check-out limit. remember those days? what if you could replicate that same library experience and access it every single day, from the convenience of your web browser? well, thanks to the new open library explorer, you can experience the joys of a physical library right in your web browser, as well as leverage superpowers which enable you to explore in ways which may have previously been impossible. before we dive into the bells-and-whistles of the library explorer, it’s worth learning how and why such innovations came to be. who needs library explorer? this year we’ve seen systems stressed to their max due to the covid- pandemic. with libraries and schools closing their doors globally and stay-at-home orders hampering our access, there has been a paradigm shift in the needs of researchers, educators, students, and families to access fundamental resources online. getting this information online is a challenge in and of itself. making it easy to discover and use materials online is another entirely. how does one faithfully compress the entire experience of a reliable, unbiased, expansive public library and its helpful, friendly staff into a ” computer screen? some sites, like netflix or youtube, solve this problem with recommendation engines that populate information based on what people have previously seen or searched. consequently, readers may unknowingly find themselves caught in a sort of “algorithmic bubble.” an algorithmic bubble (or “filter bubble”) is a state of intellectual or informational isolation that’s perpetuated by personalized content. algorithmic bubbles can make it difficult for users to access information beyond their own opinions—effectively isolating them in their own cultural or ideological silos. drini cami, the creator of library explorer, says that users’ caught inside these algorithmic bubbles “won’t be exposed to information that is completely foreign to [them]. there is no way to systematically and feasibly explore.” hence the reasoning behind the library explorer’s intelligence comes out of a need to discover information without the constraints of algorithmic bubbles. as readers are exposed to more information, the question becomes, how can readers fully explore swaths of new information and still enjoy the experience? let’s take a look at how the library explorer tackles that half of the problem. humanity’s knowledge brought to life earlier this year, open library added the ability to search materials by both dewey decimal classification and library of congress classification. these systems contain embedded within them over years of librarian experience, and provide a systematized approach to sort through the entirety of humanity’s knowledge embedded in books. it is important to note, the systematization of knowledge alone does not necessarily make it easily discoverable. this is what makes the library explorer so special. its digital interface opens the door for readers to seamlessly navigate centuries of books anywhere online. thanks to innovations such as the library explorer, readers can explore more books and access more knowledge with a better experience. a tour of library explorer’s features if you’re pulling up a chair for the first time, the library explorer presents you with tall, clickable bookshelves situated across your screen. each shelf has its own identity that can morph into new classes of books and subject categories with a single click. and that’s only the beginning of what it offers. in addition to those smart filters, the library explorer wants you to steer the ship… not the other way around. in other words, you can personalize single rows of books, expand entire shelves, or construct an entire library-experience that evolves around your exact interests. you can custom tailor your own personal library from the comfort of your device, wherever you may be. quick question: as a kid, did you ever layout your newly checked-out library books on your bed to admire them? well, the creators behind the library explorer found a way to mimic that same experience. if you so choose, you can zoom out of the library explorer interface to get a complete view of the library you’ve constructed. let’s explore one more set of cool features the library explorer offers by clicking on the “filter” icon at the bottom of the page. by selecting “juvenile,” you can instantly transform your entire library into a children’s library, but keep all the useful organization and structure provided by the bookshelves. it’s as if your own personal librarian ran in at lightning speed and removed every book from each shelf that didn’t meet your criteria. or you may type in “subject:biography” and suddenly your entire library shows you a tailored collection of just biographies on every subject. the sky is your limit. if you click on the settings tab, you’re given several options to customize the look and feel of your personal library explorer. you can switch between using library of congress or dewey decimal classification to organize your shelves. you can also choose from a variety of delightful options to see your books in d. each book has the correct thickness determined by its actual number of pages. to see your favorite book in d, click the settings icon at the bottom of the screen and then press the d button. library explorer’s d view maybe you’ve experienced a time where you had limited space in your book bag. perhaps because of that, you chose to wait on checking out heavier books. or, maybe you judged a book’s strength of knowledge based on its thickness. if that’s you, guess what? the open library explorer lets you do that. it gets personal… the primary goal of the library explorer was to create an experimental interface that ‘opens the door’ for readers to locate new books and engage with their favorite books. the library explorer is one of many steps that both the internet archive and the open library have made towards making knowledge easy to discover. as you know, such innovation couldn’t be possible without people who believe in the necessity of reading. here is a list of the names of those who contributed to the creation of the library explorer: drini cami, open library developer and library explorer creator mek karpeles, open library program lead jim shelton, ux designer, internet archive ziyad basheer, product designer tinnei pang, illustrator and product designer james hill-khurana, product designer nick norman, open library storyteller & volunteer communications lead well, this is the moment you’ve been waiting for. go here and give the library explorer a beta test-run. also, follow @openlibrary on twitter to learn about other features as soon as they’re released. but before you go… in the comments below, tell us your favorite library experience. we’d love to hear! posted in uncategorized | responses importing your goodreads & accessing them with open library’s apis by mek | published: december , by mek today joe alcorn, founder of readng, published an article (https://joealcorn.co.uk/blog/ /goodreads-retiring-api) sharing news with readers that amazon’s goodreads service is in the process of retiring their developer apis, with an effective start date of last tuesday, december th, . a screenshot taken from joe alcorn’s post the topic stirred discussion among developers and book lovers alike, making the front-page of the popular hacker news website. hacker news at - - : pm pacific. the importance of apis for those who are new to the term, an api is a method of accessing data in a way which is designed for computers to consume rather than people. apis often allow computers to subscribe to (i.e. listen for) events and then take actions. for example, let’s say you wanted to tweet every time your favorite author published a new book. one could sit on goodreads and refresh the website every fifteen minutes. or, one might write a twitter bot which automatically connects to goodreads and checks real-time data using its api. in fact, the reason why twitter bots work, is that they use twitter’s api, a mechanism which lets specially designed computer programs submit tweets to the platform. as one of the more popular book services online today, tens of thousands of readers and organizations rely on amazon’s goodreads apis to lookup information about books and to power their book-related applications across the web. some authors rely on the data to showcase their works on their personal homepages, online book stores to promote their inventory, innovative new services like thestorygraph are using this data to help readers discover new insights, and even librarians and scholastic websites rely on book data apis to make sure their catalog information is as up to date and accurate as possible for their patrons. for years, the open library team has been enthusiastic to share the book space with friends like goodreads who have historically shown great commitment by enabling patrons to control (download and export) their own data and enabling developers to create flourishing ecosystems which promote books and readership through their apis. when it comes to serving an audience of book lovers, there is no “one size fits all” and we’re glad so many different platforms and apis exist to provide experiences which meet the needs of different communities. and we’d like to do our part to keep the landscape flourishing. “the sad thing is it [retiring their apis] really only hurts the hobbyist projects and goodreads users themselves.” — joe alcorn picture of aaron swartz by noah berger/landov from thedailybeast at open library, our top priority is pursuing aaron swartz‘s original mission: to serve as an open book catalog for the public (one page for every book ever published) and ensure our community always has free, open data to unlock a world of possibilities. a world which believes in the power of reading to preserve our cultural heritage and empower education and understanding. we sincerely hope that amazon will decide it’s in goodreads’ best interests to re-instate their apis. but either way, open library is committed to helping readers, developers, and all book lovers have autonomy over their data and direct access to the data they rely on. one reason patrons appreciate open library is that it aligns with their values imports & exports in august , one of our google summer of code contributors tabish shaikh helped us implement an export option for open library reading logs to help everyone retain full control of their book data. we also created a goodreads import feature to help patrons who may want an easy way to check which goodreads titles may be available to borrow from the internet archive’s controlled digital lending program via openlibrary.org and to help patrons organize all their books in one place. we didn’t make a fuss about this feature at the time, because we knew patrons have a lot of options. but things can change quickly and we want patrons to be able to make that decision for themselves. for those who may not have known, amazon’s goodreads website provides an option for downloading/exporting a list of books from one’s bookshelves. you may find instructions on this goodreads export process here. open library’s goodreads importer enables patrons to take this exported dump of their goodreads bookshelves and automatically add matching titles to their open library reading logs. the goodreads import feature from https://openlibrary.org/account/import known issues. currently, open library’s goodreads importer only works for (a) titles that are in the open library catalog and (b) which are new enough to have isbns. our staff and community are committed to continuing to improve our catalog to include more titles (we added more than m titles this year) and we plan to improve our importer to support other id types like oclc and loc. apis & data developers and book overs who have been relying on amazon’s goodreads apis are not out of luck. there are several wonderful services, many of them open-source, including open library, which offer free apis: wikidata.org (by the same group who brought us wikipedia) is a treasure trove of metadata on authors and books. open library gratefully leverages this powerful resource to enrich our pages. inventaire.io is a wonderful service which uses wikidata and openlibrary data (api: api.inventaire.io) bookbrainz.org (by the group who runs musicbrainz) is a up-and-coming catalog of books worldcat by oclc offers various metadata apis did we miss any? please let us know! we’d love to work together, build stronger integrations with, and support other book-loving services. open library’s apis. and of course, open library has a free, open, book api which spans nearly million books. bulk data. if you need access to all our data, open library releases a free monthly bulk data dump of authors, books, and more. spoiler: everything on open library is an api! one of my favorite parts of open library is that practically every page is an api. all that is required is adding “.json” to the end. here are some examples: search https://openlibrary.org/search?q=lord+of+the+rings is our search page for humans… https://openlibrary.org/search.json?q=lord+of+the+rings is our search api! books https://openlibrary.org/books/ol m/harry_potter_and_the_methods_of_rationality is the human page for harry potter and the methods of rationality… https://openlibrary.org/books/ol m.json is its api! authors https://openlibrary.org/authors/ol a/rik_roots is a human readable author page… https://openlibrary.org/authors/ol a.json and here is the api! did we mention: full-text search over m books? major hat tip to the internet archive’s giovanni damiola for this one: folks may also appreciate the ability to full-text search across m of the internet archive’s books (https://blog.openlibrary.org/ / / /search-full-text-within- m-books) on open library: you can try it directly here: http://openlibrary.org/search/inside?q=thanks% for% all% the% fish as per usual, nearly all open library urls are themselves apis, e.g.: http://openlibrary.org/search/inside.json?q=thanks% for% all% the% fish get involved questions? open library is an free, open-source, nonprofit project run by the internet archive. we do our development transparently in public (here’s our code) and our community spanning more than volunteers meets every week, tuesday @ : am pacific. please contact us to join our call and participate in the process. bugs? if something isn’t working as expected, please let us know by opening an issue or joining our weekly community calls. want to share thanks? please follow up on twitter: https://twitter.com/openlibrary and let us know how you’re using our apis! thank you a special thank you to our lead developers drini cami, chris clauss, and one of our lead volunteer engineers, aaron, for spending their weekend helping fix a python bug which was temporarily preventing goodreads imports from succeeding. a decentralized future the internet archive has a history cultivating and supporting the decentralized web. we operate a decentralized version of archive.org and host regular meetups and summits to galvanize the distributed web community. in the future, we can imagine a world where no single website controls all of your data, but rather patrons can participate in a decentralized, distributed network. you may be interested to try bookwyrm, an open-source decentralized project by mouse, former engineer on the internet archive’s archive-it team. posted in uncategorized | responses on bookstores, libraries & archives in the digital age by brewster kahle | published: october , the following was a guest post by brewster kahle on against the grain (atg) – linking publishers, vendors, & librarians on bookstores, libraries & archives in the digital age-an atg guest post see the original article here on atg’s website by: brewster kahle, founder & digital librarian, internet archive back in , i was honored to give a keynote at the meeting of the society of american archivists, when the president of the society presented me with a framed blown-up letter “s.” this was an inside joke about the internet archive being named in the singular, archive, rather than the plural archives. of course, he was right, as i should have known all along. the internet archive had long since grown out of being an “archive of the internet”—a singular collection, say of web pages—to being “archives on the internet,” plural. my evolving understanding of these different names might help focus a discussion that has become blurry in our digital times: the difference between the roles of publishers, bookstores, libraries, archives, and museums. these organizations and institutions have evolved with different success criteria, not just because of the shifting physical manifestation of knowledge over time, but because of the different roles each group plays in a functioning society. for the moment, let’s take the concepts of library and archive. the traditional definition of a library is that it is made up of published materials, while an archive is made up of unpublished materials. archives play an important function that must be maintained—we give frightfully little attention to collections of unpublished works in the digital age. think of all the drafts of books that have disappeared once we started to write with word processors and kept the files on fragile computer floppies and disks. think of all the videotapes of lectures that are thrown out or were never recorded in the first place. bookstores: the thrill of the hunt let’s try another approach to understanding distinctions between bookstores, libraries and archives. when i was in my ’s living in boston—before amazon.com and before the world wide web (but during the early internet)—new and used bookstores were everywhere. i thought of them as catering to the specialized interests of their customers: small, selective, and only offering books that might sell and be taken away, with enough profit margin to keep the store in business. i loved them. i especially liked the used bookstore owners—they could peer into my soul (and into my wallet!) to find the right book for me. the most enjoyable aspect of the bookstore was the hunt—i arrived with a tiny sheet of paper in my wallet with a list of the books i wanted, would bring it out and ask the used bookstore owners if i might go home with a bargain. i rarely had the money to buy new books for myself, but i would give new books as gifts. while i knew it was okay to stay for awhile in the bookstore just reading, i always knew the game. libraries: offering conversations not answers the libraries that i used in boston—mit libraries, harvard libraries, the boston public library—were very different. i knew of the private boston athenæum but i was not a member, so i could not enter. libraries for me seemed infinite, but still tailored to individual interests. they had what was needed for you to explore and if they did not have it, the reference librarian would proudly proclaim: “we can get it for you!” i loved interlibrary loans—not so much in practice, because it was slow, but because they gave you a glimpse of a network of institutions sharing what they treasured with anyone curious enough to want to know more. it was a dream straight out of borges’ imagination (if you have not read borges’ short stories, they are not to be missed, and they are short. i recommend you write them on the little slip of paper you keep in your wallet.) i couldn’t afford to own many of the books i wanted, so it turned off that acquisitive impulse in me. but the libraries allowed me to read anything, old and new. i found i consumed library books very differently. i rarely even brought a book from the shelf to a table; i would stand, browse, read, learn and search in the aisles. dipping in here and there. the card catalog got me to the right section and from there i learned as i explored. libraries were there to spark my own ideas. the library did not set out to tell a story as a museum would. it was for me to find stories, to create connections, have my own ideas by putting things together. i would come to the library with a question and end up with ideas. rarely were these facts or statistics—but rather new points of view. old books, historical newspapers, even the collection of reference books all illustrated points of view that were important to the times and subject matter. i was able to learn from others who may have been far away or long deceased. libraries presented me with a conversation, not an answer. good libraries cause conversations in your head with many writers. these writers, those librarians, challenged me to be different, to be better. staying for hours in a library was not an annoyance for the librarians—it was the point. yes, you could check books out of the library, and i would, but mostly i did my work in the library—a few pages here, a few pages there—a stack of books in a carrel with index cards tucked into them and with lots of handwritten notes (uh, no laptops yet). but libraries were still specialized. to learn about draft resisters during the vietnam war, i needed access to a law library. mit did not have a law collection and this was before lexis/nexis and westlaw. i needed to get to the volumes of case law of the united states. harvard, up the road, had one of the great law libraries, but as an mit student, i could not get in. my mit professor lent me his id that fortunately did not include a photo, so i could sneak in with that. i spent hours in the basement of harvard’s law library reading about the cases of conscientious objectors and others. but why was this library of law books not available to everyone? it stung me. it did not seem right. a few years later i would apply to library school at simmons college to figure out how to build a digital library system that would be closer to the carved words over the boston public library’s door in copley square: “free to all.” archives: a wonderful place for singular obsessions when i quizzed the archivist at mit, she explained what she did and how the mit archives worked. i loved the idea, but did not spend any time there—it was not organized for the busy undergraduate. the mit library was organized for easy access; the mit archives included complete collections of papers, notes, ephemera from others, often professors. it struck me that the archives were collections of collections. each collection faithfully preserved and annotated. i think of them as having advertisements on them, beckoning the researcher who wants to dive into the materials in the archive and the mindset of the collector. so in this formulation, an archive is a collection, archives are collections of collections. archivists are presented with collections, usually donations, but sometimes there is some money involved to preserve and catalog another’s life work. personally, i appreciate almost any evidence of obsession—it can drive toward singular accomplishments. archives often reveal such singular obsessions. but not all collections are archived, as it is an expensive process. the cost of archiving collections is changing, especially with digital materials, as is cataloging and searching those collections. but it is still expensive. when the internet archive takes on a physical collection, say of records, or old repair manuals, or materials from an art group, we have to weigh the costs and the potential benefits to researchers in the future. archives take the long view. one hundred years from now is not an endpoint, it may be the first time a collection really comes back to light. digital libraries: a memex dream, a global brain so when i helped start the internet archive, we wanted to build a digital library—a “complete enough” collection, and “organized enough” that everything would be there and findable. a universal library. a library of alexandria for the digital age. fulfilling the memex dream of vanevar bush (do read “as we may think“), of ted nelson‘s xanadu, of tim berners-lee‘s world wide web, of danny hillis‘ thinking machine, raj reddy’s universal access to all knowledge, and peter russell’s global brain. could we be smarter by having people, the library, networks, and computers all work together? that is the dream i signed on to. i dreamed of starting with a collection—an archive, an internet archive. this grew to be a collection of collections: archives. then a critical mass of knowledge complete enough to inform citizens worldwide: a digital library. a library accessible by anyone connected to the internet, “free to all.” about the author: brewster kahle, founder & digital librarian, internet archive brewster kahle a passionate advocate for public internet access and a successful entrepreneur, brewster kahle has spent his career intent on a singular focus: providing universal access to all knowledge. he is the founder and digital librarian of the internet archive, one of the largest digital libraries in the world, which serves more than a million patrons each day. creator of the wayback machine and lending millions of digitized books, the internet archive works with more than library and university partners to create a free digital library, accessible to all. soon after graduating from the massachusetts institute of technology where he studied artificial intelligence, kahle helped found the company thinking machines, a parallel supercomputer maker. he is an internet pioneer, creating the internet’s first publishing system called wide area information server (wais). in , kahle co-founded alexa internet, with technology that helps catalog the web, selling it to amazon.com in . elected to the internet hall of fame, kahle is also a fellow of the american academy of arts and sciences, a member of the national academy of engineering, and holds honorary library doctorates from simmons college and university of alberta. posted in discussion, librarianship, uncategorized | comments closed amplifying the voices behind books by mek | published: september , exploring how open library uses author data to help readers move from imagination to impact by nick norman, edited by mek & drini image source: pexels / pixabay from popsugar according to rené descartes, a creative mathematician, “the reading of all good books is like a conversation with the finest [people] of past centuries.” if that’s true, then who are some of the people you’re talking to? if you’re not sure how to answer that question, you’ll definitely appreciate the ‘author stats’ feature developed by open library. a deep dive into author stats author stats give readers clear insights about their favorite authors that go much deeper than the front cover: such as birthplace, gender, works by time, ethnicity, and country of citizenship. these bits and pieces of knowledge about authors can empower readers in some dynamic ways. but how exactly? to answer that question, consider a reader who’s passionate about the topic of cultural diversity. however, after the reader examines their personalized author stats, they realize that their reading history lacks diversity. this doesn’t mean the reader isn’t passionate about cultural diversity; rather, author stats empowers the reader to pinpoint specific stats that can be diversified. take a moment … or a day, and think about all the books you’ve read — just in the last year or as far back as you can. what if you could align the pages of each of those books with something meaningful … something that matters? what if each time you cracked open a book, the voices inside could point you to places filled with hope and opportunity? according to drini cami — open library’s lead developer behind author stats , “these stats let readers determine where the voices they read are coming from.” drini continues saying, “a book can be both like a conversation as well as a journey.” he also says, “statistics related to the authors might help provide readers with feedback as to where the voices they are listening to are coming from, and hopefully encourage the reading of books from a wider variety of perspectives.” take a moment to let that sink in. data with the power to change while open library’s author stats can show author-related demographics, those same stats can do a lot more than that. drini cami went on to say that, “author stats can help readers intelligently alter their behavior (if they wish to).” a profound statement that mark twain — one of the best writers in american history — might even shout from the rooftop. broad, wholesome, charitable views of [people] … cannot be acquired by vegetating in one little corner of the earth all one’s lifetime. — mark twain in the eyes of drini cami and mark twain, books are like miniature time machines that have the power to launch readers into new spaces while changing their behaviors at the same time. for it is only when a reader steps out of their corner of the earth that they can step forward towards becoming a better person — for the entire world. connecting two worlds of data open library has gone far beyond the extra mile to provide data about author demographics that some readers may not realize. it started with open library’s commitment to providing its readers with what drini cami describes as “clean, organized, structured, queryable data.” simply put, readers can trust that open library’s data can be used to provide its audiences with maximum value. which begs the question, where is all that ‘value’ coming from? drini cami calls it “linked data”. in not so complex terms, you may think of linked data as being two or more storage sheds packed with data. when these storage sheds are connected, well… that’s when the magic happens. for open library, that magic starts at the link between wikidata and open library knowledge bases. wikidata, a non-profit community-powered project run by wikimedia, the same team which brought us wikipedia, is a “free and open knowledge base that can be read and edited by both humans and machines”. it’s like wikipedia except for storing bite-sized encyclopedic data and facts instead of articles. if you look closely, you may even find some of wikidata’s data being leveraged within wikipedia articles. wikipedia’s summary info box source data in wikidata wikidata is where open library gets its author demographic data from. this is possible because the entries on wikidata often include links to source material such as books, authors, learning materials, e-journals, and even to other knowledge bases like open library’s. because of these links, open library is able to share its data with wikidata and often times get back detailed information and structured data in return. such as author demographics. wrangling in the data linking-up services like wikidata and open library doesn’t happen automatically. it requires the hard work of “metadata wranglers”. that’s where charles horn comes in, the lead data engineer at open library — without his work, author stats would not be possible. charles horn works closely with drini cami and also the team at wikidata to connect book and author resources on open library with the data kept inside wikidata. by writing clever bots and scripts, charles and drini are able to make tens of thousands of connections at scale. to put it simply, as both open library and wikidata grow, their resources and data will become better connected and more accurate. thanks to the help of “metadata wranglers”, open library users will always have the smartest results — right at their fingertips. it’s in a book … once upon a time, ten-time grammy award winner chaka kahn greeted television viewers with her bright voice on the once-popular book reading program, reading rainbow. in her words, she sang … “friends to know, and ways to grow, a reading rainbow. i can be anything. take a look, it’s in a book …” thanks to open library’s author stats, not only do readers have the power to “take a look” into books, they can see further, and truly change what they see. try browsing your author stats and consider following open library on twitter. the “my reading stats” option may be found under the “my books” drop down menu within the main site’s top navigation. what did you learn about your favorite authors? please share in the comments below. posted in community, cultural resources, data | comments closed giacomo cignoni: my internship at the internet archive by drini cami | published: august , this summer, open library and the internet archive took part in google summer of code (gsoc), a google initiative to help students gain coding experience by contributing to open source projects. i was lucky enough to mentor giacomo while he worked on improving our bookreader experience and infrastructure. we have invited giacomo to write a blog post to share some of the wonderful work he has done and his learnings. it was a pleasure working with you giacomo, and we all wish you the best of luck with the rest of your studies! – drini hi, i am giacomo cignoni, a nd year computer science student from italy. i submitted my google summer of code (gsoc) project to work with the internet archive and i was selected for it. in this blogpost, i want to tell you about my experience and my accomplishments working this summer on bookreader, internet archive’s open source book reading web application. the bookreader features i enjoyed the most working on are page filters (which includes “dark mode”) and the text selection layer for certain public domain books. they were both challenging, but mostly had a great impact on the user experience of bookreader. the first allows text to be selected and copied directly from the page images (currently in internal testing), and the second permits turning white-background black-text pages into black-background-white-text ones. short summary of implemented features: end-to-end testing (search, autoplay, right-to-left books) generic book from internet archive demo mobile bookreader table of contents checkbox for filters on book pages (including dark mode) text selection layer plugin for public domain books bug fixes for page flipping using high resolution book images bug fix first approach to gsoc experience once i received the news that i had been selected for gsoc with internet archive for my bookreader project, i was really excited, as it was the beginning of a new experience for me. for the same reason, i will not hide that i was a little bit nervous because it was my first internship-like experience. fortunately, even from the start, my mentor drini and also mek were supportive and also ready to offer help. moreover, the fact that i was already familiar with bookreader was helpful, as i had already used it (and even modified it a little bit) for a personal project. for most of the month of may, since the th, the day of the gsoc selection, i mainly focused on getting to know the other members of the ux team at internet archive, whom i would be working with for the rest of the summer, and also define a more precise roadmap of my future work with my mentor, as my proposed project was open to any improvements for bookreader. end to end testing the first tasks i worked on, as stated in the project, were about end-to-end testing for bookreader. i learned about the testcafe tool that was to be used, and my first real task was to remove and explore some old qunit tests (# ). then i started to make end-to-end tests for the search feature in bookreader, both for desktop (# ) and mobile (# ). lastly, i fixed the existent autoplay end-to-end test (# ) that was causing problems and i also had prepared end-to-end tests for right-to-left books (# ), but it wasn’t merged immediately because it needed a feature that i would have implemented later; a system to choose different books from the ia servers to be displayed specifying the book id in the url. this work on testing (which lasted until the ~ th of june) was really helpful at the beginning as it allowed me to gain more confidence with the codebase without trying immediately harder tasks and also to gain more confidence with javascript es . the frequent meetings with my mentor and other members of the team made me really feel part of the workplace. working on the source code the table of contents panel in bookreader mobile my first experience working on core bookreader source code was during the internet archive hackathon on may the th when, with the help of my mentor, i created the first draft for the table of content panel for mobile bookreader. i would then resume to work on this feature in july, refining it until it was released (# ). i then worked on a checkbox to apply different filters to the book page images, still on mobile bookreader (# ), which includes a sort of “dark mode”. this feature was probably the one i enjoyed the most working on, as it was challenging but not too difficult, it included some planning and was not purely technical and received great appreciation from users. page filters for bookreader mobile let you read in a “dark mode” https://twitter.com/openlibrary/status/ then i worked on the generic demo feature; a particular demo for bookreader which allows you to choose a book from the internet archive servers to be displayed, by simply adding the book id in the url as a parameter (# ). this allowed the right to left e e test to be merged and proved to be useful for manually testing the text selection plugin. in this period i also fixed two page flipping issues: one more critical (when flipping pages in quick succession the pages started turning back and forth randomly) (# ), and the other one less urgent, but it was an issue a user specifically pointed out (in an old bookreader demo it was impossible to turn pages at all) (# ). another issue i solved was bookreader not correctly displaying high resolution images on high resolution displays (# ). open source project experience one aspect i really enjoyed of my gsoc is the all-around experience of working on an open source project. this includes leaving more approachable tasks for the occasional member of the community to take on and helping them out. also, i found it interesting working with other members of the team aside from my mentor, both for more technical reasons and for help in ui designing and feedback about the user experience: i always liked having more points of view about my work. moreover, direct user feedback from the users, which showed appreciation for the new implemented features (such as bookreader “dark mode”), was very motivating and pushed me to do better in the following tasks. text selection layer the normally invisible text layer shown red here for debugging the biggest feature of my gsoc was implementing the ability to select text directly on the page image from bookreader for public domain books, in order to copy and paste it elsewhere (# ). this was made possible because internet archive books have information about each word and its placement in the page, which is collected by doing ocr. to implement this feature we decided to use an invisible text layer placed on top of the page image, with words being correctly positioned and scaled. this made it possible to use the browser’s text selection system instead of creating a new one. the text layer on top of the page was implemented using an svg element, with subelements for each paragraph and word in the page. the use of the svg instead of normal html text elements made it a lot easier to overcome most of the problems we expected to find regarding the correct placement and scaling of words in the layer. i started working sporadically on this feature since the start of july and this led to having a workable demo by the first day of august. the rest of the month of august was spent refining this feature to make it production-ready. this included refining word placement in the layer, adding unit tests, adding support for more browsers, refactoring some functions, making the experience more fluid, making the selected text to be accurate for newlines and spaces on copy. the most challenging part was probably to integrate well the text selection actions in the two page view of bookreader, without disrupting the click-to-flip-page and other functionalities related to mouse-click events. this feature is currently in internal testing, and scheduled for release in the next few weeks. the text selection experience conclusions overall, i was extremely satisfied with my gsoc at the internet archive. it was a great opportunity to learn new things for me. i got much more fluent in javascript and css, thanks to both my mentor and using these languages in practice while coding. i learnt a lot about working on an open source project, but a part that i probably found really interesting was attending and participating in the decision making processes, even about projects i was not involved in. it was also interesting for me to apply concepts i had studied on a more theoretical level at university in a real workplace environment. to sum things up, the ability to work on something i liked that had an impact on users and the ability to learn useful things for my personal development really made this experience worthwhile for me. i would % recommend doing a gsoc at the internet archive! posted in bookreader, community, google summer of code (gsoc), open source | comments closed open library is an initiative of the internet archive, a (c)( ) non-profit, building a digital library of internet sites and other cultural artifacts in digital form. other projects include the wayback machine, archive.org and archive-it.org. your use of the open library is subject to the internet archive's terms of use. « older posts search recent posts introducing the open library explorer importing your goodreads & accessing them with open library’s apis on bookstores, libraries & archives in the digital age amplifying the voices behind books giacomo cignoni: my internship at the internet archive archives archives select month december october september august july may november october january october august july june may march december october june may february january november february january december november october august july june may april march april january august december november october july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november theme customized from thematic theme framework. the carpentries the carpentries nav donate search contact home about about us our values code of conduct governance supporters testimonials annual reports equity, inclusion, and accessibility teach what is a workshop? data carpentry lessons software carpentry lessons library carpentry lessons community lessons become an instructor for instructors online workshop recommendations learn our workshops our curricula upcoming workshops past workshops our impact join us get involved help develop lessons become a member organisation job vacancies our community our team core team projects community overview our instructors our maintainers our mentors our regional coordinators our trainers committees and task forces current member organisations connect blog community calendar community discussions community handbook newsletter twitter we teach foundational coding and data science skills to researchers worldwide. what we do the carpentries teaches foundational coding, and data science skills to researchers worldwide. software carpentry, data carpentry, and library carpentry workshops are based on our lessons. workshop hosts, instructors, and learners must be prepared to follow our code of conduct. more › who we are our diverse, global community includes instructors, helpers, trainers, maintainers, mentors, community champions, member organisations, supporters, workshop organisers, staff and a whole lot more. more › get involved see all the ways you can engage with the carpentries. get information about upcoming events such as workshops, meetups, and discussions from our community calendar, or from our twice-monthly newsletter, carpentry clippings. follow us on twitter, facebook, and slack. more › subscribe to our newsletter "carpentry clippings" events, community updates, teaching tips, in your inbox, twice a month new blog posts introducing our new workshop administrator danielle sieh danielle sieh joins the carpentries as workshop administrator read more › more posts improving accessibility in the carpentries help inform translation priorities for carpentries resources the carpentries at rstudio::global( ) more › resources for online workshops official carpentries' recommendations this page holds an official set of recommendations by the carpentries to help you organise and run online carpentries workshops. the page is updated periodically as we continue to receive input and feedback from our community. go to page. community-created resources this resource is a section in our handbook containing an evolving list of all community-created resources and conversations around teaching carpentries workshops online. the section is updated periodically to include newer resources and emerging conversations on the subject. go to page. upcoming carpentries workshops click on an individual event to learn more about that event, including contact information and registration instructions. ucla ischool instructors: geno sanchez, doug daniels, tim dennis, elizabeth mcaulay, jamie jamison helpers: geno sanchez, doug daniels, tim dennis, elizabeth mcaulay, jamie jamison jan - feb , university of alabama at birmingham (online) instructors: annajiat alim rasel, cody hennesy, camilla bressan, mary ann warner jan - apr , west virginia university instructors: asher hudson, vivian guetler feb - feb , carpentries @ mit: intro to unixshell/python/git instructors: ye li, ece turnator, madeline wrable, daniel sheehan, christine malinowski helpers: phoebe ayers, ye li, ece turnator, madeline wrable, daniel sheehan, christine malinowski feb - feb , george mason university (online) instructors: annajiat alim rasel feb - feb , newcastle university instructors: jannetta steyn feb - feb , university of puerto rico instructors: humberto ortiz-zuazaga, sofia melendez-cartagena, diego rosado-tristani feb - feb , macquarie university ** instructors: richard miller, belinda fabian helpers: tim keighley feb - feb , kaust visualization core lab instructors: david pugh feb - feb , discord instructors: tim ribaric, daniel brett, thomas guignard helpers: john fink feb - feb , max delbrück center for molecular medicine (online) instructors: monah abou alezz, aleksander jankowski, liz ing-simmons, tobias busch feb - feb , university of california santa barbara (online) instructors: elnaz amanzadeh jajin, rohit goswami feb - feb , ohio state, university libraries and translational data analytics institute (online) instructors: annajiat alim rasel, scott peterson, jennifer lee feb - feb , maze therapeutics (online) instructors: sue mcclatchy, aditya bandla, akshay paropkari, rachel lombardi feb - feb , nwu instructors: sebastian mosidi, martin dreyer feb - feb , noaa/nos (online) instructors: mike trizna, ian carroll, camilla bressan feb - feb , kaust visualization core lab ** instructors: david pugh feb - feb , kaust visualization core lab ** instructors: david pugh helpers: glendon holst mar - mar , united states department of agriculture (usda) (online) instructors: annajiat alim rasel, manisha munasinghe, sichong peng mar - mar , mid-atlantic chapter of the medical library association (online) instructors: saranya canchi, trisha adamus, jennifer lee mar - mar , institute for modeling collaboration and innovation @ the university of idaho (online) instructors: james van leuven, breanna sipley helpers: amanda stahlke, travis seaborn mar - mar , george washington university (online) instructors: alain wescott, monah abou alezz, preethy nair mar - mar , arizona state university (online) instructors: ryan peek, annajiat alim rasel, lisa rosenthal mar - mar , west virginia university (online) instructors: sally chang, aditya bandla, elnaz amanzadeh jajin mar - mar , kaust visualization core lab ** instructors: david pugh mar - mar , kaust visualization core lab ** instructors: david pugh mar - mar , uppsala university, scilifelab, bmc, ** instructors: niclas jareborg, elin kronander, wolmar nyberg Åkerström apr - apr , institute for modeling collaboration and innovation @ the university of idaho (online) instructors: erich seamon helpers: travis seaborn apr - apr , ** workshops marked with asterisks are based on curriculum from the carpentries lesson programs but may not follow our standard workshop format. workshops with a globe icon are being held online. the corresponding flag notes the country where the host organization is based. click here to see our past workshops.  about the carpentries the carpentries is a fiscally sponsored project of community initiatives, a registered (c) non-profit organisation based in california, usa. we are a global community teaching foundational computational and data science skills to researchers in academia, industry and government. more › services contact rss atom sitemap.xml links our code of conduct our community handbook our privacy policy our annual reports software carpentry website data carpentry website library carpentry website the carpentries the carpentries nav donate search contact home about about us our values code of conduct governance supporters testimonials annual reports equity, inclusion, and accessibility teach what is a workshop? data carpentry lessons software carpentry lessons library carpentry lessons community lessons become an instructor for instructors online workshop recommendations learn our workshops our curricula upcoming workshops past workshops our impact join us get involved help develop lessons become a member organisation job vacancies our community our team core team projects community overview our instructors our maintainers our mentors our regional coordinators our trainers committees and task forces current member organisations connect blog community calendar community discussions community handbook newsletter twitter we teach foundational coding and data science skills to researchers worldwide. what we do the carpentries teaches foundational coding, and data science skills to researchers worldwide. software carpentry, data carpentry, and library carpentry workshops are based on our lessons. workshop hosts, instructors, and learners must be prepared to follow our code of conduct. more › who we are our diverse, global community includes instructors, helpers, trainers, maintainers, mentors, community champions, member organisations, supporters, workshop organisers, staff and a whole lot more. more › get involved see all the ways you can engage with the carpentries. get information about upcoming events such as workshops, meetups, and discussions from our community calendar, or from our twice-monthly newsletter, carpentry clippings. follow us on twitter, facebook, and slack. more › subscribe to our newsletter "carpentry clippings" events, community updates, teaching tips, in your inbox, twice a month new blog posts introducing our new workshop administrator danielle sieh danielle sieh joins the carpentries as workshop administrator read more › more posts improving accessibility in the carpentries help inform translation priorities for carpentries resources the carpentries at rstudio::global( ) more › resources for online workshops official carpentries' recommendations this page holds an official set of recommendations by the carpentries to help you organise and run online carpentries workshops. the page is updated periodically as we continue to receive input and feedback from our community. go to page. community-created resources this resource is a section in our handbook containing an evolving list of all community-created resources and conversations around teaching carpentries workshops online. the section is updated periodically to include newer resources and emerging conversations on the subject. go to page. upcoming carpentries workshops click on an individual event to learn more about that event, including contact information and registration instructions. ucla ischool instructors: geno sanchez, doug daniels, tim dennis, elizabeth mcaulay, jamie jamison helpers: geno sanchez, doug daniels, tim dennis, elizabeth mcaulay, jamie jamison jan - feb , university of alabama at birmingham (online) instructors: annajiat alim rasel, cody hennesy, camilla bressan, mary ann warner jan - apr , west virginia university instructors: asher hudson, vivian guetler feb - feb , carpentries @ mit: intro to unixshell/python/git instructors: ye li, ece turnator, madeline wrable, daniel sheehan, christine malinowski helpers: phoebe ayers, ye li, ece turnator, madeline wrable, daniel sheehan, christine malinowski feb - feb , george mason university (online) instructors: annajiat alim rasel feb - feb , newcastle university instructors: jannetta steyn feb - feb , university of puerto rico instructors: humberto ortiz-zuazaga, sofia melendez-cartagena, diego rosado-tristani feb - feb , macquarie university ** instructors: richard miller, belinda fabian helpers: tim keighley feb - feb , kaust visualization core lab instructors: david pugh feb - feb , discord instructors: tim ribaric, daniel brett, thomas guignard helpers: john fink feb - feb , max delbrück center for molecular medicine (online) instructors: monah abou alezz, aleksander jankowski, liz ing-simmons, tobias busch feb - feb , university of california santa barbara (online) instructors: elnaz amanzadeh jajin, rohit goswami feb - feb , ohio state, university libraries and translational data analytics institute (online) instructors: annajiat alim rasel, scott peterson, jennifer lee feb - feb , maze therapeutics (online) instructors: sue mcclatchy, aditya bandla, akshay paropkari, rachel lombardi feb - feb , nwu instructors: sebastian mosidi, martin dreyer feb - feb , noaa/nos (online) instructors: mike trizna, ian carroll, camilla bressan feb - feb , kaust visualization core lab ** instructors: david pugh feb - feb , kaust visualization core lab ** instructors: david pugh helpers: glendon holst mar - mar , united states department of agriculture (usda) (online) instructors: annajiat alim rasel, manisha munasinghe, sichong peng mar - mar , mid-atlantic chapter of the medical library association (online) instructors: saranya canchi, trisha adamus, jennifer lee mar - mar , institute for modeling collaboration and innovation @ the university of idaho (online) instructors: james van leuven, breanna sipley helpers: amanda stahlke, travis seaborn mar - mar , george washington university (online) instructors: alain wescott, monah abou alezz, preethy nair mar - mar , arizona state university (online) instructors: ryan peek, annajiat alim rasel, lisa rosenthal mar - mar , west virginia university (online) instructors: sally chang, aditya bandla, elnaz amanzadeh jajin mar - mar , kaust visualization core lab ** instructors: david pugh mar - mar , kaust visualization core lab ** instructors: david pugh mar - mar , uppsala university, scilifelab, bmc, ** instructors: niclas jareborg, elin kronander, wolmar nyberg Åkerström apr - apr , institute for modeling collaboration and innovation @ the university of idaho (online) instructors: erich seamon helpers: travis seaborn apr - apr , ** workshops marked with asterisks are based on curriculum from the carpentries lesson programs but may not follow our standard workshop format. workshops with a globe icon are being held online. the corresponding flag notes the country where the host organization is based. click here to see our past workshops.  about the carpentries the carpentries is a fiscally sponsored project of community initiatives, a registered (c) non-profit organisation based in california, usa. we are a global community teaching foundational computational and data science skills to researchers in academia, industry and government. more › services contact rss atom sitemap.xml links our code of conduct our community handbook our privacy policy our annual reports software carpentry website data carpentry website library carpentry website open source exile open source exile an open sourcer in exile #christchurchmosqueshootings how would we know when it was time to move from tei/xml to tei/json? whither tei? the next thirty years thoughts on the ndfnz wikipedia panel feedback on nlnz ‘digitalnz concepts api‘ bibframe a wikipedia strategy for the royal society of new zealand prep notes for ndf demonstration metadata vocabularies lodlam nz cares about unexpected advice goodbye 'social-media' world recreational authority control thoughts on "letter about the tei" from martin mueller unit testing framework for xsl transformations? is there a place for readers' collectives in the bright new world of ebooks? howto: deep linking into the nzetc site epubs and quality what librarything metadata can the nzetc reasonable stuff inside it's cc'd epubs? interlinking of collections: the quest continues ebook readers need openurl resolvers thoughts on koha data and data modelling and underlying assumptions learning xslt . part ; finding names legal māori archive why card-based records aren't good enough bootstrappable builds benefits best practices projects contact do you know how to make yoghurt? the first step is to add yoghurt to milk! how can you build a compiler like gcc? the first step is to get a compiler that can compile the compiler. compilers are often written in the language they are compiling. this creates a chicken-and-egg problem that leads users and distributors to rely on opaque, pre-built binaries of those compilers that they use to build newer versions of the compiler. to gain trust in our computing platforms, we need to be able to tell how each part was produced from source. we believe that opaque binaries are a threat to user security and user freedom since they are not auditable; our goal is to minimize the amount of bootstrap binaries. benefits this is nice, but what are the actual benefits of “bootstrappable” implementations? find out what additional benefits there are to achieving bootstrappable builds. best practices are you developing or contributing to software that is affected by the bootstrapping problem? here we list best practices and practical examples that can help you pull yourself up by your own bootstraps. collaboration projects solving bootstrapping problems in existing compilers and build systems requires collaboration. here is a list of long-term high-impact projects that we would like to work on collaboratively. more projects and status updates can be found on the bootstrapping wiki. join the mailing list and/or the irc channel #bootstrappable on freenode for news and communication! further reading ken thompson's acceptance speech for the turing award: reflections on trusting trust toy example of a subverted rust compiler what is a coder's worst nightmare? defending against compiler-based backdoors deniable backdoors using compiler bugs made with ♥ by humans and powered by gnu guile. source code under the gnu agpl. net zero challenge - application form net zero challenge - application form this is the application for for the net zero challenge. www.netzerochallenge.info the deadline for applications is friday th march , pm pacific standard time (san francisco time) this application will then be assessed by our panel of experts. next never submit passwords through google forms. this form was created inside of open knowledge foundation. report abuse forms certificate authority - wikipedia certificate authority from wikipedia, the free encyclopedia jump to navigation jump to search an entity that issues digital certificates in cryptography, a certificate authority or certification authority (ca) is an entity that issues digital certificates. a digital certificate certifies the ownership of a public key by the named subject of the certificate. this allows others (relying parties) to rely upon signatures or on assertions made about the private key that corresponds to the certified public key. a ca acts as a trusted third party—trusted both by the subject (owner) of the certificate and by the party relying upon the certificate. the format of these certificates is specified by the x. or emv standard. one particularly common use for certificate authorities is to sign certificates used in https, the secure browsing protocol for the world wide web. another common use is in issuing identity cards by national governments for use in electronically signing documents.[ ] contents overview providers validation standards validation weaknesses issuing a certificate . example . security . authority revocation lists industry organizations . baseline requirements ca compromise key storage implementation weakness of the trusted third party scheme see also references external links overview[edit] trusted certificates can be used to create secure connections to a server via the internet. a certificate is essential in order to circumvent a malicious party which happens to be on the route to a target server which acts as if it were the target. such a scenario is commonly referred to as a man-in-the-middle attack. the client uses the ca certificate to authenticate the ca signature on the server certificate, as part of the authorizations before launching a secure connection. usually, client software—for example, browsers—include a set of trusted ca certificates. this makes sense, as many users need to trust their client software. a malicious or compromised client can skip any security check and still fool its users into believing otherwise. the clients of a ca are server supervisors who call for a certificate that their servers will bestow to users. commercial cas charge money to issue certificates, and their customers anticipate the ca's certificate to be contained within the majority of web browsers, so that safe connections to the certified servers work efficiently out-of-the-box. the quantity of internet browsers, other devices and applications which trust a particular certificate authority is referred to as ubiquity. mozilla, which is a non-profit business, issues several commercial ca certificates with its products.[ ] while mozilla developed their own policy, the ca/browser forum developed similar guidelines for ca trust and so on. a single ca certificate may be shared among multiple cas or their resellers. a root ca certificate may be the base to issue multiple intermediate ca certificates with varying validation requirements. in addition to commercial cas, some non-profits issue digital certificates to the public without charge; notable examples are cacert and let's encrypt. large organizations or government bodies may have their own pkis (public key infrastructure), each containing their own cas. any site using self-signed certificates acts as its own ca. commercial banks that issue emv payment cards are governed by the emv certificate authority,[ ] payment schemes that route payment transactions initiated at point of sale terminals (pos) to a card issuing bank to transfer the funds from the card holder's bank account to the payment recipient’s bank account. each payment card presents along with its card data also the card issuer certificate to the pos. the issuer certificate is signed by emv ca certificate. the pos retrieves the public key of emv ca from its storage, validates the issuer certificate and authenticity of the payment card before sending the payment request to the payment scheme. browsers and other clients of sorts characteristically allow users to add or do away with ca certificates at will. while server certificates regularly last for a relatively short period, ca certificates are further extended,[ ] so, for repeatedly visited servers, it is less error-prone importing and trusting the ca issued, rather than confirm a security exemption each time the server's certificate is renewed. less often, trustworthy certificates are used for encrypting or signing messages. cas dispense end-user certificates too, which can be used with s/mime. however, encryption entails the receiver's public key and, since authors and receivers of encrypted messages, apparently, know one another, the usefulness of a trusted third party remains confined to the signature verification of messages sent to public mailing lists. providers[edit] worldwide, the certificate authority business is fragmented, with national or regional providers dominating their home market. this is because many uses of digital certificates, such as for legally binding digital signatures, are linked to local law, regulations, and accreditation schemes for certificate authorities. however, the market for globally trusted tls/ssl server certificates is largely held by a small number of multinational companies. this market has significant barriers to entry due to the technical requirements.[ ] while not legally required, new providers may choose to undergo annual security audits (such as webtrust[ ] for certificate authorities in north america and etsi in europe[ ]) to be included as a trusted root by a web browser or operating system. as of august [update], root certificates, representing organizations, are trusted in the mozilla firefox web browser,[ ] root certificates, representing organizations, are trusted by macos,[ ] and root certificates, representing organizations, are trusted by microsoft windows.[ ] as of android . (jelly bean), android currently contains over cas that are updated with each release.[ ] on november , , a group of companies and nonprofit organizations, including the electronic frontier foundation, mozilla, cisco, and akamai, announced let's encrypt, a nonprofit certificate authority that provides free domain validated x. certificates as well as software to enable installation and maintenance of certificates.[ ] let's encrypt is operated by the newly formed internet security research group, a california nonprofit recognized as federally tax-exempt.[ ] according to netcraft in may , the industry standard for monitoring active tls certificates, "although the global [tls] ecosystem is competitive, it is dominated by a handful of major cas — three certificate authorities (symantec, comodo, godaddy) account for three-quarters of all issued [tls] certificates on public-facing web servers. the top spot has been held by symantec (or verisign before it was purchased by symantec) ever since [our] survey began, with it currently accounting for just under a third of all certificates. to illustrate the effect of differing methodologies, amongst the million busiest sites symantec issued % of the valid, trusted certificates in use — significantly more than its overall market share."[ ] an updated w techs survey shows that identrust, a cross-signer of let's encrypt intermediates,[ ] has remained as the most popular ssl certificate authority, while symantec has dropped out of the chart, due to its security services being acquired by digicert. sectigo (formerly comodo ca) is the third-largest ssl certificate authority with . % of the market:[ ][ ] digicert maintains the geotrust, digicert, thawte, and rapidssl brands. [ ] rank issuer usage market share identrust . % . % digicert . % . % sectigo . % . % godaddy . % . % globalsign . % . % certum . % . % actalis . % . % entrust . % . % secom . % . % let's encrypt . % . % trustwave . % . % wisekey group < . % . % startcom < . % . % network solutions < . % . % validation standards[edit] the commercial cas that issue the bulk of certificates for https servers typically use a technique called "domain validation" to authenticate the recipient of the certificate. the techniques used for domain validation vary between cas, but in general domain validation techniques are meant to prove that the certificate applicant controls a given domain name, not any information about the applicant's identity. many certificate authorities also offer extended validation (ev) certificates as a more rigorous alternative to domain validated certificates. extended validation is intended to verify not only control of a domain name, but additional identity information to be included in the certificate. some browsers display this additional identity information in a green box in the url bar. one limitation of ev as a solution to the weaknesses of domain validation is that attackers could still obtain a domain validated certificate for the victim domain, and deploy it during an attack; if that occurred, the difference observable to the victim user would be the absence of a green bar with the company name. there is some question as to whether users would be likely to recognise this absence as indicative of an attack being in progress: a test using internet explorer in showed that the absence of ie 's ev warnings were not noticed by users, however microsoft's current browser, edge, shows a significantly greater difference between ev and domain validated certificates, with domain validated certificates having a hollow, grey lock. validation weaknesses[edit] domain validation suffers from certain structural security limitations. in particular, it is always vulnerable to attacks that allow an adversary to observe the domain validation probes that cas send. these can include attacks against the dns, tcp, or bgp protocols (which lack the cryptographic protections of tls/ssl), or the compromise of routers. such attacks are possible either on the network near a ca, or near the victim domain itself. one of the most common domain validation techniques involves sending an email containing an authentication token or link to an email address that is likely to be administratively responsible for the domain. this could be the technical contact email address listed in the domain's whois entry, or an administrative email like admin@, administrator@, webmaster@, hostmaster@ or postmaster@ the domain.[ ][ ] some certificate authorities may accept confirmation using root@,[citation needed] info@, or support@ in the domain.[ ] the theory behind domain validation is that only the legitimate owner of a domain would be able to read emails sent to these administrative addresses. domain validation implementations have sometimes been a source of security vulnerabilities. in one instance, security researchers showed that attackers could obtain certificates for webmail sites because a ca was willing to use an email address like ssladmin@domain.com for domain.com, but not all webmail systems had reserved the "ssladmin" username to prevent attackers from registering it.[ ] prior to , there was no standard list of email addresses that could be used for domain validation, so it was not clear to email administrators which addresses needed to be reserved. the first version of the ca/browser forum baseline requirements, adopted november , specified a list of such addresses. this allowed mail hosts to reserve those addresses for administrative use, though such precautions are still not universal. in january , a finnish man registered the username "hostmaster" at the finnish version of microsoft live and was able to obtain a domain-validated certificate for live.fi, despite not being the owner of the domain name.[ ] issuing a certificate[edit] the procedure of obtaining a public key certificate a ca issues digital certificates that contain a public key and the identity of the owner. the matching private key is not made available publicly, but kept secret by the end user who generated the key pair. the certificate is also a confirmation or validation by the ca that the public key contained in the certificate belongs to the person, organization, server or other entity noted in the certificate. a ca's obligation in such schemes is to verify an applicant's credentials, so that users and relying parties can trust the information in the ca's certificates. cas use a variety of standards and tests to do so. in essence, the certificate authority is responsible for saying "yes, this person is who they say they are, and we, the ca, certify that".[ ] if the user trusts the ca and can verify the ca's signature, then they can also assume that a certain public key does indeed belong to whoever is identified in the certificate.[ ] example[edit] public-key cryptography can be used to encrypt data communicated between two parties. this can typically happen when a user logs on to any site that implements the http secure protocol. in this example let us suppose that the user logs on to their bank's homepage www.bank.example to do online banking. when the user opens www.bank.example homepage, they receive a public key along with all the data that their web-browser displays. the public key could be used to encrypt data from the client to the server but the safe procedure is to use it in a protocol that determines a temporary shared symmetric encryption key; messages in such a key exchange protocol can be enciphered with the bank's public key in such a way that only the bank server has the private key to read them.[ ] the rest of the communication then proceeds using the new (disposable) symmetric key, so when the user enters some information to the bank's page and submits the page (sends the information back to the bank) then the data the user has entered to the page will be encrypted by their web browser. therefore, even if someone can access the (encrypted) data that was communicated from the user to www.bank.example, such eavesdropper cannot read or decipher it. this mechanism is only safe if the user can be sure that it is the bank that they see in their web browser. if the user types in www.bank.example, but their communication is hijacked and a fake website (that pretends to be the bank website) sends the page information back to the user's browser, the fake web-page can send a fake public key to the user (for which the fake site owns a matching private key). the user will fill the form with their personal data and will submit the page. the fake web-page will then get access to the user's data. this is what the certificate authority mechanism is intended to prevent. a certificate authority (ca) is an organization that stores public keys and their owners, and every party in a communication trusts this organization (and knows its public key). when the user's web browser receives the public key from www.bank.example it also receives a digital signature of the key (with some more information, in a so-called x. certificate). the browser already possesses the public key of the ca and consequently can verify the signature, trust the certificate and the public key in it: since www.bank.example uses a public key that the certification authority certifies, a fake www.bank.example can only use the same public key. since the fake www.bank.example does not know the corresponding private key, it cannot create the signature needed to verify its authenticity.[ ] security[edit] it is difficult to assure correctness of match between data and entity when the data are presented to the ca (perhaps over an electronic network), and when the credentials of the person/company/program asking for a certificate are likewise presented. this is why commercial cas often use a combination of authentication techniques including leveraging government bureaus, the payment infrastructure, third parties' databases and services, and custom heuristics. in some enterprise systems, local forms of authentication such as kerberos can be used to obtain a certificate which can in turn be used by external relying parties. notaries are required in some cases to personally know the party whose signature is being notarized; this is a higher standard than is reached by many cas. according to the american bar association outline on online transaction management the primary points of us federal and state statutes enacted regarding digital signatures has been to "prevent conflicting and overly burdensome local regulation and to establish that electronic writings satisfy the traditional requirements associated with paper documents." further the us e-sign statute and the suggested ueta code[ ] help ensure that: a signature, contract or other record relating to such transaction may not be denied legal effect, validity, or enforceability solely because it is in electronic form; and a contract relating to such transaction may not be denied legal effect, validity or enforceability solely because an electronic signature or electronic record was used in its formation. despite the security measures undertaken to correctly verify the identities of people and companies, there is a risk of a single ca issuing a bogus certificate to an imposter. it is also possible to register individuals and companies with the same or very similar names, which may lead to confusion. to minimize this hazard, the certificate transparency initiative proposes auditing all certificates in a public unforgeable log, which could help in the prevention of phishing.[ ][ ] in large-scale deployments, alice may not be familiar with bob's certificate authority (perhaps they each have a different ca server), so bob's certificate may also include his ca's public key signed by a different ca , which is presumably recognizable by alice. this process typically leads to a hierarchy or mesh of cas and ca certificates. authority revocation lists[edit] an authority revocation list (arl) is a form of certificate revocation list (crl) containing certificates issued to certificate authorities, contrary to crls which contain revoked end-entity certificates. industry organizations[edit] certificate authority security council (casc) – in february , the casc was founded as an industry advocacy organization dedicated to addressing industry issues and educating the public on internet security. the founding members are the seven largest certificate authorities.[ ][ ] common computing security standards forum (ccsf) – in the ccsf was founded to promote industry standards that protect end users. comodo group ceo melih abdulhayoğlu is considered the founder of the ccsf.[ ] ca/browser forum – in , a new consortium of certificate authorities and web browser vendors was formed to promote industry standards and baseline requirements for internet security. comodo group ceo melih abdulhayoğlu organized the first meeting and is considered the founder of the ca/browser forum.[ ][ ] baseline requirements[edit] the ca/browser forum publishes the baseline requirements,[ ] a list of policies and technical requirements for cas to follow. these are a requirement for inclusion in the certificate stores of firefox[ ] and safari.[ ] ca compromise[edit] if the ca can be subverted, then the security of the entire system is lost, potentially subverting all the entities that trust the compromised ca. for example, suppose an attacker, eve, manages to get a ca to issue to her a certificate that claims to represent alice. that is, the certificate would publicly state that it represents alice, and might include other information about alice. some of the information about alice, such as her employer name, might be true, increasing the certificate's credibility. eve, however, would have the all-important private key associated with the certificate. eve could then use the certificate to send digitally signed email to bob, tricking bob into believing that the email was from alice. bob might even respond with encrypted email, believing that it could only be read by alice, when eve is actually able to decrypt it using the private key. a notable case of ca subversion like this occurred in , when the certificate authority verisign issued two certificates to a person claiming to represent microsoft. the certificates have the name "microsoft corporation", so they could be used to spoof someone into believing that updates to microsoft software came from microsoft when they actually did not. the fraud was detected in early . microsoft and verisign took steps to limit the impact of the problem.[ ][ ] in fraudulent certificates were obtained from comodo and diginotar,[ ][ ] allegedly by iranian hackers. there is evidence that the fraudulent diginotar certificates were used in a man-in-the-middle attack in iran.[ ] in , it became known that trustwave issued a subordinate root certificate that was used for transparent traffic management (man-in-the-middle) which effectively permitted an enterprise to sniff ssl internal network traffic using the subordinate certificate.[ ] key storage[edit] an attacker who steals a certificate authority's private keys is able to forge certificates as if they were ca, without needed ongoing access to the ca's systems. key theft is therefore one of the main risks certificate authorities defend against. publicly trusted cas almost always store their keys on a hardware security module (hsm), which allows them to sign certificates with a key, but generally prevent extraction of that key with both physical and software controls. cas typically take the further precaution of keeping the key for their long-term root certificates in an hsm that is kept offline, except when it is needed to sign shorter-lived intermediate certificates. the intermediate certificates, stored in an online hsm, can do the day-to-day work of signing end-entity certificates and keeping revocation information up to date. cas sometimes use a key ceremony when generating signing keys, in order to ensure that the keys are not tampered with or copied. implementation weakness of the trusted third party scheme[edit] the critical weakness in the way that the current x. scheme is implemented is that any ca trusted by a particular party can then issue certificates for any domain they choose. such certificates will be accepted as valid by the trusting party whether they are legitimate and authorized or not.[ ] this is a serious shortcoming given that the most commonly encountered technology employing x. and trusted third parties is the https protocol. as all major web browsers are distributed to their end-users pre-configured with a list of trusted cas that numbers in the dozens this means that any one of these pre-approved trusted cas can issue a valid certificate for any domain whatsoever.[ ] the industry response to this has been muted.[ ] given that the contents of a browser's pre-configured trusted ca list is determined independently by the party that is distributing or causing to be installed the browser application there is really nothing that the cas themselves can do. this issue is the driving impetus behind the development of the dns-based authentication of named entities (dane) protocol. if adopted in conjunction with domain name system security extensions (dnssec) dane will greatly reduce if not completely eliminate the role of trusted third parties in a domain's pki. see also[edit] safe-biopharma association - an example of a non-https ca. validation authority contact page people for internet responsibility web of trust chain of trust digital signature references[edit] ^ https://searchsecurity.techtarget.com/definition/certificate-authority ^ "mozilla included ca certificate list — mozilla". mozilla.org. archived from the original on - - . retrieved - - . ^ "emv ca". emv certificate authority worldwide. october . retrieved february , . ^ zakir durumeric; james kasten; michael bailey; j. alex halderman ( september ). "analysis of the https certificate ecosystem" (pdf). the internet measurement conference. sigcomm. archived (pdf) from the original on december . retrieved december . ^ "what is ssl certificate?". archived from the original on - - . retrieved - - . ^ "webtrust". webtrust. archived from the original on - - . retrieved - - . ^ kirk hall (april ). "standards and industry regulations applicable to certification authorities" (pdf). trend micro. archived (pdf) from the original on - - . retrieved - - . ^ "ca:includedcas - mozillawiki". wiki.mozilla.org. archived from the original on - - . retrieved - - . ^ "list of available trusted root certificates in macos high sierra". apple support. retrieved - - . ^ "microsoft included ca certificate list". ccadb-public.secure.force.com. retrieved - - . ^ "security with https and ssl". developer.android.com. archived from the original on - - . retrieved - - . ^ "let's encrypt: delivering ssl/tls everywhere" (press release). let's encrypt. archived from the original on - - . retrieved - - . ^ "about". let's encrypt. archived from the original on - - . retrieved - - . ^ "counting ssl certificates - netcraft". news.netcraft.com. archived from the original on - - . ^ "let's encrypt - chain of trust". let's encrypt. retrieved - - . ... [let's encrypt's] intermediate is ... cross-signed by another certificate authority, identrust, whose root is already trusted in all major browsers. ^ "digicert closes acquisition of symantec's website security business". symantec (press release). october , . retrieved - - . ^ "usage of ssl certificate authorities for websites". - - . retrieved - - . ^ https://www.eweek.com/security/symantec-selling-ssl-security-business-to-digicert-for- m ^ "archived copy" (pdf). archived (pdf) from the original on - - . retrieved - - .cs maint: archived copy as title (link) ^ "ca/forbidden or problematic practices - mozillawiki". wiki.mozilla.org. archived from the original on - - . retrieved - - . ^ "ssl faq - frequently asked questions - rapid ssl". www.rapidssl.com. archived from the original on - - . ^ zusman, mike ( ). criminal charges are not pursued: hacking pki (pdf). def con . las vegas. archived (pdf) from the original on - - . ^ "a finnish man created this simple email account - and received microsoft's security certificate". tivi.fi. archived from the original on - - . ^ "responsibilities of certificate authority". archived from the original on - - . retrieved - - . ^ "network world". january . ^ https://www.google.com/books/edition/applied_cryptography_and_network_securit/gggflavxyvqc?hl=en&gbpv= &dq=applied+cryptography+certificate+issuance&pg=pa &printsec=frontcover ^ https://www.google.com/books/edition/the_shortcut_guide_to_managing_certifica/bdji aqrzsic?hl=en&gbpv= &dq=digital+certificate+issuance&pg=pa &printsec=frontcover ^ "electronic signatures and records" (pdf). archived (pdf) from the original on - - . retrieved - - . ^ "certificate transparency". archived from the original on - - . retrieved - - . ^ "certificate transparency". internet engineering task force. archived from the original on - - . retrieved - - . ^ "multivendor power council formed to address digital certificate issues". network world. february , . archived from the original on july , . ^ "major certificate authorities unite in the name of ssl security". dark reading. february , . archived from the original on april , . ^ "ca/browser forum founder". archived from the original on - - . retrieved - - . ^ "ca/browser forum". archived from the original on - - . retrieved - - . ^ wilson, wilson. "ca/browser forum history" (pdf). digicert. archived (pdf) from the original on - - . retrieved - - . ^ "baseline requirements". cab forum. archived from the original on january . retrieved april . ^ "mozilla root store policy". mozilla. archived from the original on april . retrieved april . ^ "apple root certificate program". apple. archived from the original on march . retrieved april . ^ "ca- - ". cert.org. archived from the original on - - . retrieved - - . ^ microsoft, inc. ( - - ). "microsoft security bulletin ms - : erroneous verisign-issued digital certificates pose spoofing hazard". archived from the original on - - . retrieved - - . ^ bright, peter ( march ). "independent iranian hacker claims responsibility for comodo hack". ars technica. archived from the original on august . retrieved - - . ^ bright, peter ( - - ). "another fraudulent certificate raises the same old questions about certificate authorities". ars technica. archived from the original on - - . retrieved - - . ^ leyden, john ( - - ). "inside 'operation black tulip': diginotar hack analysed". the register. archived from the original on - - . ^ "trustwave issued a man-in-the-middle certificate". the h security. - - . archived from the original on - - . retrieved - - . ^ osborne, charlie. "symantec sacks staff for issuing unauthorized google certificates - zdnet". zdnet.com. archived from the original on - - . ^ "unauthorized google digital certificates discovered". linkedin.com. august . ^ "in the wake of unauthorized certificate issuance by the indian ca nic, can government cas still be considered "trusted third parties"?". casecurity.org. july . archived from the original on october . external links[edit] how secure is https today? how often is it attacked?, electronic frontier foundation ( october ) v t e tls and ssl protocols and technologies transport layer security / secure sockets layer (tls/ssl) datagram transport layer security (dtls) server name indication (sni) application-layer protocol negotiation (alpn) dns-based authentication of named entities (dane) dns certification authority authorization (caa) https http strict transport security (hsts) http public key pinning (hpkp) ocsp stapling opportunistic tls perfect forward secrecy public-key infrastructure automated certificate management environment (acme) certificate authority (ca) ca/browser forum certificate policy certificate revocation list (crl) domain-validated certificate (dv) extended validation certificate (ev) online certificate status protocol (ocsp) public key certificate public-key cryptography public key infrastructure (pki) root certificate self-signed certificate see also domain name system security extensions (dnssec) internet protocol security (ipsec) secure shell (ssh) history export of cryptography from the united states server-gated cryptography implementations bouncy castle boringssl botan bsafe cryptlib gnutls jsse libressl matrixssl mbed tls nss openssl s n schannel ssleay stunnel wolfssl notaries certificate transparency convergence https everywhere perspectives project vulnerabilities theory man-in-the-middle attack padding oracle attack cipher bar mitzvah attack protocol beast breach crime drown logjam poodle (in regards to ssl . ) implementation certificate authority compromise random number generator attacks freak goto fail heartbleed lucky thirteen attack poodle (in regards to tls . ) kazakhstan mitm attack retrieved from "https://en.wikipedia.org/w/index.php?title=certificate_authority&oldid= " categories: certificate authorities public-key cryptography key management public key infrastructure transport layer security hidden categories: cs maint: archived copy as title articles with short description short description matches wikidata articles containing potentially dated statements from august all articles containing potentially dated statements all articles with unsourced statements articles with unsourced statements from march navigation menu personal tools not logged in talk contributions create account log in namespaces article talk variants views read edit view history more search navigation main page contents current events random article about wikipedia contact us donate contribute help learn to edit community portal recent changes upload file tools what links here related changes upload file special pages permanent link page information cite this page wikidata item print/export download as pdf printable version languages العربية català Čeština deutsch español euskara فارسی français 한국어 bahasa indonesia italiano nederlands 日本語 norsk bokmål polski português Русский svenska ไทย türkçe Українська tiếng việt 中文 edit links this page was last edited on february , at : (utc). text is available under the creative commons attribution-sharealike license; additional terms may apply. by using this site, you agree to the terms of use and privacy policy. wikipedia® is a registered trademark of the wikimedia foundation, inc., a non-profit organization. privacy policy about wikipedia disclaimers contact wikipedia mobile view developers statistics cookie statement csv,conf,v - may - - online csv,conf,v home submit session proposal watch talks code of conduct faq a community conference for data makers everywhere featuring stories about data sharing and data analysis from science, journalism, government, and open source. save the date! may - , , online. talk and birds of a feather submissions are now open! submit your session proposal by february , csv,conf is about: building community csv,conf brings diverse groups together to discuss data topics, and features stories about data sharing and data analysis from science, journalism, government, and open source. people who love data csv,conf is a non-profit community conference run by folks who really love data and sharing knowledge. if you are as passionate about data and its application to society as we are, then this is the conference for you. big and small csv,conf conferences aren't just about spreadsheets. we curate content on broader topics like advancing the art of data collaboration- from putting your data on github, to producing meaningful insight by running large scale distributed processing on a cluster. csv,conf is a group effort to put on a non-profit, community conference with dozens of people putting in time, including but not limited to the organizing team: karthik ram, university of california, berkeley. program chair john chodacki, california digital library elaine wong, canadian broadcasting corporation jessica hardwicke, code for science & society jonathan cain, university of oregon jo barratt, open knowledge foundation danielle robinson, code for science & society lilly winfree, open knowledge foundation serah rono, the carpentries we strive to be a supportive and welcoming environment to all attendees. we encourage you to read the conf code of conduct and will be enforcing it. you can revisit csv,conf,v , csv,conf,v , csv,conf,v , csv,conf,v and csv,conf,v conference websites, or watch csv,conf talks on youtube. the code for this website is open source. view it on github. want to talk to us? join us on slack. password strong at duckduckgo duckduckgo you are being redirected to the non-javascript site. click here if it doesn't happen automatically. coral sheldon-hess – librarian, developer, engineer, maker, bird nerd skip to content coral sheldon-hess librarian, developer, engineer, maker, bird nerd open primary menu twitter linkedin rss github ravelry about blog contact sidebar posts by category librarianship leadership teaching and learning social justice alaska technology travel crafts geekery hiring and employment on a personal note past projects coral sheldon-hess posts the online unconference of niche interests the online unconference of niche interests published by coral sheldon-hess on november , if you’re looking for a fun and educational thing to do this weekend, you might consider attending the second quarterly(??) online unconference of niche interests (“ouni” for short), scheduled to run from pm until a bit after pm eastern standard time, this sunday, november . we have a set of volunteer presenters who will each talk for up to minutes about a niche topic they’re into. continue readingthe online unconference of niche interests what i’m telling family about covid- what i’m telling family about covid- published by coral sheldon-hess on june , a family member asked me to tell them about covid- . it was a general question, which i chose to interpret as “how does transmission work, and what is the real risk?” this is what i said. as i told them, i’m not a biologist of any sort, and i will accept corrections (both from people who are biologists and from those who can cite sources), of course. both this person and i have autoimmune issues, so i take that as a given in this post. continue readingwhat i’m telling family about covid- what i’ve been up to during all this what i’ve been up to during all this published by coral sheldon-hess on april , how my household is doing perhaps the best place to start writing about what i’ve been up to is to be really clear: i’m ok, and, at least for now, so are my loved ones. my spouse and i are both incredibly lucky to have jobs that can be done… continue readingwhat i’ve been up to during all this get that bread get that bread published by coral sheldon-hess on february , i want to tell you about my take on the new artisan bread in five minutes a day recipe. the things i have to add to the discussion: ) a couple of hacks for people who, like me, do not have a kitchen fan that vents outdoors (i promise i’ll explain why this matters) and who like at least a little bit of whole grain in their bread, plus ) photos of some of the steps they don’t show as clearly in the book. i’m still experimenting (always!), but i have a base recipe/approach that i like and that i think is good enough to share. continue readingget that bread year-end post year-end post published by coral sheldon-hess on december , we’re rapidly approaching the time for the traditional year-end post, which i’ve been known to skip in recent years—i had a run of several really rough years, there. while wasn’t without personal challenges and setbacks (and a whole lot of frightening developments in the us and abroad), it brought… continue reading year-end post belated update belated update published by coral sheldon-hess on october , right now i should be grading or preparing for classes, but honestly i’m three blog posts behind where i wanted to be by now (i haven’t forgotten my wiscon promise to make a post about tabletop roleplaying games) and fighting a pretty nasty headache. so what if i take a… continue readingbelated update doing data things doing data things published by coral sheldon-hess on december , tldr: i took two classes this semester, and i’m going to teach at least one, probably . , classes next semester. i’m super psyched about it. i’ll still work for the library where i’m an adjunct, too, but fewer hours per week. i’m still available for full-time hire, if you have data for me to work with. continue readingdoing data things published by coral sheldon-hess on december , i usually do a year-end post. that’s not happening in . this year took so much from me, and from people i care about, that i refuse to write about it. but i’d like to write about . not “resolutions” so much as “plans and goals”—and maybe not even those… continue reading a librarian again a librarian again published by coral sheldon-hess on october , over the past few years, i’ve come to dread the “what do you do?” question, because what people generally mean is “where do you work?” and it’s awkward when you can’t have that conversation the way they expect. continue readinga librarian again dlf in pittsburgh dlf in pittsburgh published by coral sheldon-hess on september , (updated) this is just a really quick post to say that dlf forum is in my hometown, this year, and i’d love to meet up with some of my internet-and-conference friends, if travel and conference scheduling makes that a possibility for any of you! i live here, and most of our public transit is downtown-centered; i can come to you, or i can give you easy directions to meet me somewhere if you’re feeling adventurous. continue readingdlf in pittsburgh posts navigation … next author wordpress theme by compete themes data carpentry data carpentry navigation sponsors contact search home about mission & vision media our team code of conduct governance assessment workshops about our workshops request a workshop attend a workshop upcoming workshops past workshops get involved become a member organization become an instructor join our community newsletter help develop lessons jobs donate lessons ecology genomics social sciences geospatial in development semester materials curriculum advisory committeee blog blog for instructors instructor training checkout workshop checklists self-organized workshops mentoring opportunities what is data carpentry? data carpentry develops and teaches workshops on the fundamental data skills needed to conduct research. our mission is to provide researchers high-quality, domain-specific training covering the full lifecycle of data-driven research. data carpentry is now a lesson program within the carpentries, having merged with software carpentry in january, . data carpentry's focus is on the introductory computational skills needed for data management and analysis in all domains of research. our lessons are domain-specific, and build on the existing knowledge of learners to enable them to quickly apply skills learned to their own research. our initial target audience is learners who have little to no prior computational experience. we create a friendly environment for learning to empower researchers and enable data driven discovery. host a workshop find out how to host a workshop at your institution or organization more › attend a workshop most workshops are coordinated by local hosts. find out if there is a workshop in your area or how to attend one. more › get involved data carpentry is a volunteer effort. find out how you can be an instructor, help develop content, or support us. more › upcoming workshops click on an individual event to learn more about that event, including contact information and registration instructions. leiden university and delft university of technology (online) instructors: kristina hettne, jeff love helpers: esther plomp, yan wang jan - jan , university of wisconsin-madison (online) instructors: christina koch, sarah stevens, trisha adamus, clare michaud helpers: heather shimon, cid freitag, scott wildman jan - jan , rutgers university (online) instructors: vasileios panagiotis, aleksander jankowski, monah abou alezz, nadine bestard jan - jan , betterlab instructors: nelly sélem, josé manuel villalobos escobedo, abraham avelar jan - jan , amos instructors: damien irving, holger wolff, claire trenham feb - feb , max delbrück center for molecular medicine (online) instructors: liz ing-simmons, aleksander jankowski, monah abou alezz, tobias busch feb - feb , ohio state, university libraries and translational data analytics institute (online) instructors: scott peterson, jennifer lee, annajiat alim rasel feb - feb , institute for modeling collaboration and innovation @ the university of idaho (online) instructors: james van leuven, breanna sipley helpers: travis seaborn, amanda stahlke mar - mar , institute for modeling collaboration and innovation @ the university of idaho (online) instructors: erich seamon helpers: travis seaborn apr - apr , ** workshops marked with asterisks are based on curriculum from the carpentries lesson programs but may not follow our standard workshop format. workshops with a globe icon are being held online. the corresponding flag notes the country where the host organisation is based. check out our past workshops. blog posts in addition to the posts below, find out what's happening in our community through the carpentries blog, a great resource that collates posts from data carpentry, library carpentry, and software carpentry, and publishes updates of general interest to the community. lessons · genomics workshop pilot and bugbbq at university of arizonajason williams, erin becker, françois michonneau / - - lessons · atmos ocean launchdamien irving / - - lessons · geospatial launcherin becker / - - more ›  about this site data carpentry is a lesson program of the carpentries that develops and provides data skills training to researchers. more › services contact rss sitemap.xml created with ♥ by phlow with jekyll using feeling responsive acceptance of election as general secretary | world council of churches from: this is an annotated view created by jez cope. code lib | we are developers and technologists for libraries, museums, and archives who are dedicated to being a diverse and inclusive community, seeking to share ideas and build collaboration. about chat conference jobs journal local mailing list planet wiki code lib.org was migrated from drupal to jekyll in june . some links may still be broken. to report issues or help fix see: https://github.com/code lib/code lib.github.io posts nov , code lib sep , code lib aug , code lib apr , code lib journal issue call for papers oct , issue of the code lib journal aug , code lib jul , issue of the code lib journal jun , code lib journal issue call for papers oct , code lib journal # oct , c l : call for presentation/panel proposals oct , code lib jul , code lib journal # apr , code lib journal # sep , jobs.code lib.org studied aug , code lib jul , code lib northern california: stanford, ca jul , code lib journal # apr , code lib journal # : special issue on diversity in library technology mar , code lib will be in philadelphia mar , code lib conference proposals feb , code lib north : st. catharines, on feb , code lib videos jan , code of conduct dec , code lib diversity scholarships dec , your code does not exist in a vacuum dec , your chocolate is in my peanut butter! mixing up content and presentation layers to build smarter books in browsers with rdfa, schema.org, and linked data topics dec , you gotta keep 'em separated: the case for "bento box" discovery interfaces dec , refinery — an open source locally deployable web platform for the analysis of large document collections dec , programmers are not projects: lessons learned from managing humans dec , our $ , problem: why library school? dec , making your digital objects embeddable around the web dec , leveling up your git workflow dec , level up your coding with code club (yes, you can talk about it) dec , how to hack it as a working parent: or, should your face be bathed in the blue glow of a phone at am? dec , helping google (and scholars, researchers, educators, & the public) find archival audio dec , heiðrún: dpla's metadata harvesting, mapping and enhancement system dec , got git? getting more out of your github repositories dec , feminist human computer interaction (hci) in library software dec , dynamic indexing: a tragic solr story dec , docker? vms? ec ? yes! with packer.io dec , digital content integrated with ils data for user discovery: lessons learned dec , designing and leading a kick a** tech team dec , consuming big linked open data in practice: authority shifts and identifier drift dec , byob: build your own bootstrap dec , book reader bingo: which page-turner should i use? dec , beyond open source dec , awesome pi, lol! dec , annotations as linked data with fedora and triannon (a real use case for rdf!) dec , american (archives) horror story: lto failure and data loss dec , a semantic makeover for cms data dec , code lib lighting talks nov , store nov , voting for code lib prepared talks is now open. nov , keynote voting for the conference is now open! sep , code lib : call for proposals sep , code lib north (ottawa): tuesday october th, sep , code libbc: november and , sep , conference schedule jul , code lib journal issue jul , code lib norcal july in san mateo jul , code lib apr , code lib trip report - zahra ashktorab apr , code lib trip report- nabil kashyap apr , code lib trip report - junior tidal apr , code lib trip report - jennifer maiko kishi apr , code lib trip report - j. (jenny) gubernick apr , code lib trip report - emily reynolds apr , code lib trip report - coral sheldon hess apr , code lib trip report - christina harlow apr , code lib trip report - arie nugraha mar , call for proposals: code lib journal, issue feb , code of conduct jan , code lib call for host proposals jan , code lib sponsors jan , websockets for real-time and interactive interfaces jan , we are all disabled! universal web design making web services accessible for everyone jan , visualizing solr search results with d .js for user-friendly navigation of large results sets jan , visualizing library resources as networks jan , under the hood of hadoop processing at oclc research jan , towards pasta code nirvana: using javascript mvc to fill your programming ravioli jan , sustaining your open source project through training jan , structured data now: seeding schema.org in library systems jan , quick and easy data visualization with google visualization api and google chart libraries jan , queue programming -- how using job queues can make the library coding world a better place jan , phantomjs+selenium: easy automated testing of ajax-y uis jan , personalize your google analytics data with custom events and variables jan , organic free-range api development - making web services that you will actually want to consume jan , next generation catalogue - rdf as a basis for new services jan , more like this: approaches to recommending related items using subject headings jan , lucene's latest (for libraries) jan , discovering your discovery system in real time jan , dead-simple video content management: let your filesystem do the work jan , building for others (and ourselves): the avalon media system jan , behold fedora : the incredible shrinking repository! jan , all tiled up jan , a reusable application to enable self deposit of complex objects into a digital preservation environment jan , a book, a web browser and a tablet: how bibliotheca alexandrina's book viewer framework makes it possible jan , conference schedule jan , code lib conference diversity scholarship recipients nov , code lib diversity scholarships (application deadline: dec. , , pm est) nov , code lib keynote speakers sep , code lib jun , code lib conference prospectus for sponsors mar , code lib conference proposals jan , ask anything! dec , code lib call for host proposals dec , the care and feeding of a crowd dec , the avalon media system: a next generation hydra head for audio and video delivery dec , solr update dec , rest is your mobile strategy dec , practical relevance ranking for million books. dec , pitfall! working with legacy born digital materials in special collections dec , n characters in search of an author dec , linked open communism: better discovery through data dis- and re- aggregation dec , hybrid archival collections using blacklight and hydra dec , html video now! dec , hands off! best practices and top ten lists for code handoffs dec , hacking the dpla dec , google analytics, event tracking and discovery tools dec , evolving towards a consortium marcr redis datastore dec , ead without xslt: a practical new approach to web-based finding aids dec , de-sucking the library user experience dec , data-driven documents: visualizing library data with d .js dec , creating a commons dec , citation search in solr and second-order operators dec , browser/javascript integration testing with ruby dec , architecting scholarsphere: how we built a repository app that doesn't feel like yet another janky old repository app dec , all teh metadatas re-revisited dec , actions speak louder than words: analyzing large-scale query logs to improve the research experience nov , code lib scholarship (deadline: december , ) nov , code lib nov , code lib schedule oct , code lib conference call for propoosals sep , keynote voting for the conference is now open! jul , dates set for code lib in chicago may , code lib journal - call for proposals may , ruby-marc . . released apr , code lib journal: editors wanted feb , code lib journal issue is published! feb , ask anything! – facilitated by carmen mitchell- code lib jan , relevance ranking in the scholarly domain - tamar sadeh, phd jan , kill the search button ii - the handheld devices are coming - jørn thøgersen, michael poltorak nielsen jan , stack view: a library browsing tool - annie cain jan , search engine relevancy tuning - a static rank framework for solr/lucene - mike schultz jan , practical agile: what's working for stanford, blacklight, and hydra - naomi dushay jan , nosql bibliographic records: implementing a native frbr datastore with redis - jeremy nelson jan , lies, damned lies, and lines of code per day - james stuart jan , indexing big data with tika, solr & map-reduce - scott fisher, erik hetzner jan , in-browser data storage and me - jason casden jan , how people search the library from a single search box - cory lown jan , discovering digital library user behavior with google analytics - kirk hess jan , building research applications with mendeley - william gunn jan , your ui can make or break the application (to the user, anyway) - robin schaaf jan , your catalog in linked data - tom johnson jan , the golden road (to unlimited devotion): building a socially constructed archive of grateful dead artifacts - robin chandler jan , quick and ~~dirty~~ clean usability: rapid prototyping with bootstrap - shaun ellis jan , “linked-data-ready” software for libraries - jennifer bowen jan , html microdata and schema.org - jason ronallo jan , hathitrust large scale search: scalability meets usability - tom burton-west jan , design for developers - lisa kurt jan , beyond code: versioning data with git and mercurial - charlie collett, martin haye jan , all teh metadatas! or how we use rdf to keep all of the digital object metadata formats thrown at us - declan fleming dec , discussion for elsevier app challenge during code lib dec , so you want to start a kindle lending program dec , code lib call for host proposals nov , code lib scholarship (deadline: december , ) oct , code lib sponsor listing oct , code lib schedule jul , code lib feb , code lib sponsorship jan , vufind beyond marc: discovering everything else - demian katz jan , one week | one tool: ultra-rapid open source development among strangers - scott hanrath jan , letting in the light: using solr as an external search component - jay luker and benoit thiell jan , kuali ole: architecture for diverse and linked data - tim mcgeary and brad skiles jan , keynote address - diane hillmann jan , hey, dilbert. where's my data?! - thomas barker jan , enhancing the mobile experience: mobile library services at illinois - josh bishoff - josh bishoff jan , drupal as rapid application development tool - cary gordon jan , code lib in seattle jan , lightning talks jan , breakout sessions jan , (yet another) home-grown digital library system, built upon open source xml technologies and metadata standards - david lacy jan , why (code ) libraries exist - eric hellman jan , visualizing library data - karen coombs jan , sharing between data repositories - kevin s. clarke jan , practical relevancy testing - naomi dushay jan , opinionated metadata (om): bringing a bit of sanity to the world of xml metadata - matt zumwalt jan , mendeley's api and university libraries: three examples to create value - ian mulvany jan , let's get small: a microservices approach to library websites - sean hannan jan , gis on the cheap - mike graves jan , fiwalk with me: building emergent pre-ingest workflows for digital archival records using open source forensic software - mark m jan , enhancing the performance and extensibility of the xc’s metadataservicestoolkit - ben anderson jan , chicago underground library’s community-based cataloging system - margaret heller and nell taylor jan , building an open source staff-facing tablet app for library assessment - jason casden and joyce chapman jan , beyond sacrilege: a couchapp catalog - gabriel farrell jan , ask anything! – facilitated by dan chudnov jan , a community-based approach to developing a digital exhibit at notre dame using the hydra framework - rick johnson and dan brubak dec , code lib schedule dec , code lib call for host proposals nov , scholarships to attend the code lib conference (deadline dec. , ) sep , code lib sponsorship jun , issue of the code lib journal mar , location of code lib mar , code lib : get ready for the best code lib conference yet! mar , issue of the code lib journal mar , vote on code lib hosting proposals feb , you either surf or you fight: integrating library services with google wave - sean hannan - code lib feb , vampires vs. werewolves: ending the war between developers and sysadmins with puppet - bess sadler - code lib feb , the linked library data cloud: stop talking and start doing - ross singer - code lib feb , taking control of library metadata and websites using the extensible catalog - jennifer bowen - code lib feb , public datasets in the cloud - rosalyn metz and michael b. klein - code lib feb , mobile web app design: getting started - michael doran - code lib feb , metadata editing – a truly extensible solution - david kennedy and david chandek-stark - code lib feb , media, blacklight, and viewers like you (pdf, . mb) - chris beer - code lib feb , matching dirty data – yet another wheel - anjanette young and jeff sherwood - code lib feb , library/mobile: developing a mobile catalog - kim griggs - code lib feb , keynote # : catfish, cthulhu, code, clouds and levenshtein distance - paul jones - code lib feb , keynote # : cathy marshall - code lib feb , iterative development done simply - emily lynema - code lib feb , i am not your mother: write your test code - naomi dushay, willy mene, and jessie keck - code lib feb , how to implement a virtual bookshelf with solr - naomi dushay and jessie keck - code lib feb , hive: a new tool for working with vocabularies - ryan scherle and jose aguera - code lib feb , enhancing discoverability with virtual shelf browse - andreas orphanides, cory lown, and emily lynema - code lib feb , drupal : a more powerful platform for building library applications - cary gordon - code lib feb , do it yourself cloud computing with apache and r - harrison dekker - code lib feb , cloud lib - jeremy frumkin and terry reese - code lib feb , becoming truly innovative: migrating from millennium to koha - ian walls - code lib feb , ask anything! – facilitated by dan chudnov - code lib feb , a better advanced search - naomi dushay and jessie keck - code lib feb , ways to enhance library interfaces with oclc web services - karen coombs - code lib feb , code lib lightning talks feb , code lib breakout sessions feb , code lib participant release form feb , code lib hosting proposals solicited jan , code lib scholarship recipients jan , code lib north dec , scholarships to attend the code lib conference dec , code lib registration dec , conference info dec , code lib schedule dec , code lib sponsorship nov , code lib conference prepared talks voting now open! oct , code lib call for prepared talk proposals sep , vote for code lib keynotes! jul , code lib jun , code lib journal: new issue now available may , visualizing media archives: a case study may , the open platform strategy: what it means for library developers may , if you love something...set it free may , what we talk about when we talk about frbr may , the rising sun: making the most of solr power may , great facets, like your relevance, but can i have links to amazon and google book search? may , freecite - an open source free-text citation parser may , freebasing for fun and enhancement may , extending biblios, the open source web based metadata editor may , complete faceting may , a new platform for open data - introducing ‡biblios.net web services may , sebastian hammer, keynote address may , blacklight as a unified discovery platform may , a new frontier - the open library environment (ole) may , the dashboard initiative may , restafarian-ism at the nla may , open up your repository with a sword! may , lusql: (quickly and easily) getting your data from your dbms into lucene may , like a can opener for your data silo: simple access through atompub and jangle may , libx . may , how i failed to present on using dvcs for managing archival metadata may , djatoka for djummies may , a bookless future for libraries: a comedy in acts may , why libraries should embrace linked data mar , code lib journal: new issue now available feb , see you next year in asheville feb , code lib lightning talks feb , code lib venue voting feb , oclc grid services boot camp ( preconference) feb , code lib hosting proposals jan , code lib logo jan , code lib logo debuts jan , code lib breakout sessions jan , call for code lib hosting proposals jan , code lib scholarship recipients jan , code lib t-shirt design contest dec , code lib registration open! dec , code lib journal issue published dec , code lib gender diversity and minority scholarships dec , calling all code libers attending midwinter dec , logo design process launched dec , code lib schedule dec , pre-conferences nov , voting on presentations for code lib open until december nov , drupal lib unconference ( / / darien, ct) oct , call for proposals, code lib conference oct , ne.code lib.org sep , code lib keynote voting sep , logo? you decide sep , solrpy google code project sep , code lib sep , code lib sponsorship aug , code libnyc aug , update from linkedin jul , linkedin group growing fast jul , code lib group on linkedin apr , elpub open scholarship: authority, community and sustainability in the age of web . mar , code libcon lightning talks mar , brown university to host code lib feb , desktop presenter software feb , presentations from libraryfind pre-conference feb , vote for code lib host! feb , karen coyle keynote - r&d: can resource description become rigorous data? feb , code libcon breakout sessions feb , call for code lib hosting proposals jan , code lib conference t-shirt design jan , code lib registration now open! dec , zotero and you, or bibliography on the semantic web dec , xforms for metadata creation dec , working with the worldcat api dec , using a css framework dec , the wayback machine dec , the making of the code lib journal dec , the code lib future dec , show your stuff, using omeka dec , second life web interoperability - moodle and merlot.org dec , rdf and rda: declaring and modeling library metadata dec , ÖpënÜrl dec , oss web-based cataloging tool dec , marcthing dec , losing sleep over rest? dec , from idea to open source dec , finding relationships in marc data dec , dlf ils discovery interface task force api recommendation dec , delivering library services in the web . environment: osu libraries publishing system for and by librarians dec , couchdb is sacrilege... mmm, delicious sacrilege dec , building the open library dec , building mountains out of molehills dec , a metadata registry dec , code lib gender diversity and minority scholarships dec , conference schedule nov , code lib keynote survey oct , code lib call for proposals oct , code lib schedule jul , code lib conference jul , random #code lib quotes jun , request for proposals: innovative uses of crossref metadata may , library camp nyc, august , apr , code lib - video, audio and podcast available mar , code lib - day video available mar , erik hatcher keynote mar , my adventures in getting data into the archiviststoolkit mar , karen schneider keynote "hurry up please it's time" mar , code lib conference feedback available mar , code lib video trickling in mar , code lib.org restored feb , code lib will be in portland, or feb , code lib blog anthology feb , the intellectual property disclosure process: releasing open source software in academia feb , polling for interest in a european code lib feb , call for proposals to host code lib feb , code lib scholarship recipients feb , delicious! flare + simile exhibit jan , open access self-archiving mandate jan , evergreen keynote jan , code lib t-shirt contest jan , stone soup jan , #code lib logging jan , two scholarships to attend the code lib conference dec , conference schedule now available dec , code lib pre-conference workshop: lucene, solr, and your data dec , traversing the last mile dec , the xquery exposé: practical experiences from a digital library dec , the bibapp dec , smart subjects - application independent subject recommendations dec , open-source endeca in lines or less dec , on the herding of cats dec , obstacles to agility dec , myresearch portal: an xml based catalog-independent opac dec , libraryfind dec , library-in-a-box dec , library data apis abound! dec , get groovy at your public library dec , fun with zeroconfmetaopensearch dec , free the data: creating a web services interface to the online catalog dec , forget the lipstick. this pig just needs social skills. dec , atom publishing protocol primer nov , barton data nov , mit catalog data oct , code lib downtime oct , call for proposals aug , code lib audio aug , book club jul , code libcon site proposals jul , improving code libcon * jun , code lib conference hosting jun , learning to scratch our own itches jun , code lib conference jun , code lib conference schedule jun , code lib conference lightning talks jun , code lib conference breakouts mar , results of the journal name vote mar , #dspace mar , #code lib logging mar , regulars on the #code lib irc channel mar , code lib journal name vote mar , code lib journal: mission, format, guidelines mar , #code lib irc channel faq feb , cufts aim/aol/icq bot feb , code lib journal: draft purpose, format, and guidelines feb , code lib breakout sessions feb , unapi revision feb , code lib presentations will be available feb , planet update feb , weather in corvallis for code lib feb , holiday inn express feb , conference wiki jan , portland hostel jan , lightning talks jan , code lib t-shirt design vote! jan , portland jazz festival jan , unapi version jan , conference schedule in hcalendar jan , code lib t-shirt design contest jan , conference schedule set jan , code lib registration count pool jan , wikid jan , the case for code lib c( ) jan , teaching the library and information community how to remix information jan , practical aspects of implementing open source in armenia jan , lipstick on a pig: ways to improve the sex life of your opac jan , generating recommendations in opacs: initial results and open areas for exploration jan , erp options in an oss world jan , ahah: when good is better than best jan , , lines of code, and other topics from oclc research jan , what blog applications can teach us about library software architecture jan , standards, reusability, and the mating habits of learning content jan , quality metrics jan , library text mining jan , connecting everything with unapi and opa jan , chasing babel jan , anatomy of adore jan , voting on code lib presentation proposals jan , one more week for proposals dec , code lib card dec , planet facelift dec , registration is open dec , planet code lib & blogs dec , code lib call for proposals nov , code lib conference : schedule nov , panizzi nov , drupal installed nov , code lib subscribe via rss code lib code lib code lib code lib.social code lib code lib we are developers and technologists for libraries, museums, and archives who are dedicated to being a diverse and inclusive community, seeking to share ideas and build collaboration. coral sheldon-hess coral sheldon-hess librarian, developer, engineer, maker, bird nerd the online unconference of niche interests if you're looking for a fun and educational thing to do this weekend, you might consider attending the second quarterly(??) online unconference of niche interests ("ouni" for short), scheduled to run from pm until a bit after pm eastern standard time, this sunday, november . we have a set of volunteer presenters who will each talk for up to minutes about a niche topic they're into.continue readingthe online unconference of niche interests what i’m telling family about covid- a family member asked me to tell them about covid- . it was a general question, which i chose to interpret as "how does transmission work, and what is the real risk?" this is what i said. as i told them, i'm not a biologist of any sort, and i will accept corrections (both from people who are biologists and from those who can cite sources), of course. both this person and i have autoimmune issues, so i take that as a given in this post.continue readingwhat i&# ;m telling family about covid- what i’ve been up to during all this how my household is doing perhaps the best place to start writing about what i&# ;ve been up to is to be really clear: i&# ;m ok, and, at least for now, so are my loved ones. my spouse and i are both incredibly lucky to have jobs that can be done&# ;continue readingwhat i&# ;ve been up to during all this get that bread i want to tell you about my take on the new artisan bread in five minutes a day recipe. the things i have to add to the discussion: ) a couple of hacks for people who, like me, do not have a kitchen fan that vents outdoors (i promise i'll explain why this matters) and who like at least a little bit of whole grain in their bread, plus ) photos of some of the steps they don't show as clearly in the book. i'm still experimenting (always!), but i have a base recipe/approach that i like and that i think is good enough to share.continue readingget that bread year-end post we&# ;re rapidly approaching the time for the traditional year-end post, which i&# ;ve been known to skip in recent years—i had a run of several really rough years, there. while wasn&# ;t without personal challenges and setbacks (and a whole lot of frightening developments in the us and abroad), it brought&# ;continue reading year-end post belated update right now i should be grading or preparing for classes, but honestly i&# ;m three blog posts behind where i wanted to be by now (i haven&# ;t forgotten my wiscon promise to make a post about tabletop roleplaying games) and fighting a pretty nasty headache. so what if i take a&# ;continue readingbelated update doing data things tldr: i took two classes this semester, and i'm going to teach at least one, probably . , classes next semester. i'm super psyched about it. i'll still work for the library where i'm an adjunct, too, but fewer hours per week. i'm still available for full-time hire, if you have data for me to work with.continue readingdoing data things i usually do a year-end post. that&# ;s not happening in . this year took so much from me, and from people i care about, that i refuse to write about it. but i&# ;d like to write about . not &# ;resolutions&# ; so much as &# ;plans and goals&# ;&# ;and maybe not even those&# ;continue reading a librarian again over the past few years, i’ve come to dread the “what do you do?” question, because what people generally mean is “where do you work?” and it’s awkward when you can’t have that conversation the way they expect. continue readinga librarian again dlf in pittsburgh (updated) this is just a really quick post to say that dlf forum is in my hometown, this year, and i'd love to meet up with some of my internet-and-conference friends, if travel and conference scheduling makes that a possibility for any of you! i live here, and most of our public transit is downtown-centered; i can come to you, or i can give you easy directions to meet me somewhere if you're feeling adventurous. continue readingdlf in pittsburgh none bonsai elasticsearch - add-ons - heroku elements skip navigation show nav heroku products heroku platform heroku dx heroku flow continuous delivery continuous integration heroku opex heroku runtime heroku dynos heroku data services heroku postgres heroku redis apache kafka on heroku heroku enterprise heroku private spaces heroku connect heroku shield heroku teams marketplace add-ons buttons buildpacks about pricing documentation support more resources what is heroku? help customers careers events podcasts compliance center heroku is for developers ctos team collaboration startups enterprises agencies students see more languages node ruby java php python go scala clojure see more latest news from the heroku blog heroku blog find out what's new with heroku on our blog. more news view all blog posts search: log in or sign up add-ons buttons buildpacks bonsai elasticsearch heroku’s first elasticsearch offering, from free instances to private spaces starting at $ /mo. keep your clusters green we’ve spent the last several years developing and maintaining a platform that ensures that your cluster is always running as efficiently as possible. we run optimal versions of elasticsearch for you, and you can take your data with you at any time. we provide top-notch visibility, custom tooling and configurations, and guarantee that if you’re having a problem with your cluster, we’ll be on the hook to help get it green again. start easily, scale massively click a button, launch a cluster. as your application grows and your needs change, no worries – we’ve got an instance for that. from hobby projects to private spaces and everything in between, regardless of the type of instance you need, we can help. we ♥️ the heroku workflow bonsai on heroku supports the heroku workflow – because that’s how we work too. review apps, metrics, cli access, and more. everything you’d expect from the operational excellence that defines heroku, you can expect from us as well. heroku’s first elasticsearch integration we’ve been hosting full-text search for thousands of applications and billions of documents since . our team developed extensive tooling to build and manage search clusters, and continuously monitor hundreds of system metrics and health checks / . anyone can run elasticsearch for you – only bonsai guarantees to keep you green. private spaces for critical use if you’re deploying your application on heroku private spaces and need a search solution, look no further. we integrate deeply with the heroku platform and can guarantee your data is secure with us. region availability the available application locations for this add-on are shown below, and depend on whether the application is deployed to a common runtime region or private space. learn more common runtime private spaces region available united states available europe available region available installable in space virginia available oregon available frankfurt available tokyo available sydney available dublin available plans & pricing sandbox free staging $ /mo standard sm $ /mo standard md $ /mo standard lg $ /mo business compute lg $ /mo business capacity lg $ /mo private compute lg $ /mo business compute xl $ /mo private capacity lg $ /mo private compute xl $ /mo business capacity xl $ /mo business compute x $ /mo private capacity xl $ /mo private compute x $ /mo business capacity x $ /mo private capacity x $ /mo need a larger plan? let our customer success team help! learn more. / monitoring high-availability replication and failover business-hours email & ticket support / enterprise support capacity mb memory limit mb cpu cores hourly backups shards documents , concurrent searches concurrent indexing daily requests , free kibana custom plugins / monitoring high-availability replication and failover business-hours email & ticket support / enterprise support capacity gb memory limit mb cpu cores hourly backups shards documents , concurrent searches concurrent indexing daily requests free kibana custom plugins / monitoring high-availability replication and failover business-hours email & ticket support / enterprise support capacity gb memory limit gb cpu cores hourly backups shards documents , , concurrent searches concurrent indexing daily requests , , free kibana custom plugins / monitoring high-availability replication and failover business-hours email & ticket support / enterprise support capacity gb memory limit gb cpu cores hourly backups shards documents , , concurrent searches concurrent indexing daily requests , , free kibana custom plugins / monitoring high-availability replication and failover business-hours email & ticket support / enterprise support capacity gb memory limit gb cpu cores hourly backups shards documents , , concurrent searches concurrent indexing daily requests , , free kibana custom plugins / monitoring high-availability replication and failover business-hours email & ticket support / enterprise support capacity gb memory limit gb cpu cores hourly backups shards documents concurrent searches concurrent indexing daily requests free kibana custom plugins / monitoring high-availability replication and failover business-hours email & ticket support / enterprise support capacity gb memory limit gb cpu cores hourly backups shards documents concurrent searches concurrent indexing daily requests free kibana custom plugins / monitoring high-availability replication and failover business-hours email & ticket support / enterprise support capacity gb memory limit gb cpu cores hourly backups shards documents concurrent searches concurrent indexing daily requests free kibana custom plugins / monitoring high-availability replication and failover business-hours email & ticket support / enterprise support capacity gb memory limit gb cpu cores hourly backups shards documents concurrent searches concurrent indexing daily requests free kibana custom plugins / monitoring high-availability replication and failover business-hours email & ticket support / enterprise support capacity gb memory limit gb cpu cores hourly backups shards documents concurrent searches concurrent indexing daily requests free kibana custom plugins / monitoring high-availability replication and failover business-hours email & ticket support / enterprise support capacity gb memory limit gb cpu cores hourly backups shards documents concurrent searches concurrent indexing daily requests free kibana custom plugins / monitoring high-availability replication and failover business-hours email & ticket support / enterprise support capacity . tb memory limit gb cpu cores hourly backups shards documents concurrent searches concurrent indexing daily requests free kibana custom plugins / monitoring high-availability replication and failover business-hours email & ticket support / enterprise support capacity gb memory limit gb cpu cores hourly backups shards documents concurrent searches concurrent indexing daily requests free kibana custom plugins / monitoring high-availability replication and failover business-hours email & ticket support / enterprise support capacity . tb memory limit gb cpu cores hourly backups shards documents concurrent searches concurrent indexing daily requests free kibana custom plugins / monitoring high-availability replication and failover business-hours email & ticket support / enterprise support capacity gb memory limit gb cpu cores hourly backups shards documents concurrent searches concurrent indexing daily requests free kibana custom plugins / monitoring high-availability replication and failover business-hours email & ticket support / enterprise support capacity . tb memory limit gb cpu cores hourly backups shards documents concurrent searches concurrent indexing daily requests free kibana custom plugins / monitoring high-availability replication and failover business-hours email & ticket support / enterprise support capacity . tb memory limit gb cpu cores hourly backups shards documents concurrent searches concurrent indexing daily requests free kibana custom plugins install bonsai elasticsearch heroku addons:create bonsai to provision, copy the snippet into your cli or use the install button above. bonsai elasticsearch documentation installing the add-on the bonsai dashboard non-production tier clusters migrating between plans removing the add-on learn more terms of service support additional resources install bonsai elasticsearch quick links add-on details region availability plans & pricing documentation heroku elements terms of use (default) shareable details not shareable across apps multiple installs/app add-on category search language support clojure go java node php python ruby scala add-on provider admin become an add-on provider products heroku platform heroku data services heroku postgres heroku redis apache kafka on heroku heroku enterprise heroku private spaces heroku connect heroku shield heroku teams elements marketplace pricing resources documentation compliance center training & education blog podcasts get started about about us what is heroku heroku & salesforce our customers careers partners elements marketplace support help center status premium support contact us subscribe to our monthly newsletter your email address: rss heroku blog heroku news blog heroku engineering blog dev center articles dev center changelog heroku status podcasts twitter heroku dev center articles dev center changelog heroku status facebook instagram github linkedin youtube heroku is a company heroku.com terms of service privacy cookies © salesforce.com the open library blog the open library blog a web page for every book introducing the open library explorer try it here! if you like it, share it. bringing years of librarian-knowledge to life by nick norman with drini cami &# ; mek at the library leaders forum (demo), open library unveiled the beta for what it&# ;s calling the library explorer: an immersive interface which powerfully recreates and enhances the experience of navigating [&# ;] importing your goodreads & accessing them with open library’s apis by mek today joe alcorn, founder of readng, published an article (https://joealcorn.co.uk/blog/ /goodreads-retiring-api) sharing news with readers that amazon&# ;s goodreads service is in the process of retiring their developer apis, with an effective start date of last tuesday, december th, . the topic stirred discussion among developers and book lovers alike, making the front-page of the [&# ;] on bookstores, libraries & archives in the digital age the following was a guest post by brewster kahle on against the grain (atg) &# ; linking publishers, vendors, &# ; librarians by:&# ;brewster kahle, founder &# ; digital librarian, internet archive back in ,&# ;i was honored to give a keynote at the meeting of the&# ;society of american archivists, when the president of the society presented me with a [&# ;] amplifying the voices behind books exploring how open library uses author data to help readers move from imagination to impact by nick norman, edited by mek &# ; drini according to rené descartes, a creative mathematician, “the reading of all good books is like a conversation with the finest [people] of past centuries.” if that’s true, then who are some of [&# ;] giacomo cignoni: my internship at the internet archive this summer, open library and the internet archive took part in google summer of code (gsoc), a google initiative to help students gain coding experience by contributing to open source projects. i was lucky enough to mentor giacomo while he worked on improving our bookreader experience and infrastructure. we have invited giacomo to write a [&# ;] google summer of code : adoption by book lovers by tabish shaikh &# ; mek openlibrary.org,the world’s best-kept library secret: let’s make it easier for book lovers to discover and get started with open library. hi, my name is tabish shaikh and this summer i participated in the google summer of code program with open library to develop improvements which will help book lovers discover [&# ;] open library for language learners by guyrandy jean-gilles - - a quick browse through the app store and aspiring language learners will find themselves swimming in useful programs. but for experienced linguaphiles, the never-ending challenge is finding enough raw content and media to consume in their adopted tongue. open library can help. earlier this year, open library added reading levels to [&# ;] meet the librarians of open library by lisa seaberg are you a book lover looking to contribute to a warm, inclusive library community? we’d love to work with you: learn more about volunteering @ open library behind the scenes of open library is a whole team of developers, data scientists, outreach experts, and librarians working together to make open library better [&# ;] re-thinking open library’s book pages by mek karpeles, tabish shaikh we&# ;ve redesigned our book pages: before →after. please share your feedback with us. a web page for every book&# ; this is the mission of open library: a free, inclusive, online digital library catalog which helps readers find information about any book ever published. millions of books in open library&# ;s catalog [&# ;] reading logs: going public & helping book lovers share hi book lovers, starting - - , reading logs for new open library accounts will be public by default. readers may go here to view or manage their reading log privacy preferences. this will not affect the privacy of your reading history &# ; only books which you explicitly mark as want to read, currently reading, or already [&# ;] about | code lib about chat conference jobs journal local mailing list planet wiki about code lib isn’t entirely about code or libraries. it is a volunteer-driven collective of hackers, designers, architects, curators, catalogers, artists and instigators from around the world, who largely work for and with libraries, archives and museums on technology “stuff.” it started in the fall of as a mailing list when a group of library programmers decided to create an overarching community agnostic towards any particular language or technology. code lib is dedicated to providing a harassment-free community experience for everyone regardless of gender, sexual orientation, disability, physical appearance, body size, race, or religion. for more information, please see our emerging codeofconduct lib. code lib grew out of other efforts such as the access conference, web lib, perl lib, /usr/lib/info ( - , see archive.org) and oss lib which allow technology folks in libraries, archives and museums to informally share approaches, techniques, and code across institutional and project divides. soon after the mailing list was created, the community decided to setup a #code lib irc channel (chat room) on freenode. the first face-to-face meeting was held in in chicago, illinois, usa and the now-annual conference started in in corvallis, oregon, usa, and has continued since. local meetings have also sprung up from time to time and are encouraged. a volunteer effort manages an edited online journal that publishes relevant articles from the field in a timely fashion. we existed for a number of years before deciding to develop a logo. things get done because people share ideas, step up to lead, and work together, not because anyone is in charge. we prefer to make community decisions by holding open votes, e.g. on who gets to present at our conferences, where to host them, etc. if you’ve got an idea or an itch to scratch, please join in; we welcome your participation! if you are interested in joining the community: sign up to the discussion list; join the facebook or linkedin groups; google+ community; follow us on twitter; subscribe to our blogs; or get right to the heart of it in the chat room on irc. code lib code lib code lib code lib.social code lib code lib we are developers and technologists for libraries, museums, and archives who are dedicated to being a diverse and inclusive community, seeking to share ideas and build collaboration. in the library with the lead pipe – an open access, peer reviewed journal skip to main content chat .webcam open menu home about awards & good words contact editorial board denisse solis ian beilin jaena rae cabrera kellee warren nicole cooke ryan randall emeritus announcements authors archives conduct submission guidelines lead pipe publication process style guide search home about awards & good words contact editorial board denisse solis ian beilin jaena rae cabrera kellee warren nicole cooke ryan randall emeritus announcements authors archives conduct submission guidelines lead pipe publication process style guide search dec craig arthur, freddy paige, la' portia perkins, jasmine weiss and michael williams / comments culturally responsive community engagement programming and the university library: lessons learned from half a decade of vtditc by craig e. arthur, dr. freddy paige, la’ portia perkins, jasmine weiss, and dr. michael williams (good homie signs’ “hip hop @ vt” mural / ) in brief vtditc: hip hop studies at virginia tech is an award-winning series of experiential learning-focused, culturally responsive community engagement programs. it is deeply rooted in hip hop culture and... read more oct jeremiah paschke-wood, ellen dubinsky and leslie sult / comments creating a student-centered alternative to research guides: developing the infrastructure to support novice learners in brief: research and course guides typically feature long lists of resources without the contextual or instructional framework to direct novice researchers through the research process. an investigation of guide usage and user interactions at a large university in the southwestern u.s. revealed a need to reexamine the way research guides can be developed and... read more sep danya leebaw and alexis logsdon / comment power and status (and lack thereof) in academe: academic freedom and academic librarians in brief academic librarians do not experience full academic freedom protections, despite the fact that they are expected to exercise independent judgment, be civically engaged, and practice applied scholarship. academic freedom for academic librarians is not widely studied or well understood. to learn more, we conducted a survey which received over responses from academic... read more sep jennie rose halperin / comment the library commons: an imagination and an invocation by jennie rose halperin in brief commons theory can provide important interventions within neoliberal managerial information capitalism when applied to the library as an institution. the commons and its associated practices provide a model of abundance, sharing, and cooperation. libraries can and should participate in alternative economic and management models to create an inclusive vision... read more aug dave ellenwood / comment “information has value”: the political economy of information capitalism in brief information capitalism dominates the production and flow of information across the globe. it produces massive information institutions that are as harmful to everyday people as they are powerful. to this point, information literacy (il) educators do not have a theory and pedagogy of information capitalism. this article appraises the current state of political... read more jul liz vine / comments training matters: student employment and learning in academic libraries in brief conceiving of student employment in academic libraries as an educationally purposeful experience requires adopting a learner-centered pedagogical approach to student employee job training. adopting such an approach is triply beneficial: it makes that job training more effective; it identifies training as an opportunity to pursue learning goals that support the growth of students... read more jun natalia fernández and beth filar williams / comment creating a library wide culture and environment to support mlis students of color: the diversity scholars program at oregon state university libraries in brief the work of social justice, equity, and inclusion is not a short-term investment by a limited number of people; instead, it should be a part of every library’s and librarian’s work. at the oregon state university libraries (osul), we felt that in order to create a program dedicated to employing mlis students of... read more jun nicola andrews / comments it’s not imposter syndrome: resisting self-doubt as normal for library workers in brief library workers, as with other professions, are quick to diagnose ourselves and others with imposter syndrome when we doubt or devalue our everyday work. however, methods of coping with imposter syndrome have changed little in the forty years since the term was first theorized, and often centre on feel-good fixes which do not... read more apr ean henninger / comments multilingualism, neoliberalism, and language ideologies in libraries in briefthis article calls for a more holistic and inclusive approach to the under-examined issue of language in libraries. it begins by foregrounding language as a category of difference and arguing for its consideration in discussions of access, equity, diversity, and inclusion. by drawing on literature from applied linguistics and library and information studies, it... read more apr frederick carey / comments communicating with information: creating inclusive learning environments for students with asd in brief the focus of this article is twofold: it ) considers how digital humanities techniques and methodologies increase accessibility and scholarship opportunities for students with autism spectrum disorder; and ) outlines how libraries can collaborate with existing services to provide subsequently appropriate supports for students. autism spectrum disorder (asd), one of the increasingly prevalent... read more … next › this work is licensed under a cc attribution . license. issn - about this journal | archives | submissions | conduct biology library assignment biology library assignment this interactive tutorial worksheet will help you get familiar with library resources, learn about peer review and why it is so important to all of us, and find peer-reviewed sources through the library's online databases. what is your name? your answer watch the following video about the pcc library carefully as you will need to answer a question about it below. name three things you learned about the library from this video. your answer biology research guide as you saw in the video, the librarians have created research guides to support student research in every subject. from the library website http://www.pcc.edu/library find the research guides link under shortcuts. then find the biology research guide under sciences. look around at whatâ€™s available in the guide. what are three useful things you can find in the biology research guide? your answer understanding peer review in this class you are going to be asked to find peer-reviewed journal articles (also sometimes called scholarly or academic articles) to support your research. first, you need to know what those are and how theyâ€™re different from other articles. now you are going to take a look at two articles (only one of which is peer-reviewed) and pick out some of the characteristics of peer-reviewed articles. open the following two links in another browser tab: . https://bit.ly/bi article . https://bit.ly/bi articles (this one may ask you to login first with your mypcc username and password) now, skim the articles (you don't have to read them!) to find things you notice are different about them. this could include things about the purpose of the article, the evidence, how it's structured, how it looks, how it's written, etc. for example: at the end of the first article, i can see that the author is a seattle-based science writer specializing in biology and the environment. in the second article, under the authors' names, i see that they both work at the department of food and agricultural marketing, faculty of organic agricultural sciences, university of kassel, germany, so they are university professors. so the expertise of the authors in the second article is definitely higher than the authors in the first. what are three differences you noticed between the two articles? please be as specific as possible, just like in the example provided above. your answer now, watch this quick three-minute video on peer-review and be ready to name three characteristics of peer-reviewed articles. what are three characteristics of peer-reviewed articles? your answer why is peer-review important? why should you care about peer-review as a college student? peer-reviewed science is behind the designs of things we use in our everyday life and the technologies and health innovations that keep us safe and healthy. peer review is the process that helps to assure the quality of the scientific information on which we base many important decisions. for example, we have all heard a lot of information about the coronavirus, but some of it is not true. for example, i heard that drinking water every minutes will keep you from getting sick with the coronavirus which is not actually supported by any scientific research. we want our doctors to make the best decisions if we or our loved ones get infected with covid- and it is through the process of peer review (where the research has first been examined by experts) that doctors will learn more definitively how the coronavirus can best be treated. check out this two-minute video about why scientific research matters and the following -second video on why peer-review matters for non-experts like us. in your own words, why do you think peer review is important? your answer finding peer-reviewed articles through the library databases now, you're going to find some peer-reviewed articles yourself through two library databases. if you already have a topic selected for a research project in your class, you can use that in your searching. if not, you can use one of the following topics: *does caffeine affect memory or attention? *what is more effective, compost or artificial fertilizer? *how does sleep deprivation impact memory? *does physical exercise affect memory or attention? *are genetically-modified crops safe? *is in-vitro meat a sustainable alternative to traditional meat production? *is hydroponic growing the answer to our dwindling water supply? *will getting people to embrace a vegetarian diet decrease our reliance on fossil fuels? when you search, be sure to only use the fewest, most important keywords as the more keywords you enter, the fewer results you'll get. for example, if i was looking for articles about how sleep deprivation impacts memory, i would just put in the terms sleep deprivation memory. or i might even just try sleep memory. if i was looking at the sustainability of in-vitro meat as an alternative to traditional meat production, i would search using the terms in-vitro meat sustainability. or in-vitro meat alternative. or in-vitro meat sustainable. what topic are you going to explore for this activity? your answer using the library's article search the libraryâ€™s article search on the front page of the library website http://www.pcc.edu/library searches our two largest multidisciplinary library databases, academic search premier and masterfile premier. they contain not just scholarly articles, but also magazine and newspaper articles written for a non-expert audience. before you search, be sure to click on the articles tab, or you'll find yourself searching for books and videos.to limit your search to just peer-reviewed articles, you will want to check the box for peer reviewed on the front page (shown below). watch this four-minute video on finding articles through the library to see how our library article search works and how to search it. now you're ready to start searching on your own in the library's article search. when you search, be sure to check the box that limits your search to scholarly peer-reviewed articles. try different combinations of keywords and see how they change your results list. when you find articles that look useful, email them to yourself so you'll have the article as well as the formatted citation without having to go back into the database again. keywords you used that were successful: your answer find at least one article that looks useful for your research project. click on its title and then click on the cite icon on the right-hand side of the page. copy the citation in apa format and paste it in here. your answer using the biological sciences database the biological sciences database is our only database in which every article is from a biology journal or conference. when you search, be sure to check the box for peer reviewed, so you'll only see peer reviewed articles. you can access the database from the front page of the biology research guide or on the library databases page https://www.pcc.edu/library/databases/ once you find an article that looks useful, click on the title. on that page, you'll see an abstract that describes what the article is about. if it looks useful, you can email the article to yourself along with the with citation. there is also a cite button which will provide the citation in a variety of formats, including apa and cse. now you're ready to start searching on your own in the biological sciences database (which you'll find either on the front page of the biology research guide or on the library databases page https://www.pcc.edu/library/databases/. when you search, be sure to check the box that limits your search to scholarly peer-reviewed articles. try different combinations of keywords and see how they change your results list. when you find articles that look useful, email them to yourself so you'll have the article as well as the formatted citation without having to go back into the database again. keywords you used that were successful: your answer find at least one article that looks useful for your research project. click on its title and then click on the cite icon on the upper right-hand side of the page. copy the citation in apa th edition or council of science educators cse th name-year format (whatever your instructor requires) and paste it in here. your answer what is something useful you learned from this activity? your answer did you encounter any difficulties finding sources or do you have questions about doing research for this class? please describe them here. your answer when you submit this form, you will receive a copy of your responses via email and will also have the option to edit your responses. submit page of never submit passwords through google forms. this form was created inside of portland community college. report abuse forms finding sources for your reading or writing assignment finding sources for your reading or writing assignment this interactive tutorial worksheet will help you find quality sources for your assignment. you will learn how to narrow your topic focus, build a search strategy, and then will get experience finding sources. what is your name? your answer . describe what you're researching. what are you trying to find information about? your answer is your topic researchable? sometimes the research topic or question you originally come up with needs to be changed a bit to be researchable. take a look at this list and consider if any of these might be an issue with your topic or research question. . your focus is too general/broad this is the most common problem students have. they are interested in big issues like climate change, the impact of technology on people's lives, racism, mental illness, or homelessness. but those topics are way too large for a college-level research assignment, so you need to focus on a more specific aspect of it. . your focus is too specific/narrow maybe you want to look at the issue of affordable housing in your specific neighborhood in portland, but there might not be enough information out there to support it. you often don't need to entirely give up your topic, just zoom out your focus a bit. . you will not find a diversity of viewpoints on your topic good research topics are ones where there isn't just a simple answer. for example, maybe you want to research whether children need love. the answer to that is obviously yes, they do and you're not going to find any source that suggests children don't need love. also, love is not something you can easily measure. you might be able to tweak your search to perhaps focus on the impact of secure (or insecure) childhood attachment on mental health. . there is not enough information about your topic you might be interested in a very new issue or a topic that people simply aren't studying and writing about yet. if you need to find sources for your project, you can't choose something that no one is writing about. let's test your topic! test your current topic either in google news https://news.google.com/ or google scholar https://scholar.google.com/ (or both). try searching for your topic in one or both of the search engines. look at the first page or two of results. do they look relevant to what you're looking for? if not, or if very few look relevant, i'll share some strategies on the next page that will help you make your topic more researchable. here are some examples: screenshot from google news for "children need love" where results are not relevant screenshot from google news for "children secure attachment mental health" where some results are relevant screenshot from google scholar for "children secure attachment mental health" where some results are relevant . when you searched in google scholar or google news for your topic, did the sources on the first page or two of results look relevant? yes, plenty looked relevant yes, but very few looked relevant no clear selection next page of never submit passwords through google forms. this form was created inside of portland community college. report abuse forms everything you need to know about authenticode code signing | microsoft docs skip to main content contents exit focus mode bookmark share twitter linkedin facebook email table of contents everything you need to know about authenticode code signing / / minutes to read in this article in today’s post, i’ll be discussing the use of authenticode to sign software programs; this post will be of interest primarily to software developers. large software companies (like microsoft) often have an entire team dedicated to the code-signing and release process, but even (especially) small software publishers should sign their code. in this post, i’ll explain why, and tell you everything you need to do to achieve the benefits of shipping signed code. why sign your code? as a software publisher, there are two reasons to sign your code: to prove its integrity to develop its reputation the first point is simple: digitally signing your code helps to ensure that it cannot be tampered with, either on your servers, or when it is being downloaded to a user’s computer, especially over an insecure protocol like http or ftp. this provides the user the confidence of knowing that the program they’ve just downloaded is the same one that you built. of course, that integrity alone doesn’t mean a lot. malicious files can also be signed, and a signature from, say, bad-guys-r-us doesn’t mean that the code is safe to run. that’s where reputation comes in. historically, major software publishers like microsoft, adobe, apple, ibm, etc, have worked hard to develop a good reputation in the minds of users. a user who downloads an application signed by microsoft is usually more likely to trust and install that program than if the program were unsigned, or signed by some company they’ve never heard of. one problem with this scheme is that it doesn’t work too well except for the biggest of publishers—if i’ve never heard of just great software, should i trust a package that bears their signature? internet explorer ’s smartscreen application reputation feature helps level the playing field. this feature uses a variety of signals to evaluate the reputation of a given download, including the download history and popularity, anti-virus results, reputation of the site it has been delivered from, and more. as a small software publisher, the best way to accumulate your good reputation and allow it to benefit all of your software is to digitally sign your code. signing your code allows the smartscreen application reputation service to recognize a program’s origins, and allow that origin information to influence the reputation of the program. small publishers benefit the most from this. for instance, while most users have never heard of me, my freeware programs are digitally signed by my certificate, and the clean reputation for my certificate means that smartscreen application reputation can identify them as non-malicious. when smartscreen recognizes non-malicious code, the user benefits from a more streamlined trust experience, and fewer security prompts are shown when downloading and running the program. in contrast, unsigned and unknown programs are treated with suspicion and show more security warnings; our data indicates that - % of such programs are eventually determined to be malicious. of course, if i were to ever abuse my good reputation to sign and release malicious code, the smartscreen system can easily revoke my good reputation—my certificate’s signature would turn from a badge of honor into a flashing signal of untrustworthy software. hopefully, i’ve convinced you that signing your code is the right way to go. now, on to the “how-to” portion of this post. step : acquire a software publisher’s digital certificate the first step is to get a software publisher’s certificate. internally, these certificates are slightly different than a https server certificate (they bear different usage flags), and often cost a bit more because the certificate authority will perform more due diligence in verifying your identity. certificates are issued by certificate authorities (cas), firms whose job it is to verify the identity of certificate requestors and issue cryptographically-protected certificates that map the private key (which only you know) to a public key (which is contained in the certificate itself). i purchased my personal software publisher’s certificate from comodo via tucows, a reseller. their current prices are $ for a certificate valid for one year, $ for two years, or $ for three years. generally speaking, it’s a good idea to buy a certificate with the longest possible validity period because renewing certificates is not a fun experience, and it also helps to mitigate the problem of certificate rollover. certificate rollover occurs when your old certificate expires and you begin signing your code with a new replacement certificate; all of your reputation was accumulated against the old certificate, and hence there may be a time lag for your new certificate to acquire a good reputation. the process of acquiring a certificate typically takes a few days, and for non-incorporated software publishers like me, requires a fair amount of personal information. for instance, the validation process required that i fax a copy of my drivers license and a few utility bills to the certificate authority, so there was reasonable proof of my identity. i also validated my phone number and spoke to a representative. if i were to ever “go rogue” and start using my certificate for evil, i’m sure the terms of service say that this information would be provided to law enforcement so they could come ask me some questions. after the verification process was completed, i was given the opportunity to download my new certificate. when downloading your new certificate, you should follow the certificate authorities instructions exactly as there are some mistakes that can be very difficult to recover from. in particular, choose a memorable but strong password to protect your private key file. after you have your certificate, be very sure to protect the private key file—if you lose control of your private key, a bad guy could sign code using your certificate. if your certificate is used maliciously, bad reputation will taint everything you’ve ever signed! note: yes, you can also generate a self-signed certificate, but this is only useful if your software will never be used outside of your organization. inside your organization, you could use group policy to deploy your self-signed certificate to all clients so that they trust it. however, any pc that isn’t configured to trust your certificate will block your software as bearing an invalid signature. unless your company already has an internal pki (public key infrastructure) deployment, you’re better off getting a publisher certificate that chains to one of the trusted root cas. note: in august , microsoft announced support for a new type of code-signing certificate, the extended validation (ev) certificate. these certificates tend to be more expensive and harder-to-use (requiring security token hardware) but have the benefit of providing faster accumulation of reputation for smartscreen. step : sign your code after you have a certificate, you can begin signing your installation executables or msi install packages. because you’ll want to do this for every build that you release, automating as much of the signing process as possible using a script or batch file is highly recommended. note: the .net framework has a different type of code-signing called “strong naming”. a strong name is not recognized by anything in windows except the .net framework, and hence i won’t discuss it further here. shawnfa wrote a good article explaining strong names and authenticode. to sign your program using authenticode, you can use the signcode.exe utility which shipped with the microsoft .net framework version . : signcode -spc \src\mycert.spc -v \src\mykey.pvk -n "fiddler web debugger (beta)" -i "http://www.fiddler .com/fiddler /" -a sha -t http://timestamp.comodoca.com/authenticode fiddler betasetup.exe you’ll be prompted to enter the password for your pvk file, and after doing so, your program will be signed and time-stamped. the –n parameter specifies the string that will be shown in the “name” field identifying the software. the –i parameter specifies the url that will be launched if the user clicks on the name hyperlink. the –a parameter specifies which hash algorithm should be used—this should always be sha because older hash algorithms are less secure, and will eventually be retired. internet explorer on windows sp , for instance, disables md /md hashes and treats signatures using those algorithms as invalid. the –t parameter specifies the url of the time-stamping server, discussed later. these inputs result in a signature that is displayed by various trust prompts like so: alternatively, you can use the signtool.exe shipped in the windows sdk and later versions of the .net framework. to use this tool, you’ll need to convert your .spc & .pvk files into a new pfx file, which you can do from a visual studio command prompt: pvk pfx -spc mycert.spc -pvk mykey.pvk -pfx mycert.pfx -pi pvkpassword -po newpfxpassword after that, you can use a script like the following to sign your package: set /p pfxpass=enter pfx password: signtool sign /p %pfxpass% /f c:\src\mycert.pfx /d "fiddlercap" /du "http://www.fiddlercap.com/" /t http://timestamp.comodoca.com/authenticode /v fiddlercap-en.msi set pfxpass=blank easy, huh? best practice: sign both the installer and the application when downloading installation packages from the internet, the browser and/or windows shell will only check the digital signature on the installer itself. however, as a best practice, you should also sign the main executable of your application as well. this helps prevent tampering, but more importantly, it helps ensure that any security tools on the user’s machine can more readily identify your code. for instance, many anti-malware or firewall tools will treat unsigned executables with suspicion, and may even block them if they happen to share the same filename as a malicious program. for instance, one of my utilities, slickrun shared the same filename (sr.exe) as a malicious program, and some anti-malware tools were too primitive to recognize the false positive. however, after i signed sr.exe directly, this problem went away. similarly, some firewalls assign exceptions to programs based on their digital signature or file hash. if the user approves an exception to allow your program through the local firewall, that exception may be deleted if your program is later updated and the file’s hash changed. signing your code can help prove to the firewall that the new version of your program should inherit the exemption granted to the older version. note: typically, windows does not itself check the digital signature when running a locally-installed version of your program; it only checks the signature when the program bears a mark-of-the-web indicating that it was downloaded from the internet or extracted from an archive downloaded from the internet. however, executables written in .net can be an exception to this. the .net framework has the ability to assign security permissions to code based on its signature, called “publisher evidence.” doing so necessitates that the signature be verified, and verifying the signature may require an expensive network request to check the certificate for revocation. if you are not using the “publisher evidence” feature of .net, you can modify your application’s manifest to indicate that .net should not check the signature. yourapp.exe.config: best practice: time-stamping when signing your code, you have the opportunity to timestamp your code; you should definitely do this. time-stamping adds a cryptographically-verifiable timestamp to your signature, proving when the code was signed. if you do not timestamp your code, the signature will be treated as invalid upon the expiration of your digital certificate. since it would probably be cumbersome to re-sign every package you’ve shipped when your certificate expires, you should take advantage of time-stamping. a signed, time-stamped package remains valid indefinitely, so long as the timestamp marks the package as having been signed during the validity period of the certificate. thanks for helping ensure a secure and streamlined experience for windows users! -eric update: not all publisher certificates are enabled to permit timestamping to provide indefinite lifetime. if the publisher’s signing certificate contains the lifetime signer oid (oid_kp_lifetime_signing . . . . . . . . . ), the signature becomes invalid when the publisher’s signing certificate expires, even if the signature is timestamped. this is to free a certificate authority from the burden of maintaining revocation lists (crl, ocsp) in perpetuity. update: it also appears that some cas do not provide spc and pvk files, instead providing the certificate in a different format. you can convert from a .cer file to a spc file using cert spc.exe in the windows sdk. if you have a pem file instead of a .pvk file, you can convert that using openssl's pvk command: pvk -in pem_key_file -topvk -out pvk_file. http://help.godaddy.com/article/ has a step-by-step walkthrough. curious about where the authenticode signature actually goes into the binary? see this whitepaper. update: some developers use tricks to sneak unauthenticated data inside authenticode-signed binaries. you shouldn't do that. theme light dark high contrast previous version docs blog contribute privacy & cookies terms of use site feedback trademarks © microsoft in this article theme light dark high contrast previous version docs blog contribute privacy & cookies terms of use site feedback trademarks © microsoft cryptocurrency prices, charts and market capitalizations | coinmarketcap cryptocurrencies: , markets: , market cap: $ , , , , h vol: $ , , , btc dominance: . % english usd cryptocurrencies exchanges portfolio watchlist products learn buy bitcoin search / cryptocurrencies ranking global charts spotlight gainers & losers recently added historical snapshots exchanges spot derivatives dex where to buy portfolio watchlist products converter mobile apps blockchain explorer interest jobs board crypto api site widgets learn alexandria earn glossary newsletter events calendar methodology buy bitcoin announcements english usd disclaimerrequest formterms of use privacy policyabout market cap: $ , , , , h vol: $ , , , btc dominance: . %cryptocurrencies: , markets: , today's dogecoin (doge) price the price of dogecoin (doge) is $ . , a . % increase over the last day. read more the total crypto market volume over the last hours is $ . b, which makes a . % decrease. the total volume in defi is currently $ . b, . % of the total crypto market -hour volume. the volume of all stable coins is now $ . b, which is . % of the total crypto market -hour volume. bitcoin's price is currently $ , . . bitcoin’s dominance is currently . %, a decrease of . % over the day. elon musk elon musk speaks out on bitcoin and dogecoin nfts collect all six cmc heroes as nfts! portfolio 🔥 track your trades in one place, not all over the place take a quiz! earn $grt & $ inch alexandria - deep tech dives a dive into ethereum . watchlist cryptocurrenciesderivativesdefistorage yield farming show rows filters # name price h d market cap volume circulating supply last days wallstreetbets wsb $ . . % . % $ , , , $ , , , wsb , , wsb bitcoin btc $ , . . % . % $ , , , $ , , , , , btc , , btc ethereum eth $ , . . % . % $ , , , $ , , , , , eth , , eth tether usdt $ . . % . % $ , , , $ , , , , , , usdt , , , usdt cardano ada $ . . % . % $ , , , $ , , , , , , ada , , , ada xrp xrp $ . . % . % $ , , , $ , , , , , , xrp , , , xrp polkadot dot $ . . % . % $ , , , $ , , , , , dot , , dot binance coin bnb $ . . % . % $ , , , $ , , , , , bnb , , bnb litecoin ltc $ . . % . % $ , , , $ , , , , , ltc , , ltc chainlink link $ . . % . % $ , , , $ , , , , , link , , link crypto.com defi wallet - best place to manage, swap & farm defi tokens securely manage, swap & farm defi tokens, including uni, comp, yfi, cro, aave. boost your yields by up to x when you stake cro! get app nowsponsored stellarxlm $ . . % . % bitcoin cashbch $ . - . % . % dogecoindoge $ . . % . % usd coinusdc $ . - . % - . % uniswapuni $ . - . % - . % aaveaave $ . - . % . % wrapped bitcoinwbtc $ . - . % . % bitcoin svbsv $ . - . % . % eoseos $ . . % . % cosmosatom $ . . % . % moneroxmr $ . - . % . % trontrx $ . . % . % nemxem $ . . % . % tezosxtz $ . . % . % makermkr $ . - . % . % thetatheta $ . - . % . % synthetixsnx $ . - . % . % elrondegld $ . . % . % compoundcomp $ . - . % . % vechainvet $ . - . % . % daidai $ . - . % . % solanasol $ . . % . % neoneo $ . - . % . % huobi tokenht $ . . % . % avalancheavax $ . . % . % umauma $ . - . % . % sushiswapsushi $ . - . % . % binance usdbusd $ . - . % - . % iotamiota $ . - . % . % crypto.com coincro $ . - . % . % unus sed leoleo $ . - . % . % terraluna $ . . % . % ftx tokenftt $ . - . % . % filecoinfil $ . - . % . % celsiuscel $ . - . % . % dashdash $ . - . % . % yearn.financeyfi $ . - . % - . % the graphgrt $ . . % . % algorandalgo $ . . % . % zcashzec $ . - . % . % decreddcr $ . - . % . % revainrev $ . - . % . % xzrx $ . - . % . % ethereum classicetc $ . - . % . % kusamaksm $ . - . % . % near protocolnear $ . . % . % waveswaves $ . . % . % zilliqazil $ . - . % . % nexonexo $ . . % . % swissborgchsb $ . . % . % loopringlrc $ . - . % . % renren $ . - . % . % celocelo $ . . % . % hedera hashgraphhbar $ . - . % . % renbtcrenbtc $ . - . % . % omg networkomg $ . - . % . % curve dao tokencrv $ . - . % . % inch inch $ . - . % - . % basic attention tokenbat $ . . % . % ontologyont $ . - . % . % thorchainrune $ . - . % - . % voyager tokenvgx $ . - . % - . % digibytedgb $ . - . % . % husdhusd $ . - . % - . % bittorrentbtt $ . . % . % nanonano $ . - . % - . % iconicx $ . . % . % hedgetradehedg $ . - . % - . % quantqnt $ . . % . % qtumqtum $ . . % . % okbokb $ . - . % . % trueusdtusd $ . - . % - . % siacoinsc $ . - . % . % horizenzen $ . - . % - . % alpha finance labalpha $ . - . % . % pancakeswapcake $ . . % . % decentralandmana $ . . % . % reserve rightsrsr $ . - . % . % kyber networkknc $ . - . % . % stacksstx $ . - . % . % fantomftm $ . - . % . % ampleforthampl $ . - . % . % enjin coinenj $ . - . % . % iostiost $ . - . % . % ocean protocolocean $ . - . % . % bancorbnt $ . - . % . % matic networkmatic $ . . % . % energy web tokenewt $ . . % . % funfairfun $ . . % . % vergexvg $ . - . % . % bitcoin bep btcb $ . - . % . % showing - out of show rows find out how we work by clicking here. read more today’s cryptocurrency prices, charts and data welcome to coinmarketcap.com! this site was founded in may by brandon chez to provide up-to-date cryptocurrency prices, charts and data about the emerging cryptocurrency markets. since then, the world of blockchain and cryptocurrency has grown exponentially and we are very proud to have grown with it. we take our data very seriously and we do not change our data to fit any narrative: we stand for accurately, timely and unbiased information. all your crypto market data needs in one place here at coinmarketcap, we work very hard to ensure that all the relevant and up-to-date information about cryptocurrencies, coins and tokens can be located in one easily discoverable place. from the very first day, the goal was for the site to be the number one location online for crypto market data, and we work hard to empower our users with our unbiased and accurate information. we provide live and historic crypto charts for free each of our coin data pages has a graph that shows both the current and historic price information for the coin or token. normally, the graph starts at the launch of the asset, but it is possible to select specific to and from dates to customize the chart to your own needs. these charts and their information are free to visitors of our website. how do we calculate our cryptocurrency prices? we receive updated cryptocurrency prices directly from many exchanges based on their pairs. we then convert the number to usd. a full explanation can be found here. related links new to crypto? learn how to buy bitcoin today. ready to learn more? visit our crypto glossary and learning hub. want to look up a transaction? visit our blockchain explorer. how do we calculate our crypto valuations? we calculate our valuations based on the total circulating supply of an asset multiplied by the currency reference price. the topic is explained in more detail here. how do we calculate the cryptocurrency market cap? we calculate the total cryptocurrency market capitalization as the sum of all cryptocurrencies listed on the site. does coinmarketcap.com list all cryptocurrencies? no, we do not list all cryptocurrencies at coinmarketcap. as a company and team, we are very aware that not all coins and projects have good intentions. while we cannot guarantee to exclude them all, we have a vetting process that each coin goes through before it is listed on the site. if we suspect that a coin or project is a scam, it does not get listed. how big is the global coin market? at the time of writing, we estimate that there are around , coins, tokens and projects in the global coin market. as mentioned above, we have a due diligence process that we apply to new coins before they are listed. this process controls how many of the cryptocurrencies from the global market are represented on our site. what is an altcoin? the very first cryptocurrency was bitcoin. since it is open source, it is possible for other people to use the majority of the code, make a few changes and then launch their own separate currency. many people have done exactly this. some of these coins are very similar to bitcoin, with just one or two amended features (such as litecoin), while others are very different, with varying models of security, issuance and governance. however, they all share the same moniker — every coin issued after bitcoin is considered to be an altcoin. what is an ico? ico stands for initial coin offering. many of the smaller projects in the crypto space — and a few of the largest ones — raised money from private investors around the world in the crypto equivalent of a crowdfunding campaign. investors would send funds — usually in the form of bitcoin — to the project and receive coin or tokens in return. most icos happened in and early and used ethereum as a platform of operation via the erc- standard. in , the united states securities and exchange commission (sec) clarified their rules relating to fundraising for assets, which made it much harder for new cryptocurrency projects to issue their own tokens in this way. since the appearance of the sec guidance and the organization’s heightened interest in regulating icos for u.s. citizens, the number of icos has been reduced substantially. what is a stablecoin? price volatility has long been one of the features of the cryptocurrency market. when asset prices move quickly in either direction and the market itself is relatively thin, it can sometimes be difficult to conduct transactions as might be needed. to overcome this problem, a new type of cryptocurrency tied in value to existing currencies — ranging from the u.s. dollar, other fiats or even other cryptocurrencies — arose. these new cryptocurrency are known as stablecoins, and they can be used for a multitude of purposes due to their stability. which is the best cryptocurrency to invest in? coinmarketcap does not offer financial or investment advice about which cryptocurrency, token or asset does or does not make a good investment, nor do we offer advice about the timing of purchases or sales. we are strictly a data company. please remember that the prices, yields and values of financial assets change. this means that any capital you may invest is at risk. we recommend seeking the advice of a professional investment advisor for guidance related to your personal circumstances. if you are investing in cryptocurrency — coinmarketcap.com is for you the data at coinmarketcap updates throughout the day, which means that it is possible to check in on the value of your investments and assets at any time and from anywhere in the world. we look forward to seeing you regularly! be the first to know about crypto news everyday get crypto analysis, news and updates right to your inbox! sign up here so you don't miss a single newsletter. subscribe now products blockchain explorer crypto api crypto indices interest jobs board sitemap company about us terms of use privacy policy disclaimer methodology careerswe’re hiring! support request form contact support faq glossary socials facebook twitter telegram instagram interactive chat © coinmarketcap. all rights reserved learning (lib)tech – stories from my life as a technologist skip to content learning (lib)tech stories from my life as a technologist menu about me about this blog contact me twitter github linkedin flickr rss choosing not to go into management (again) often, to move up and get a higher pay, you have to become a manager, but not everyone is suited to become a manager, and sometimes given the preference, it’s not what someone wants to do. thankfully at gitlab, in every engineering team including support, we have two tracks: technical (individual contributor), and management. continue reading “choosing not to go into management (again)” author cynthiaposted on february , february , categories work culturetags career growth, management, reflectionleave a comment on choosing not to go into management (again) prioritization in support: tickets, slack, issues, and more i mentioned in my gitlab reflection that prioritization has been quite different working in support compared to other previous work i’ve done. in most of my previous work, i’ve had to take “desk shifts” but those are discreet where you’re focused on providing customer service during that period of time and you can focus on other things the rest of the time. in support, we have to constantly balance all the different work that we have, especially in helping to ensure that tickets are responded to within the service level agreement (sla). it doesn’t always happen, but i ultimately try to reach inbox (with read-only items possibly left), and gitlab to-do by the end of the every week. people often ask me how i manage to do that, so hopefully this provides a bit of insight. continue reading “prioritization in support: tickets, slack, issues, and more” author cynthiaposted on december , december , categories methodologytags productivityleave a comment on prioritization in support: tickets, slack, issues, and more reflection part : my second year at gitlab and on becoming senior again this reflection is a direct continuation of part of my time at gitlab so far. if you haven’t, please read the first part before beginning this one. continue reading “reflection part : my second year at gitlab and on becoming senior again” author cynthiaposted on june , january , categories update, work culturetags gitlab, organizational culture, reflectionleave a comment on reflection part : my second year at gitlab and on becoming senior again reflection part : my first year at gitlab and becoming senior about a year ago, i wrote a reflection on summit and contribute, our all staff events, and later that year, wrote a series of posts on the gitlab values and culture from my own perspective. there is a lot that i mention in the blog post series and i’ll try not to repeat myself (too much), but i realize i never wrote a general reflection at year , so i’ve decided to write about both years now but split into parts. continue reading “reflection part : my first year at gitlab and becoming senior” author cynthiaposted on june , january , categories update, work culturetags gitlab, organizational culture, reflectionleave a comment on reflection part : my first year at gitlab and becoming senior is blog reading dead? there was a bit more context to the question, but a friend recently asked me: what you do think? is blogging dead? continue reading “is blog reading dead?” author cynthiaposted on may , may , categories updatetags reflectionleave a comment on is blog reading dead? working remotely at home as a remote worker during a pandemic i’m glad that i still have a job, that my life isn’t wholly impacted by the pandemic we’re in, but to say that nothing is different just because i was already a remote worker would be wrong. the effect the pandemic is having on everyone around you has affects your life. it seems obvious to me, but apparently that fact is lost on a lot of people. i’d expect that’s not the case for those who read my blog, but i thought it’d be worth reflecting on anyway. continue reading “working remotely at home as a remote worker during a pandemic” author cynthiaposted on may , may , categories work culturetags remoteleave a comment on working remotely at home as a remote worker during a pandemic code libbc lightning talk notes: day code libbc day lightning talk notes! continue reading “code libbc lightning talk notes: day ” author cynthiaposted on november , categories eventstags authentication, big data, c lbc, code, code lib, digital collections, privacy, reference, teachingleave a comment on code libbc lightning talk notes: day code libbc lightning talk notes: day code libbc day lightning talk notes! continue reading “code libbc lightning talk notes: day ” author cynthiaposted on november , categories eventstags c lbc, digital collections, intranet, marc, metadata, teachingleave a comment on code libbc lightning talk notes: day presentation: implementing values in practical ways this was presented at code libbc . continue reading “presentation: implementing values in practical ways” author cynthiaposted on november , november , categories events, work culturetags c lbc, organizational culture, presentation, valuesleave a comment on presentation: implementing values in practical ways implementing values: learning from gitlab: transparency this is the sixth value covered in a series of blog posts on what we can learn in implementing values that are the same or similar to gitlab’s credit values. for background and links to the other posts, please check out the overview post. continue reading “implementing values: learning from gitlab: transparency” author cynthiaposted on september , august , categories work culturetags gitlab, organizational culture, reflection, valuesleave a comment on implementing values: learning from gitlab: transparency posts navigation page page … page next page cynthia technologist, librarian, metadata and technical services expert, educator, mentor, web developer, uxer, accessibility advocate, documentarian view full profile → follow us twitter linkedin github telegram search for: search categories events librarianship library academic public special tours methodology project work technology tools update web design work culture follow via email enter your email address to receive notifications of new posts by email. email address: follow about me about this blog contact me twitter github linkedin flickr rss learning (lib)tech add your thoughts here... (optional) post to cancel none colgate university archives - documenting your experiences of covid- | colgate university libraries skip to main content my library account home about staff directory hours floor plans technology in the libraries borrowing rules after hours access policies about case & cooley libraries research course & subject guides course reserves zotero worldcat special collections & university archives digital commons connecting from off-campus classic catalog services library liaisons reserve a room library instruction digital scholarship suggest purchase faculty course reserves faculty research support copyright guide ask a librarian home about staff directory hours floor plans technology in the libraries borrowing rules after hours access policies about case & cooley libraries research course & subject guides course reserves zotero worldcat special collections & university archives digital commons connecting from off-campus classic catalog services library liaisons reserve a room library instruction digital scholarship suggest purchase faculty course reserves faculty research support copyright guide ask a librarian submit documenting your experiences of covid- - colgate university archives as members of the global community, we are living in a time of crisis and change, and we are all affected in some way. as members of the colgate university community, we have stories to share to help ourselves and future generations understand these historic moments, even as we look to the past to understand how the university has weathered other crises. the colgate university archives invites you to share your experiences related to the covid- pandemic in a form that is meaningful to you, so that your stories can be preserved. how to participate participation is open to anyone in the colgate community. each one of us will experience these moments differently, reflecting a diversity of physical locations and roles within, or relationships to, the university. we welcome documentation of your experiences and stories in a variety of forms, such as journal entries, artwork, poetry, social media posts, or audio or video recordings. here are some ideas about what you might capture in your documentation: - how have you been affected by the shift to remote learning/teaching/working? - what is your daily routine or experience during this time? how has it changed? - how are you affected by the relocation from campus spaces you would normally inhabit? what do you miss about those spaces, or how do you see them differently? - what are your frustrations and challenges during this time? - what happy, humorous, or grateful moments would you want to share? - how are you maintaining connections to friends, family, and others? - what are your hopes for the future? if you choose to include others in your documentation, such as a recorded conversation, please follow all relevant public health orders to ensure everyone’s safety. also, please respect the privacy and rights of others to not be included or named in your documentation. given the nature of the current public health crisis, please be aware of what personal health information you might be sharing about yourself, and please do not share personally identifiable health information about others. how to submit content - digital content can be donated through this google form. you may upload up to files at a time. - for physical materials, we ask that you let us know you intend to donate items using the same form, but please retain the items until we are able to accept them at a later date. we will contact you at that time. - if the content involves other individuals, such as an interviewee, co-creator, or creative or intellectual contributor, they must also fill out the above form so that the university archives has their permission to preserve and share the material. only one person needs to upload the content though. - facebook and twitter allow you to download your data. instagram currently does not have that function, but a screen capture of a post you created may also work. - if you have any questions, please email university archivist sarah keen. when you complete the form, you will be provided with important information about how your content can be shared with and used by others and whether access to your content should be delayed for years. you will also be asked to agree to a personal health information agreement. if you permit us to share your content upon submission, we may post it to our instagram and tumblr accounts as examples of ways that individuals can document their experiences. we hope to include most content in a future digital collection. support and safety this is a difficult time for most of us, and your health and well-being is important. while you are documenting your experiences, please follow all relevant public health guidelines and orders. the university website provides information and guidance on university services and covid- resources for students, faculty, and staff. acknowledgements thank you to unc charlotte special collections and university archives and the indiana university archives for inspiring and informing this project. twitter instagram federal depository library program oak drive hamilton, ny, ( ) - accessibility services guest user guide moodle portal © colgate university libraries | libraries staff login | colgate university | join us oak drive hamilton, ny, ( ) - cygnus solutions - wikipedia cygnus solutions from wikipedia, the free encyclopedia jump to navigation jump to search this article needs additional citations for verification. please help improve this article by adding citations to reliable sources. unsourced material may be challenged and removed. find sources: "cygnus solutions" – news · newspapers · books · scholar · jstor (september ) (learn how and when to remove this template message) cygnus solutions industry computer software fate merged with red hat successor red hat founded ; years ago ( ) defunct ; years ago ( ) key people john gilmore, michael tiemann, and david henkel-wallace products compilers, debuggers, development tools, ecos, cygwin cygnus solutions, originally cygnus support, was founded in by john gilmore, michael tiemann and david henkel-wallace to provide commercial support for free software. its tagline was: making free software affordable. for years, employees of cygnus solutions were the maintainers of several key gnu software products, including the gnu debugger and gnu binutils (which included the gnu assembler and linker). it was also a major contributor to the gcc project and drove the change in the project's management from having a single gatekeeper to having an independent committee. cygnus developed bfd, and used it to help port gnu to many architectures, in a number of cases working under non-disclosure to produce tools used for initial bringup of software for a new chip design. cygnus was also the original developer of cygwin, a posix layer and the gnu toolkit port to the microsoft windows operating system family, and of ecos, an embedded real-time operating system. in the documentary film revolution os, tiemann indicates that the name "cygnus" was chosen from among several names that incorporated the acronym "gnu". according to stan kelly-bootle, it was recursively defined as cygnus, your gnu support.[ ] on november , , cygnus solutions announced its merger with red hat, and it ceased to exist as a separate company in early .[ ] as of [update], a number of cygnus employees continue to work for red hat, including tiemann, who serves as red hat's vice president of open source affairs, and formerly served as cto. references[edit] ^ binstock, andrew ( - - ). "farewell, devil's advocate". dr. dobb's journal. archived from the original on - - . retrieved - - . (nb. the article refers to and cites from stan kelly-bootle's original article "faqs of life" in the "devil's advocate" column of unix review, july , where cygnus is recursively defined as "cygnus, your gnu support".) ^ "red hat to acquire cygnus and create global open source powerhouse". red hat. november , . retrieved - - . external links[edit] tiemann, michael (january ), "future of cygnus solutions — an entrepreneur's account", open sources: voices from the open source revolution. "marketing cygnus support", free software history, toad, september , . free and open-source software portal retrieved from "https://en.wikipedia.org/w/index.php?title=cygnus_solutions&oldid= " categories: free software companies red hat software companies disestablished in software companies established in hidden categories: articles needing additional references from september all articles needing additional references articles containing potentially dated statements from all articles containing potentially dated statements navigation menu personal tools not logged in talk contributions create account log in namespaces article talk variants views read edit view history more search navigation main page contents current events random article about wikipedia contact us donate contribute help learn to edit community portal recent changes upload file tools what links here related changes upload file special pages permanent link page information cite this page wikidata item print/export download as pdf printable version languages العربية deutsch español français italiano עברית 日本語 norsk bokmål polski português Русский svenska 中文 edit links this page was last edited on december , at : (utc). text is available under the creative commons attribution-sharealike license; additional terms may apply. by using this site, you agree to the terms of use and privacy policy. wikipedia® is a registered trademark of the wikimedia foundation, inc., a non-profit organization. privacy policy about wikipedia disclaimers contact wikipedia mobile view developers statistics cookie statement the thingology blog the librarything blog thingology monday, april th, new syndetics unbound feature: mark and boost electronic resources proquest and librarything have just introduced a major new feature to our catalog-enrichment suite, syndetics unbound, to meet the needs of libraries during the covid- crisis. our friends at proquest blogged about it briefly on the proquest blog. this blog post goes into greater detail about what we did, how we did it, and what efforts like this may mean for library catalogs in the future. what it does the feature, “mark and boost electronic resources,” turns syndetics unbound from a general catalog enrichment tool to one focused on your library’s electronic resources—the resources patrons can access during a library shutdown. we hope it encourages libraries to continue to promote their catalog, the library’s own and most complete collection repository, instead of sending patrons to a host of partial, third-party eresource platforms. the new feature marks the library’s electronic resources and “boosts,” or promotes, them in syndetics unbound’s discovery enhancements, such as “you may also like,” “other editions,” “tags” and “reading levels.” here’s a screenshot showing the feature in action. how it works the feature is composed of three settings. by default, they all turn on together, but they can be independently turned off and on. boost electronic resources chooses to show electronic editions of an item where they exist, and boosts such items within discovery elements. mark electronic resources with an “e” icon marks all electronic resources—ebooks, eaudio, and streaming video. add electronic resources message at top of page adds a customizable message to the top of the syndetics unbound area. “mark and boost electronic holdings” works across all enrichments. it is particularly important for “also available as” which lists all the other formats for a given title. enabling this feature sorts electronic resources to the front of the list. we also suggest that, for now, libraries may want to put “also available as” at the top of their enrichment order. why we did it your catalog is only as good as your holdings. faced with a world in which physical holdings are off-limits and electronic resources essential, many libraries have discouraged use of the catalog, which is dominated by non-digital resources, in favor of linking directly to overdrive, hoopla, freegal and so forth. unfortunately, these services are silos, containing only what you bought from that particular vendor. “mark and boost electronic resources” turns your catalog toward digital resources, while preserving what makes a catalog important—a single point of access to all library resources, not a vendor silo. maximizing your electronic holdings to make the best use of “mark and boost electronic resources,” we need to know about all your electronic resources. unfortunately, some systems separate marc holdings and electronic holdings; all resources appear in the catalog, but only some are available for export to syndetics unbound. other libraries send us holding files with everything, but they are unable to send us updates every time new electronic resources are added. to address this issue, we have therefore advanced a new feature—”auto-discover electronic holdings.” turn this on and we build up an accurate representation of your library’s electronic resource holdings, without requiring any effort on your part. adapting to change “mark and boost electronic resources” is our first feature change to address the current crisis. but we are eager to do others, and to adapt the feature over time, as the situation develops. we are eager to get feedback from librarians and patrons! — the proquest and librarything teams labels: new features, new product, syndetics unbound posted by tim @ : pm comments » share thursday, october th, introducing syndetics unbound short version today we’re going public with a new product for libraries, jointly developed by librarything and proquest. it’s called syndetics unbound, and it makes library catalogs better, with catalog enrichments that provide information about each item, and jumping-off points for exploring the catalog. to see it in action, check out the hartford public library in hartford, ct. here are some sample links: the raven boys by maggie stiefvater alexander hamilton by ron chernow faithful place by tana french we’ve also got a press release and a nifty marketing site. update: webinars every week! we’re now having weekly webinars, in which you can learn all about syndetics unbound, and ask us questions. visit proquest’s webex portal to see the schedule and sign up! long version the basic idea syndetics unbound aims to make patrons happier and increase circulation. it works by enhancing discovery within your opac, giving patrons useful information about books, movies, music, and video games, and helping them find other things they like. this means adding elements like cover images, summaries, recommendations, series, tags, and both professional and user reviews. in one sense, syndetics unbound combines products—the proquest product syndetics plus and the librarything products librarything for libraries and book display widgets. in a more important sense, however, it leaps forward from these products to something new, simple, and powerful. new elements were invented. static elements have become newly dynamic. buttons provide deep-dives into your library’s collection. and—we think—everything looks better than anything syndetics or librarything have done before! (that’s one of only two exclamation points in this blog post, so we mean it.) simplicity syndetics unbound is a complete and unified solution, not a menu of options spread across one or even multiple vendors. this simplicity starts with the design, which is made to look good out of the box, already configured for your opac and look. the installation requirements for syndetics unbound are minimal. if you already have syndetics plus or librarything for libraries, you’re all set. if you’ve never been a customer, you only need to add a line of html to your opac, and to upload your holdings. although it’s simple, we didn’t neglect options. libraries can reorder elements, or drop them entirely. we expect libraries will pick and choose, and evaluate elements according to patron needs, or feedback from our detailed usage stats. libraries can also tweak the look and feel with custom css stylesheets. and simplicity is cheap. to assemble a not-quite-equivalent bundle from proquest’s and librarything’s separate offerings would cost far more. we want everyone who has syndetics unbound to have it in its full glory. comprehensiveness and enrichments syndetics unbound enriches your catalog with some sixteen enrichments, but the number is less important than the options they encompass. these include both professional and user-generated content, information about the item you’re looking at, and jumping-off points to explore similar items. quick descriptions of the enrichments: boilterplate covers for items without covers. premium cover service. syndetics offers the most comprehensive cover database in existence for libraries—over million full-color cover images for books, videos, dvds, and cds, with thousands of new covers added every week. for syndetics unbound, we added boilerplate covers for items that don’t have a cover, which include the title, author, and media type. summaries. over million essential summaries and annotations, so patrons know what the book’s about. about the author. this section includes the author biography and a small shelf of other items by the author. the section is also adorned by a small author photo—a first in the catalog, although familiar elsewhere on the web. look inside. includes three previous syndetics enrichments—first chapters or excerpts, table of contents and large-size covers—newly presented as a “peek inside the book” feature. series. shows a book’s series, including reading order. if the library is missing part of the series, those covers are shown but grayed out. you may also like. provides sharp, on-the-spot readers advisory in your catalog, with the option to browse a larger world of suggestions, drawn from librarything members and big-data algorithms. in this and other enrichments, syndetics unbound only recommends items that your library owns. the syndetics unbound recommendations cover far more of your collection than any similar service. for example, statistics from the hartford public library show this feature on % of items viewed. professional reviews includes more than . million reviews from library journal, school library journal, new york times, the guardian, the horn book, booklist, bookseller + publisher magazine, choice, publisher’s weekly, and kirkus. a la carte review sources include voice of youth advocates: voya, doody’s medical reviews and quill and quire. reader reviews includes more than . million vetted, reader reviews from librarything members. it also allows patrons and librarians to add their own ratings and reviews, right in your catalog, and then showcase them on a library’s home page and social media. also available as helps patrons find other available formats and versions of a title in your collection, including paper, audio, ebook, and translations. exploring the tag system tags rethinks librarything’s celebrated tag clouds—redesigning them toward simplicity and consistency, and away from the “ransom note” look of most clouds. as data, tags are based on over million tags created by librarything members, and hand-vetted by our staff librarians for quality. a new exploration interface allows patrons to explore what librarything calls “tag mashes”—finding books by combinations of tags—in a simple faceted way. i’m going to be blogging about the redesign of tag clouds in the near future. considering dozens of designs, we decided on a clean break with the past. (i expect it will get some reactions.) book profile is a newly dynamic version of what bowker has done for years—analyzing thousands of new works of fiction, short-story collections, biographies, autobiographies, and memoirs annually. now every term is clickable, and patrons can search and browse over one million profiles. explore reading levels reading level is a newly dynamic way to see and explore other books in the same age and grade range. reading level also includes metametrics lexile® framework for reading. click the “more” button to get a new, super-powered reading-level explorer. this is one my favorite features! (second and last exclamation point.) awards highlights the awards a title has won, and helps patrons find highly-awarded books in your collection. includes biggies like the national book award and the booker prize, but also smaller awards like the bram stoker award and oklahoma’s sequoyah book award. browse shelf gives your patrons the context and serendipity of browsing a physical shelf, using your call numbers. includes a mini shelf-browser that sits on your detail pages, and a full-screen version, launched from the detail page. video and music adds summaries and other information for more than four million video and music titles including annotations, performers, track listings, release dates, genres, keywords, and themes. video games provides game descriptions, esrb ratings, star ratings, system requirements, and even screenshots. book display widgets. finally, syndetics unbound isn’t limited to the catalog, but includes the librarything product book display widgets—virtual book displays that go on your library’s homepage, blog, libguides, facebook, twitter, pinterest, or even in email newsletters. display widgets can be filled with preset content, such as popular titles, new titles, dvds, journals, series, awards, tags, and more. or you point them at a web page, rss feed, or list of isbns, upcs, or issns. if your data is dynamic, the widget updates automatically. here’s a page of book display widget examples. find out more made it this far? you really need to see syndetics unbound in action. check it out. again, here are some sample links of syndetics unbound at hartford public library in hartford, ct: the raven boys by maggie stiefvater, alexander hamilton by ron chernow, faithful place by tana french. webinars. we hold webinars every tuesday and walk you through the different elements and answer questions. to sign up for a webinar, visit this webex page and search for “syndetics unbound.” interested in syndetics unbound at your library? go here to contact a representative at proquest. or read more about at the syndetics unbound website. or email us at ltflsupport@librarything.com and we’ll help you find the right person or resource. labels: librarything for libraries, new feature, new features, new product posted by tim @ : am comments » share thursday, january th, alamw in boston (and free passes)! abby and kj will be at ala midwinter in boston this weekend, showing off librarything for libraries. since the conference is so close to librarything headquarters, chances are good that a few other lt staff members may appear, as well! visit us. stop by booth # to meet abby & kj (and potential mystery guests!), get a demo, and learn about all the new and fun things we’re up to with librarything for libraries, tinycat, and librarything. get in free. are you in the boston area and want to go to alamw? we have free exhibit only passes. click here to sign up and get one! note: it will get you just into the exhibit hall, not the conference sessions themselves. labels: uncategorized posted by kate @ : pm comments » share thursday, june th, for ala : three free opac enhancements for a limited time, librarything for libraries (ltfl) is offering three of its signature enhancements for free! there are no strings attached. we want people to see how librarything for libraries can improve your catalog. check library. the check library button is a “bookmarklet” that allows patrons to check if your library has a book while on amazon and most other book websites. unlike other options, librarything knows all of the editions out there, so it finds the edition your library has. learn more about check library other editions let your users know everything you have. don’t let users leave empty-handed when the record that came up is checked out. other editions links all your holdings together in a frbr model—paper, audiobook, ebook, even translations. lexile measures put metametrics’ the lexile framework® for reading in your catalog, to help librarians and patrons find material based on reading level. in addition to showing the lexile numbers, we also include an interactive browser. easy to add ltfl enhancements are easy to install and can be added to every major ils/opac system and most of the minor ones. enrichments can be customized and styled to fit your catalog, and detailed usage reporting lets you know how they’re doing. see us at ala. stop by booth at ala annual this weekend in san francisco to talk to tim and abby and see how these enhancements work. if you need a free pass to the exhibit hall, details are in this blog post. sign up we’re offering these three enhancements free, for at least two years. we’ll probably send you links showing you how awesome other enhancements would look in your catalog, but that’s it. find out more http://www.librarything.com/forlibraries or email abby blachly at abby@librarything.com. labels: alaac , lexile measures, librarything for libraries, ltfl posted by abby @ : pm comments » share tuesday, june rd, ala in san francisco (free passes) our booth. but this is kate, not tim or abby. she had the baby. tim and i are headed to san francisco this weekend for the ala annual conference. visit us. stop by booth # to talk to us, get a demo, and learn about all the new and fun things we’re up to with librarything for libraries! stay tuned this week for more announcements of what we’ll be showing off. no, really. it’s going to be awesome. get in free. in the sf area and want to go to ala? we have free exhibit only passes. click here to sign up and get one. it will get you just into the exhibit hall, not the conference sessions themselves. labels: ala, alaac posted by abby @ : pm comments » share monday, february th, new “more like this” for librarything for libraries we’ve just released “more like this,” a major upgrade to librarything for libraries’ “similar items” recommendations. the upgrade is free and automatic for all current subscribers to librarything for libraries catalog enhancement package. it adds several new categories of recommendations, as well as new features. we’ve got text about it below, but here’s a short ( : ) video: what’s new similar items now has a see more link, which opens more like this. browse through different types of recommendations, including: similar items more by author similar authors by readers same series by tags by genre you can also choose to show one or several of the new categories directly on the catalog page. click a book in the lightbox to learn more about it—a summary when available, and a link to go directly to that item in the catalog. rate the usefulness of each recommended item right in your catalog—hovering over a cover gives you buttons that let you mark whether it’s a good or bad recommendation. try it out! click “see more” to open the more like this browser in one of these libraries: spokane county library district arapahoe public library waukegan public library cape may public library sails library network find out more find more details for current customers on what’s changing and what customizations are available on our help pages. for more information on librarything for libraries or if you’re interested in a free trial, email abby@librarything.com, visit http://www.librarything.com/forlibraries, or register for a webinar. labels: librarything for libraries, ltfl, recommendations, similar books posted by abby @ : pm comments » share thursday, february th, subjects and the ship of theseus i thought i might take a break to post an amusing photo of something i wrote out today: the photo is a first draft of a database schema for a revamp of how librarything will do library subjects. all told, it has tables. gulp. about eight of the tables do what a good cataloging system would do: distinguishes the various subject systems (lcsh, medical subjects, etc.) preserves the semantic richness of subject cataloging, including the stuff that never makes it into library systems. breaks subjects into their facets (e.g., “man-woman relationships — fiction”) has two subject facets most of the tables, however, satisfy librarything’s unusual core commitments: to let users do their own thing, like their own little library, but also to let them benefit from and participate in the data and contributions of others.( ) so it: links to subjects from various “levels,” including book-level, edition-level, isbn-level and work-level. allows members to use their own data, or “inherit” subjects from other levels. allows for members to “play librarian,” improving good data and suppressing bad data.( ) allows for real-time, fully reversible aliasing of subjects and subject facets. the last is perhaps the hardest. nine years ago (!) i compared librarything to the “ship of theseus,” a ship which is “preserved” although its components are continually changed. the same goes for much of its data, although “shifting sands” might be a better analogy. accounting for this makes for some interesting database structures, and interesting programming. not every system at librarything does this perfectly. but i hope this structure will help us do that better for subjects.( ) weird as all this is, i think it’s the way things are going. at present most libraries maintain their own data, which, while generally copied from another library, is fundamentally siloed. like an evolving species, library records descend from each other; they aren’t dynamically linked. the data inside the records are siloed as well, trapped in a non-relational model. the profession that invented metadata, and indeed invented sharing metadata, is, at least as far as its catalogs go, far behind. eventually that will end. it may end in a “library goodreads,” every library sharing the same data, with global changes possible, but reserved for special catalogers. but my bet is on a more librarything-like future, where library systems will both respect local cataloging choices and, if they like, benefit instantly from improvements made elsewhere in the system. when that future arrives, we got the schema! . i’m betting another ten tables are added before the system is complete. . the system doesn’t presume whether changes will be made unilaterally, or voted on. voting, like much else, existings in a separate system, even if it ends up looking like part of the subject system. . this is a long-term project. our first steps are much more modest–the tables have an order-of-use, not shown. first off we’re going to duplicate the current system, but with appropriate character sets and segmentation by thesaurus and language. labels: cataloging, subjects posted by tim @ : pm comments » share tuesday, january th, librarything recommends in bibliocommons does your library use bibliocommons as its catalog? librarything and bibliocommons now work together to give you high-quality reading recommendations in your bibliocommons catalog. you can see some examples here. look for “librarything recommends” on the right side. not that kind of girl (daniel boone regional library) carthage must be destroyed (ottowa public library) the martian (edmonton public library) little bear (west vancouver memorial library) station eleven (chapel hill public library) the brothers karamazov (calgary public library) quick facts: as with all librarything for libraries products, librarything recommends only recommends other books within a library’s catalog. librarything recommends stretches across media, providing recommendations not just for print titles, but also for ebooks, audiobooks, and other media. librarything recommends shows up to two titles up front, with up to three displayed under “show more.” recommendations come from librarything’s recommendations system, which draws on hundreds of millions of data points in readership patterns, tags, series, popularity, and other data. not using bibliocommons? well, you can get librarything recommendations—and much more—integrated in almost every catalog (opac and ils) on earth, with all the same basic functionality, like recommending only books in your catalog, as well as other librarything for libraries feaures, like reviews, series and tags. check out some examples on different systems here. sirsidynix enterprise (saint louis public library) sirsidynix horizon information portal (hume libraries) sirsidynix elibrary (spokane county public library) iii encore (arapahoe public library) iii webpac pro (waukegan public library) polaris (cape may county library) ex libris voyager (university of wisconsin-eau claire) interested? bibliocommons: email info@bibliocommons.com or visit http://www.bibliocommons.com/augmentedcontent. see the full specifics here. other systems: email abby@librarything.com or visit http://www.librarything.com/forlibraries. labels: uncategorized posted by tim @ : pm comments » share thursday, october th, new: annotations for book display widgets our book display widgets is getting adopted by more and more libraries, and we’re busy making it better and better. last week we introduced easy share. this week we’re rolling out another improvement—annotations! book display widgets is the ultimate tool for libraries to create automatic or hand-picked virtual book displays for their home page, blog, facebook or elsewhere. annotations allows libraries to add explanations for their picks. some ways to use annotations . explain staff picks right on your homepage. . let students know if a book is reserved for a particular class. . add context for special collections displays. how it works check out the librarything for libraries wiki for instructions on how to add annotations to your book display widgets. it’s pretty easy. interested? watch a quick screencast explaining book display widgets and how you can use them. find out more about librarything for libraries and book display widgets. and sign up for a free trial of either by contacting ltflsupport@librarything.com. labels: book display widgets, librarything for libraries, new feature, new features, widgets posted by kj @ : am comments » share tuesday, october th, send us a programmer, win $ , in books. we just posted a new job post job: library developer at librarything (telecommute). to sweeten the deal, we are offering $ , worth of books to the person who finds them. that’s a lot of books. rules! you get a $ , gift certificate to the local, chain or online bookseller of your choice. to qualify, you need to connect us to someone. either you introduce them to us—and they follow up by applying themselves—or they mention your name in their email (“so-and-so told me about this”). you can recommend yourself, but if you found out about it from someone else, we hope you’ll do the right thing and make them the beneficiary. small print: our decision is final, incontestable, irreversible and completely dictatorial. it only applies when an employee is hired full-time, not part-time, contract or for a trial period. if we don’t hire someone for the job, we don’t pay. the contact must happen in the next month. if we’ve already been in touch with the candidate, it doesn’t count. void where prohibited. you pay taxes, and the insidious hidden tax of shelving. employees and their families are eligible to win, provided they aren’t work contacts. tim is not. » job: library developer at librarything (telecommute) labels: jobs posted by tim @ : am comment » share page of ... ...»last » thingology is librarything's ideas blog, on the philosophy and methods of tags, libraries and suchnot. the librarything blog rss feed combined feed search for: recent posts new syndetics unbound feature: mark and boost electronic resources introducing syndetics unbound alamw in boston (and free passes)! for ala : three free opac enhancements ala in san francisco (free passes) recent comments máy phun phân bón on the librarything programming quiz! janis jones on book display widgets from librarything for libraries marie seltenrych on book display widgets from librarything for libraries tye bishop on introducing thingisbn walter clark on new group: “books in —the future of the book world” archives april october january june february january october june may april march december october september june april march february january november october august june april march february january november october september august july june may april march february january december november october august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november october september august july june may categories signals aaron swartz academics ads ahml ala ala ala anaheim ala midwinter ala ala ala ala alaac alamw alamw alamw alamw aleph alexandria egypt amazon amusement android apis app arl arlington heights armenian astroturfing ato attention australia australian australian tax office authenticity awards barcode scanning bea ben franklin berkman center bhutan biblios bigwig blogging book blogs book covers book display widgets book reviews bookpsychic books bookstores booksurge boston bowdoin bowker branded apps brigadoon library britney spears business c.s. lewis canton cataloging categories censorship charleston chick lit chris catalfo christmas cig cil cil cil cil cil city planning claremont colleges clay shirky cluetrain code codi cognitive cost collection development commemorations common knowledge communiation computers in libraries conference conferencething contests copyright covers coverthing crime csuci curiosities cutter danmarc david weinberger ddc dead or alive department of commerce department of defense department of labor dewey decimal dewey decimal classification discovery layer django doc searls dr. horrible drm durham early reviewers east brunswick ebooks ebpl ebscohost economics elton john email employment enhancement enterprise ereaders erotica event evergreen everything is miscellaneous exlibris facebook fake steve federal libraries feedback flash-mob cataloging folksonomy frbr freedom fun future of cataloging future of the book gbs gene smith getting real giraffe gmilcs google google book search groups guardian harry potter harvard coop hidden images hiring homophily houghton mifflin humor ibistro iii il indexing indiebound inspiration instruction international internet archive internet librarians internships interviews iphone app isbns it conversations itt tallaght jacob nielsen jason griffey javascript jeff atwood jobs json kelly vista kils kindle kingston koha languages lccn lccns lcsh legacies legacy libraries legacy mob lexile measures lianza lianza lib . liblime librarians libraries libraries of the dead library . gang library anywhere library blogging library journal library of congress library of congress report library of the futurue library science library technology librarycampnyc librarything librarything for libraries librarything for publishers librarything local linden labs lis los gatos lter ltfl ltfl categories ltfl libraries ltfl reviews maine marc marcthing marié digby mashups masonic control masons meet-up metadata metasexdactyly michael gorman michael porter microsoft microsoft songsmith mike wesch milestone mobile mobile catalog mobile web monopoly moose movers and shakers nc ncsu neil gaiman nela new feature new features new product newspapers nipply nook north carolina oclc oclc numbers oh opacs open data open library open shelves classification open source openness osc overdrive paid memberships palinet pay what you want physical world pla pla pla podcasts policy politics polls portland portland public library power laws print culture profile pictures qr code ra radiohead randolph county public library rcpl readers advisory reading recommendations reloadevery remixability reviews rhinos richland county rights riverine metaphors roy tennant rusa mars safe for work if you're a cataloger san francisco state university santathing scanning schaufferwaffer screencasts seattle public library second life secret santas serendipity series sfsu shelf browse shelfari shirky similar books sincerity sirsidynix slco small libraries social cataloging social media social networking songsmith sony reader sopac spam stack map stats steve lawson strangeness subjects syndetics unbound syria tag mirror tagging tagmash tags talis talks tax exemption the thingisbn tim tipping points tools translation twitter uclassify ugc uncategorized usability user generated content users utnapishtim vc vertical social networks very short list visualizations voyager vufind web web . webinars weddings weinberger west virginia westlaw widgets wikimania wikimania wirral wirral libraries work disambiguation working group on the future of bibliographic control works worldcat worldcat local xisbn youtube zombies zoomii meta register log in entries rss comments rss wordpress.org help/faqs | about | privacy/terms | blog | contact | apis | wikithing | common knowledge | legacy libraries | early reviewers | zeitgeist copyright librarything and/or members of librarything, authors, publishers, libraries, cover designers, amazon, bol, bruna, etc. http : too many requests - websolr websolr toggle navigation home getting started quickstart guides advanced tips & tricks platform common tools compliance troubleshooting http : too many requests we enforce a limit on the number of simultaneous requests that can be made to a server (in the case of websolr, an index); these are called concurrent connections. this limit is enforced to mitigate against dos attacks, whether intentional or unintentional. the limit scales with plan level on our shared tier plans. single tenant plans don’t have concurrent limits beyond the limitations of the underlying hardware. how do concurrent connections affect thoughput? assume a typical request takes ms. with a x limit, you could serve requests per second. reducing latency to, say, ms (perhaps with field caching or reducing complexity) would double the available throughput to requests per second. besides just upgrading plan level, one method to improving thoughput is to batch your updates. this entails sending updates in batches rather than one at a time. if you have a bunch of workers handling document updates, you should decrease that number to ensure that you don’t have too many trying to send documents at once. this frees up connections to handle search traffic. keep in mind that both searches and updates count as concurrent connections, so if you have workers sending updates and a user submits a search, something will fail with an http . another strategy for avoiding concurrency limits is to use routing headers. this allows you to send searches to your replica and updates to your primary. this can be helpful in minimizing latency by distributing load over both solr cores. the caveat here is that this approach only applies to indices that support replication, and where near real-time results aren’t a requirement. managing traffic our general recommendations for managing high volumes of traffic are: when performing a bulk reindex, update operations should send documents in batches of between and , per request, from one or two indexing processes. ongoing incremental updates (e.g., generated by user activity) should be processed through a queue, so they can be paused, throttled or retried as needed. high volumes of search traffic can also benefit from a layer of caching on the application side. beyond that, we can also increase our concurrent request limits for dedicated cluster customers (titanium and custom plans) for very high traffic sites which may require hundreds of concurrent requests. last updated on may , toggle search categories error codes search no results found © omc . powered by help scout contact us name email subject message upload file anti-prom - wikipedia anti-prom from wikipedia, the free encyclopedia jump to navigation jump to search this article needs additional citations for verification. please help improve this article by adding citations to reliable sources. unsourced material may be challenged and removed. find sources: "anti-prom" – news · newspapers · books · scholar · jstor (november ) (learn how and when to remove this template message) anti-prom, also known as morp (prom spelled backwards), is a social event often staged by high school students as a protest against, or boycott of, their school's official prom, as an alternative celebration. other times, it may be an unofficial prom, planned by the students themselves so that it is not under the control of the school. some of the more common reasons for the creation of an anti-prom include the desire to curb the large cost of a traditional prom, to listen to music other than that expected to be played at the official prom, to have a smaller, more personal get-together, ones that cannot find a date or have been rejected for a prom date, don't like the food, or have looser and less strict rules than the school's (often relating to dress rules or alcohol consumption).[citation needed] another common anti-prom is an unofficial dance set up by freshmen and sophomores as they cannot go to prom without a junior or senior. the attendees of an anti-prom usually disagree with the values of the high school in-crowd who, stereotypically, organize the prom from the preparatory stages to the after-parties. in particular, anti-prom attendees protest what they regard as the vanity, excess, and conformity that the prom culture expects from students.[ ] anti-proms do not follow any prescribed format, catering instead to the varied tastes of the large spectrum of students who feel dissociated from prom culture. nevertheless, anti-prom participants are generally concerned with arranging social activities that are not only fun and enjoyable, but which also serve as an assertion of solidarity and of the legitimacy of social difference.[citation needed] in the united states, youth belonging to the church of jesus christ of latter-day saints often attend morps organized through their church or stake. this is true even in the state of utah, where a substantial percentage of the youth belong to mormon families. the custom is that the girls can each ask a boy to take them, but the boys cannot ask the girls[ ] queer proms[edit] sexual orientation and gender identity sometimes plays a role in leading students to form and attend an anti-prom (sometimes referred to as a gay prom or queer prom): lesbian, gay, bisexual, transgender, queer, and questioning (lgbtqq) students who feel that attending their school's traditional prom with a same-sex partner or not identifying in traditional gender identity fashion would be problematic might choose to hold their own gathering.[ ] gay proms began to form in the s, with one of the longest running and oldest being the hayward gay prom, which was first held in .[ ] references[edit] ^ ann hulbert (november , ). "take back the prom". slate. ^ smith, darron t. (october , ). black and mormon (reprint ed.). chicago: university of illinois press. ^ rebecca parr (december , ). "hayward gay prom is focus of new film". hayward daily review. ^ brenda huang (january , ). "alliance to host tracy's first gay prom". san joaquin news service. retrieved from "https://en.wikipedia.org/w/index.php?title=anti-prom&oldid= " categories: american culture school dances lgbt culture in the united states hidden categories: articles needing additional references from november all articles needing additional references all articles with unsourced statements articles with unsourced statements from april navigation menu personal tools not logged in talk contributions create account log in namespaces article talk variants views read edit view history more search navigation main page contents current events random article about wikipedia contact us donate contribute help learn to edit community portal recent changes upload file tools what links here related changes upload file special pages permanent link page information cite this page wikidata item print/export download as pdf printable version languages Русский edit links this page was last edited on november , at : (utc). text is available under the creative commons attribution-sharealike license; additional terms may apply. by using this site, you agree to the terms of use and privacy policy. wikipedia® is a registered trademark of the wikimedia foundation, inc., a non-profit organization. privacy policy about wikipedia disclaimers contact wikipedia mobile view developers statistics cookie statement colgate skip to main content why should i log in?open modal dialog why log in? × some of the material in colgate digital library is restricted to members of the university community. by logging in, you may be able to gain additional access to certain collections or items. if you have questions about access or logging in, please use the form on the contact page. close log in toggle navigation home collections all collections andrew j. russell photographs baptist education society of the state of new york records burmese album colgate university student newspapers colgate at collection edward h. stone photograph collection film and video collection first baptist church of hamilton, ny local and regional newspapers modern sketch postcards at colgate queer activism at colgate digital history project all collections search term search colgate university digital collections the libraries have begun to digitize archival collections of interest to students, faculty, alumni, and others in order to best support the curriculum and to make colgate history more broadly accessible. explore colgate colgate at collection the - academic year marks the beginning of colgate university's third century. as part of the bicentennial celebrations, the archives has digitized images and documents related to its -year history. explore this collection as a complement to the colgate at years website. items found in collections edward h. stone photograph collection edward h. stone ( – ) was the official colgate photographer for nearly fifty years and also worked commercially in the town of hamilton and its surrounds. the edward h. stone photograph collection ( s– s) contains glass and film negatives, and photographic prints related to colgate university including images of buildings, campus landscapes, student groups, athletics, faculty, and administrators. the collection also includes images related to madison county and the village of hamilton, such as downtown shops and streetscapes, businesses, farms, and portraits. items found in collection colgate university student newspapers our entire run of the colgate university student newspaper, from the earliest, published november , , through april , . the colgate university student newspaper has gone through several title changes. this collection includes access to all of the titles. browse individual titles by date through these links: colgate maroon-news - january , - present colgate news - september , - december , note: the colgate news was an independent newspaper later merged with the colgate maroon to form the colgate maroon news. colgate maroon - april , - november , junior daily - may , - may , note: the junior daily was published by the junior class for one week each may, - . it covered the "junior week" festivities. this event was the predecessor to spring party weekend. madisonensis - august , - april , christian reformer - july , - september , hamilton student and christian reformer - april , - july , hamilton student - november , - march , items colgate university student yearbooks the complete run of colgate university's yearbook, salmagundi, from to the present. newly published yearbooks will not be available online until at least one year after publication. salmagundi idiosyncrasies: until , the focus of the yearbooks was on the junior class, because they were responsible for publishing it. the senior class was still pictured, but it was the juniors who had the spotlight with bigger pictures and more personal details. as a result, the yearbook titles differ from modern interpretation, so the “ ” yearbook was actually published in and included the senior class of and the junior class of . no yearbooks were published for , , and . between and , colgate was known as madison university. items found in collection film and video collection digitized versions of vintage film and video related to colgate university's history, including promotional programs, recordings of events, documentaries, interviews with faculty and almuni, and raw footage. items found in collection baptist education society of the state of new york records the baptist education society of the state of new york (bessny) originated from a meeting of thirteen men in hamilton, new york, in . in , the state of new york granted the society a charter to formalize the establishment of the theological institution that eventually became colgate university. item found in collection postcards at colgate the digitized postcards in this collection showcase the campus and the surrounding area as they were in the early part of the th century. items found in collection burmese album an album of sepia albumen photographs taken in burma by the expatriate german photographer philip adolphe klier. items found in collection local and regional newspapers a digital collection covering the hamilton republican from to , the years of the influenza pandemic. if you are looking for a specific issue by date, please browse the hamilton republican here. items pages about terms of use csv,conf,v - may - - online conference csv,conf,v home how we planned for a virtual csv,conf schedule and speakers faq watch talks code of conduct going online: how we organised the first ever virtual csv,conf a brief history weighing the decision to hold csv,conf,v online moving online - the challenge moving online - the opportunity planning an online conference organizing team selecting talks creating a schedule tickets and registration code of conduct communication via slack picking a video conference platform zoom concerns crowdcast benefits crowdcast challenges #commallama a big thank you to our community and supporters looking ahead - the future of csv,conf a brief history csv,conf is a community conference that brings diverse groups together to discuss data topics, and features stories about data sharing and data analysis from science, journalism, government, and open source. over the years we have had over a hundred different talks from a huge range of speakers, most of which you can still watch back on our youtube channel. csv,conf,v took place in berlin in and we were there again v in before we moved across the atlantic for v and v which were held in portland, oregon in the united states in and . for csv,conf,v we were looking forward to our first conference in washington dc, but unfortunately, like many other in-person events, this was not going to be possible in . people have asked us about our experience moving from a planned in person event to one online, in a very short space of time, so we are sharing our story with the hope that it will be helpful to others, as we move into a world where online events and conferences are going to be more prevalent than ever. the decision to take the conference online was not an easy one. until quite late on, the question csvconf organizers kept asking each other was not “how will we run the conference virtually?” but ‘will we need to cancel?’. as the pandemic intensified, this decision was taken out of our hands and it became quickly clear that cancelling our event in washington dc was not only the responsible thing to do, but the only thing we could do. weighing the decision to hold csv,conf,v online once it was clear that we would not hold an in-person event, we deliberated on whether we would hold an online event, postpone, or cancel. moving online - the challenge one of our main concerns was whether we would be able to encapsulate everything good about csv,conf in a virtual setting – the warmth you feel when you walk into the room, the interesting side conversations, and the feeling of being reunited with old friends, and naturally meeting new ones were things that we didn’t know whether we could pull off. and if we couldn’t, did we want to do this at all? we were worried about keeping a commitment to speakers who had made a commitment themselves. but at the same time we were worried speakers may not be interested in delivering something virtually, or that it would not have the same appeal. it was important to us that there was value to the speakers, and at the start of this process we were committed to making this happen. many of us have experience running events both in person and online, but this was bigger. we had some great advice and drew heavily on the experience of others in similar positions to us. but it still felt like this was different. we were starting from scratch and for all of our preparation, right up to the moment we pressed ‘go live’ inside crowdcast, we did not know whether it was going to work. but what we found was that hard work, lots of planning and support of the community made it work. there were so many great things about the format that surprised and delighted us. we now find ourselves asking whether an online format is in fact a better fit for our community, and exploring what a hybrid conference might look like in the future. moving online - the opportunity there were a great many reasons to embrace a virtual conference. once we made the decision and started to plan, this became ever clearer. not least was the fact that an online conference would give many more people the opportunity to attend. we work hard every year to reduce the barriers to attendance where possible, and we’re grateful to our supporters here, but our ability to support conference speakers is limited, and it is also probably the biggest cost from year to year. we are conscious that barriers to entry still apply to a virtual conference, but they are different, and it is clear that for csv,conf,v more people who wanted to join could be part of it. csv,conf is normally attended by around people. the in-person conferences usually fill up, with just a few attendees under capacity. it feels the right size for our community. but this year we had over , registrations. more new people could attend, and there were also more returning faces. attendees from around the world during csv,conf,v 's opening session planning an online conference despite the obvious differences, much about organising a conference remains the same whether virtual or not. indeed, by the time we made the shift to an online conference, much of this work had been done. organizing team from about september the organising team met up regularly every few weeks on a virtual call. we reviewed our list of things and assigned actions. we used a private channel on slack for core organizers to keep updated during the week. we had a good mix of skills and interests on the organizing team, from community wranglers, to writers and social media aces. we would like to give a shout out to the team of local volunteers we had on board to help with dc-specific things. in the end this knowledge just wasn't needed for the virtual conf. we recruited a group of people from the organising team to act as the programme committee. this group would be responsible for running the call for proposals (cfp) and selecting the talks. we relied on our committed team of organisers for the conference and we found it helpful to have very clear roles/responsibilities to help manage the different aspects of the ‘live’ conference. we had a host who introduced speakers, a q&a/chat monitor, a technical helper, and a safety officer/code of conduct enforcer at all times. it was also helpful to have “floaters” who were unassigned to a specific task, but could help with urgent needs. team work makes the dream work! 💜 see y'all again for csv,conf,v #commallama 🦙 #csvconf https://t.co/bxt agvicw — lilly winfree, phd (@lilscientista) may , selecting talks we were keen on making it easy for people to complete the call for proposals. we set up a google form and asked just a few simple questions. all talks were independently reviewed and scored by members of the committee and we had a final meeting to review our scores and come up with a final list. we were true to the scoring system, but there were other things to consider. some speakers had submitted several talks and we had decided that even if several talks by the same person scored highly, only one could go into the final schedule. we value diversity of speakers, and reached out to diverse communities to advertise the call for proposals and also considered diversity when selecting talks. also, where talks were scoring equally, we wanted to ensure we we’re giving priority to speakers who were new to the conference. we asked all speakers to post their slides onto the csvconf zenodo repository. this was really nice to have because attendees asked multiple times for links to slides, so we could send them to the zenodo collection. though it proved to not be relevant for s virtual event, but it’s worth mentioning that the process of granting travel or accommodation support to speakers was entirely separate from the selection criteria. this was an entirely separate process, and although we asked people to flag a request for support, this did not factor into the decision making process. creating a schedule before we could decide on a schedule, we needed to decide on the hours and timezones we would hold the conference. csvconf is usually a -day event with concurrently run sessions, and we eventually decided to have the virtual event remain days, but have one main talk session with limited concurrent talks. since the in-person conference was supposed to occur in washington, dc, many of our speakers were people in us timezones, so we focused on timezones that would work best for those speakers. we also wanted to ensure that our conference organizers would be awake during the conference. we started at am eastern, which was very early for west coast ( am) and late afternoon for non-us attendees ( pm uk; pm eastern europe; ). we decided on hours of programming, meaning the conference ended in late afternoon for us attendees and late evening for europe. unfortunately, these timezones did not work for everyone (notably the asia-pacific region), and we recommend that you pick timezones that work for your speakers and your conference organizers whilst stretching things as far as possible if equal accessibility is important to you. we also found it was important to clearly list the conference times in multiple timezones on our schedule so that it was easier for attendees to know what time the talks were happening. screengrab above shows the start of our deliberation on what time to start and end the conference each day. see our options across timezones tickets and registration although most of what makes csv,conf successful is human passion and attention (and time!), we also found that the costs involved in running a virtual conference are minimal. except for some extra costs for upgrading our communication platforms, and making funds available to support speakers in getting online, running the conference remotely saved us several thousand dollars. we have always used an honor system for ticket pricing. we ask people pay what they can afford, with some suggested amounts depending on the attendees situation. but we needed to make some subtle changes for the online event, as it was a different proposition. we first made it clear that tickets were free, and refunded those who had already purchased tickets. eventbrite is the platform we have always used for registering attendees for the conference, and it does the job. it’s easy to use and straightforward. we kept it running this year for consistency and to ensure we’re keeping our data organised, even though it involved importing the data into another platform. we were able to make the conference donation based thanks to the support of the sloan foundation, moore foundation and individual contributors and donations. perhaps because the overall registrations also went up, we found that the donations also went up. in future, and with more planning and promotion, it would be feasible to consider a virtual event of the scale of csv,conf funded entirely by contributions from the community it serves. code of conduct we spent significant time enhancing our code of conduct for the virtual conference. we took in feedback from last year’s conference and reviewed other organizations’ code of conducts. the main changes were to consider how a code of conduct needed to relate to the specifics of something happening online. we also wanted to create more transparency in the enforcement and decision-making processes. one new aspect was the ability to report incidents via slack. we designated two event organizers as “safety officers”, and they were responsible for responding to any incident reports and were available for direct messaging via slack (see the code of conduct for full details). we also provided a neutral party to receive incident reports if there were any conflicts of interest. communication via slack we used slack for communications during the conference, and received positive feedback about this choice. we added everyone that registered to the slack channel to ensure that everyone would receive important messages. we had a slack session-bot that would announce the beginning of each session with the link to the session and we received a lot of positive feedback about the session-bot. for people not on slack, we also had the schedule in a google spreadsheet and on the website, and everyone that registered with an email received the talk links via email too. for the session-bot, we used the google calendar for team events app on slack. another popular slack channel that was created for this conference was a dedicated q&a channel allowing speakers to interact with session attendees, providing more context around their talks, linking to resources, and chatting about possible collaborations. at the end of each talk, one organizer would copy all of the questions and post them into this q&a channel so that the conversations could continue. we received a lot of positive feedback about this and it was pleasing to see the conversations continue. we also had a dedicated speakers channel, where speakers could ask questions and offer mutual support and encouragement both before and during the event. another important channel was a backchannel for organisers, which we used mainly to coordinate and cheer each other on during the conf. we also used this to ask for technical help behind the scenes to ensure everything ran as smoothly as possible. after talks, one organizer would use slack private messaging to collate and send positive feedback for speakers, as articulated by attendees during the session. this was absolutely worth it and we were really pleased to see the effort was appreciated. slack is of course free, but its premium service does offer upgrades for charities and we were lucky enough to make use of this. the application process is very easy, and takes less that mins so this is worth considering. we made good use of twitter throughout the conference and there were active #commallama and #csvconf hashtags going throughout the event. the organisers had joint responsibility for this and this seemed to work. we announced the hashtags at the beginning of the day and people picked them up easily. we had a philosophy of 'over-communicating' - offering updates as soon as we had them, and candidly. we used it to to share updates, calls-to-action, and to amplify people's thoughts, questions and feedback picking a video conference platform zoom concerns one of the biggest decisions we had to make was picking a video conferencing platform for the conference. we originally considered using zoom, but were concerned about a few things. the first was reports of rampant “zoombombing”, where trolls join zoom meetings with the intent to disrupt the meeting. the second concern was that we are a small team of organisers and there would be great overhead in moderating a zoom room with hundreds of attendees - muting, unmuting, etc. we also worried that a giant zoom room would feel very impersonal. many of us now spend what is probably an unnecessary amount of our daily lives on zoom and we also felt that stepping away from this would help mark the occasion as something special, so we made the decision to move away from zoom and we looked to options that we’re more of a broadcast tool than meeting tool. crowdcast benefits we saw another virtual conference that used crowdcast (https://neuromatch.io/) and were impressed with how it felt to participate, so we started to investigate it as a platform before enthusiastically committing to it, with some reservations. the best parts of crowdcast to us were the friendly user interface, which includes a speaker video screen, a dedicated chat section with a prompt bar reading “say something nice”, and a separate box for questions. it felt really intuitive and the features were considered, useful, and we incorporated most of them. from the speaker, participant, and host side, the experience felt good and appropriate. the consideration on the different user types was clear in the design and appreciated. one great function was that of a green room, which is akin to a speakers' couch at the backstage of an in-person conference, helping to calm speakers' nerves, check their audio and visual settings, discuss cues, etc. before stepping out onto the stage. another benefit of crowdcast is that the talks are immediately available for re-viewing, complete with chat messages for people to revisit after the conference. this was great as it allowed people to catch up in almost real time and so catch up quickly if they missed something on the day and feel part of the conference discussions as the developed. we also released all talk videos on youtube and tweeted the links to each talk. crowdcast challenges but crowdcast was not without its limitations. everything went very well, and the following issues were not deal breakers, but acknowledging them can help future organisers plan and manage expectations. top of the list of concerns was our complete inexperienced with it, and the likely inexperience of our speakers. to ensure that our speakers were comfortable using crowdcast, we held many practice sessions with speakers before the conference, and also had an attendee ama before the conference to get attendees acquainted with the platform. these sessions were vital for us to practice all together and this time and effort absolutely paid off! if there is one piece of advice you should take away from reading this guide it is this: practice practice practice, and give others the opportunity and space to practice as well. one challenge we faced was hosting - only one account has host privileges, but we learned that many people can log into that account at the same time to share host privileges. hosts can allow other people to share their screen and unmute, and they can also elevate questions from the chat to the questions box. they can also kick people out if they are being disruptive (which didn’t happen for us, but we wanted to be prepared). this felt a bit weird, honestly, and we had to be careful to be aware of the power we had when in the hosts position. weird, but also incredibly useful and a key control feature which was essential for an event run by a group rather than an individual. in crowdcast you can only share screens at a time (so that would be people sharing screens). our usual set up was a host, with one speaker sharing their screen at a time. we could add a speaker for the talks that only had a single other speaker but any more that this we would have had problems. it was easy enough for the host to chop and change who is on screen at any time, and there’s no limit on the total number of speakers in a session. so there is some flexibility, and ultimately, we were ok. but this should be a big consideration if you are running an event with different forms of presentation. crowdcast was also not without its technical hiccups and frustrations. speakers sometimes fell off the call, or had mysterious problems sharing their screens. we received multiple comments/questions on the day about the video lagging/buffering. we often had to resort to the ol’ refresh refresh refresh approach which, to be fair, mostly worked. and on the few occasions we were stumped, there’s quite a lot of support available online and directly from crowdcast. but honestly, there were very few technical issues for a -day online conference. some attendees wanted info on the speakers (ex: name, twitter handle) during the presentation and we agree it would have been a nice touch to have a button or link in crowdcast. there is the “call to action” feature, but we were using that to link to the code of conduct. crowdcast was new to us, and new to many people in the conference community. as well as these practices we found it helpful to set up an faq page with content about how to use crowdcast and what to expect from an online conference in general. overall, it was a good decision and a platform we would recommend for consideration. #commallama finally, it would not be csv,conf if it had not been for the #commallama. the comma llama first joined us for csv,conf,v in portland and joined us again for csv,conf,v . the experience of being around a llama is both relaxing and energising at the same time, and a good way to get people mixing. taking the llama online was something we had to do and we were very pleased with how it worked. it was amazing to see how much joy people go out of the experience and also interesting to notice how well people naturally adapted to the online environment. people naturally organised into a virtual queue and took turns coming on to the screen to screengrab a selfie. thanks to our friends at mtn peaks therapy llamas & alpacas for being so accommodating and helping us to make this possible. i just took my first virtual llama selfie!!! #csvconf is the coolest comma llama. pic.twitter.com/lxjpzlmf f — sisi wei (@sisiwei) may , a big thank you to our community and supporters as we reflect on the experience this year, one things is very clear to us: the conference was only possible because of the community to speak, attend and supported us. it was a success because the community showed up, was kind, welcoming and extremely generous with their knowledge, ideas and time. the local people in dc who stepped up to offer knowledge and support on the ground in dc was a great example of this and we are incredibly grateful or the support, though this turned out not to be needed. we were lucky to have a community of developers, journalists, scientists and civic activists who intrinsically know how to interact and support one another online, and who adapted to the realities of an online conference well. from the moment speakers attended our practice sessions on the platform and started to support one another, we knew that things we’re going to work out. we knew things would not all run to plan, but we trusted that the community would be understanding and actively support us in solving problems. it’s something we are grateful for. we were also thankful to alfred p. sloan foundation, moore foundation and our + individual supporters for making the decision to support us financially. it is worth noting that none of this would have been possible without our planned venue, hotel and catering contracts being very understanding in letting us void our contracts without any penalties. looking ahead - the future of csv,conf many people have been asking us about the future of csv,conf. firstly it’s clear that the csv,conf,v has given us renewed love for the conference and made it abundantly clear to us of the need for a conference like this in the world. it’s also probably the case that the momentum generated by running the conference this year will secure enthusiasm amongst organisers for putting something together next year. so the questions will be ‘what should a future csvconf look like?’. we will certainly be considering our experience of running this years event online. it was such a success that there is an argument for keeping it online going forward, or putting together something of a hybrid. time will tell. we hope that this has been useful for others. if you are organising an event and have suggestions or further questions that could improve this resource, please let us know. our slack remains open and is the best place to get in touch with us. csv,conf is a group effort to put on a non-profit, community conference with dozens of people putting in time, including but not limited to the organizing team: john chodacki, california digital library martin fenner, datacite elaine wong, canadian broadcasting corporation jessica hardwicke, code for science & society jonathan cain, university of oregon jo barratt, open knowledge foundation danielle robinson, code for science & society lilly winfree, open knowledge foundation paul walsh, datopian serah rono, the carpentries we strive to be a supportive and welcoming environment to all attendees. we encourage you to read the conf code of conduct and will be enforcing it. you can revisit csv,conf,v , csv,conf,v , csv,conf,v and csv,conf,v conference websites, or watch csv,conf talks on youtube. the code for this website is open source. view it on github. want to talk to us? join us on slack. dshr's blog: it isn't about the technology dshr's blog i'm david rosenthal, and this is a place to discuss the work i'm doing in digital preservation. thursday, january , it isn't about the technology a year and a half ago i attended brewster kahle's decentralized web summit and wrote: i am working on a post about my reactions to the first two days (i couldn't attend the third) but it requires a good deal of thought, so it'll take a while. as i recall, i came away from the summit frustrated. i posted the tl;dr version of the reason half a year ago in why is the web "centralized"? : what is the centralization that decentralized web advocates are reacting against? clearly, it is the domination of the web by the fang (facebook, amazon, netflix, google) and a few other large companies such as the cable oligopoly. these companies came to dominate the web for economic not technological reasons. yet the decentralized web advocates persist in believing that the answer is new technologies, which suffer from the same economic problems as the existing decentralized technologies underlying the "centralized" web we have. a decentralized technology infrastructure is necessary for a decentralized web but it isn't sufficient. absent an understanding of how the rest of the solution is going to work, designing the infrastructure is an academic exercise. it is finally time for the long-delayed long-form post. i should first reiterate that i'm greatly in favor of the idea of a decentralized web based on decentralized storage. it would be a much better world if it happened. i'm happy to dream along with my friend herbert van de sompel's richly-deserved paul evan peters award lecture entitled scholarly communication: deconstruct and decentralize?. he describes a potential future decentralized system of scholarly communication built on existing web protocols. but even he prefaces the dream with a caveat that the future he describes "will most likely never exist". i agree with herbert about the desirability of his vision, but i also agree that it is unlikely. below the fold i summarize herbert's vision, then go through a long explanation of why i think he's right about the low likelihood of its coming into existence. herbert identifies three classes of decentralized web technology and explains that he decided not to deal with these two: distributed file systems. herbert is right about this. internet-scale distributed file systems were first prototyped in the late s with intermemory and oceanstore, and many successors have followed in their footsteps. none have achieved sustainability or internet platform scale. the reasons are many, the economic one of which i wrote about in is distributed storage sustainable? betteridge's law applies, so the answer is "no". blockchains. herbert is right about this too. even the blockchain pioneers have to admit that, in the real world, blockchains have failed to deliver any of their promised advantages over centralized systems. in particular, as we see with bitcoin, maintaining decentralization against economies of scale is a fundamental, unsolved problem: trying by technical means to remove the need to have viable economics and governance is doomed to fail in the medium- let alone the long-term. what is needed is a solution to the economic and governance problems. then a technology can be designed to work in that framework. and, as vitalik buterin points out, the security of blockchains depends upon decentralization: in the case of blockchain protocols, the mathematical and economic reasoning behind the safety of the consensus often relies crucially on the uncoordinated choice model, or the assumption that the game consists of many small actors that make decisions independently. herbert's reason for disregarding distributed file systems and blockchains is that they both involve entirely new protocols. he favors the approach being pursued at mit in sir tim berners-lee's solid project, which builds on existing web protocols. herbert's long experience convinces him (and me) that this is a less risky approach. my reason is different; they both reduce to previously unsolved problems. the basic idea of solid is that each person would own a web domain, the "host" part of a set of urls that they control. these urls would be served by a "pod", a web server controlled by the user that implemented a whole set of web api standards, including authentication and authorization. browser-side apps would interact with these pods, allowing the user to: export a machine-readable profile describing the pod and its capabilities. write content for the pod. control others access to the content of the pod. pods would have inboxes to receive notifications from other pods. so that, for example, if alice writes a document and bob writes a comment in his pod that links to it in alice's pod, a notification appears in the inbox of alice's pod announcing that event. alice can then link from the document in her pod to bob's comment in his pod. in this way, users are in control of their content which, if access is allowed, can be used by web apps elsewhere. in herbert's vision, institutions would host their researchers "research pods", which would be part of their personal domain but would have extensions specific to scholarly communication, such as automatic archiving upon publication. herbert demonstrates that the standards and technology needed to implement his pod-based vision for scholarly communication exist, if the implementation is currently a bit fragile. but he concludes by saying: by understanding why it is not feasible we may get new insights into what is feasible. i'll take up his challenge, but in regard to the decentralized web that underlies and is in some respects a precondition for his vision. i hope in a future post to apply the arguments that follow to his scholarly communication vision in particular. the long explanation for why i agree with herbert that the solid future "will most likely never exist" starts here. note that much of what i link to from now on is a must-read, flagged (mr). most of them are long and cover many issues that are less, but still, related to the reason i agree with herbert than the parts i cite. cory doctorow introduces his post about charlie stross' keynote for the th chaos communications congress (mr) by writing (mr): stross is very interested in what it means that today's tech billionaires are terrified of being slaughtered by psychotic runaway ais. like ted chiang and me, stross thinks that corporations are "slow ais" that show what happens when we build "machines" designed to optimize for one kind of growth above all moral or ethical considerations, and that these captains of industry are projecting their fears of the businesses they nominally command onto the computers around them. stross uses the paperclip maximizer thought experiment to discuss how the goal of these "slow ais", which is to maximize profit growth, makes them a threat to humanity. the myth is that these genius tech billionaire ceos are "in charge", decision makers. but in reality, their decisions are tightly constrained by the logic embedded in their profit growth maximizing "slow ais". here's an example of a "slow ai" responding to its prime directive and constraining the "decision makers". dave farber's ip list discussed hiroko tabuchi's new york times article how climate change deniers rise to the top in google searches, which described how well-funded climate deniers were buying ads on google that appeared at the top of search results for climate change. chuck mcmanis (chuck & i worked together at sun microsystems. he worked at google then built blekko, another search engine.) contributed a typically informative response. as previously, i have chuck's permission to quote him extensively: publications, as recently as the early st century, had a very strict wall between editorial and advertising. it compromises the integrity of journalism if the editorial staff can be driven by the advertisers. and google exploited that tension and turned it into a business model. how did they do that? when people started using google as an 'answer this question' machine, and then google created a mechanism to show your [paid] answer first, the stage was set for what has become a gross perversion of 'reference' information. why would they do that? their margins were under pressure: the average price per click (cpc) of advertisements on google sites has gone down for every year, and nearly every quarter, since . at the same time microsoft's bing search engine cpcs have gone up. as the advantage of google's search index is eroded by time and investment, primarily by microsoft, advertisers have been shifting budget to be more of a blend between the two companies. the trend suggests that at some point in the not to distant future advertising margins for both engines will be equivalent. and their other businesses weren't profitable: google has scrambled to find an adjacent market, one that could not only generate enough revenue to pay for the infrastructure but also to generate a net income . youtube, its biggest success outside of search, and the closest thing they have, has yet to do that after literally a decade of investment and effort. so what did they do? as a result google has turned to the only tools it has that work, it has reduced payments to its 'affiliate' sites (adsense for content payments), then boosted the number of ad 'slots' on google sites, and finally paying third parties to send search traffic preferentially to google (this too hurts google's overall search margin) and the effect on users is: on the search page, google's bread and butter so to speak, for a 'highly contested' search (that is what search engine marketeers call a search query that can generate lucrative ad clicks) such as 'best credit card' or 'lowest home mortgage', there are many web browser window configurations that show few, if any organic search engine results at all! in other words, for searches that are profitable, google has moved all the results it thinks are relevant off the first page and replaced them with results that people have paid to put there. which is pretty much the definition of "evil" in the famous "don't be evil" slogan notoriously dropped in . i'm pretty sure that no-one at executive level in google thought that building a paid-search engine was a good idea, but the internal logic of the "slow ai" they built forced them into doing just that. another example is that mark zuckerberg's "personal challenge" for is to "fix facebook". in facebook can't be fixed (mr) john battelle writes: you cannot fix facebook without completely gutting its advertising-driven business model. and because he is required by wall street to put his shareholders above all else, there’s no way in hell zuckerberg will do that. put another way, facebook has gotten too big to pivot to a new, more “sustainable” business model. ... if you’ve read “lost context,” you’ve already been exposed to my thinking on why the only way to “fix” facebook is to utterly rethink its advertising model. it’s this model which has created nearly all the toxic externalities zuckerberg is worried about: it’s the honeypot which drives the economics of spambots and fake news, it’s the at-scale algorithmic enabler which attracts information warriors from competing nation states, and it’s the reason the platform has become a dopamine-driven engagement trap where time is often not well spent. john battelle's “lost context is also (mr). i have personal experience of this problem. in the late s i foresaw a bleak future for sun microsystems. its profits were based on two key pieces of intellectual property, the sparc architecture and the solaris operating system. in each case they had a competitor (intel and microsoft) whose strategy was to make owning that kind of ip too expensive for sun to compete. i came up with a strategy for sun to undergo a radical transformation into something analogous to a combination of canonical and an app store. i spent years promoting and prototyping this idea within sun. one of the reasons i have great respect for scott mcnealy is that he gave me, an engineer talking about business, a very fair hearing before rejecting the idea, saying "its too risky to do with a fortune company". another way of saying this is "too big to pivot to a new, more “sustainable” business model". in the terms set by sun's "slow ai" scott was right and i was wrong. sun was taken over by oracle in ; their "slow ai" had no answer for the problems i identified two decades earlier. but in those two decades sun made its shareholders unbelievable amounts of money. in herbert's world of scholarly communication, a similar process can be seen at work in the history of open access (mr, my comments here). in may stanford libraries' highwire press pioneered the move of scholarly publishing to the web by putting the journal of biological chemistry on-line. three years later, vitek tracz was saying: with the web technology available today, publishing can potentially happen independently of publishers. if authors started depositing their papers directly into a central repository, they could bypass publishers and make it freely available. he started the first commercial open-access publisher, biomed central, in (the springer "slow ai" bought it in ). in came the budapest open access initiative: by "open access" to this literature, we mean its free availability on the public internet, permitting any users to read, download, copy, distribute, print, search, or link to the full texts of these articles, crawl them for indexing, pass them as data to software, or use them for any other lawful purpose, without financial, legal, or technical barriers other than those inseparable from gaining access to the internet itself. the only constraint on reproduction and distribution, and the only role for copyright in this domain, should be to give authors control over the integrity of their work and the right to be properly acknowledged and cited. sixteen years later, the "slow ais" which dominate scholarly publishing have succeeded in growing profits so much that roger schonfeld can tweet: i want to know how anyone can possibly suggest that elsevier is an enemy of open access. i doubt any company today profits more from oa and its growth! what elsevier means by "open access" is a long, long way from the budapest definition. the open access advocates, none of them business people, set goals which implied the demise of elsevier and the other "slow ais" without thinking through how the "slow ais" would react to this existential threat. the result was that the "slow ais" perverted the course of "open access" in ways that increased their extraction of monopoly rents, and provided them with even more resources to buy up nascent and established competitors. elsevier's research infrastructure now the "slow ais" dominate not just publishing, but the entire infrastructure of science. if i were elsevier's "slow ai" i would immediately understand that herbert's "research pods" needed to run on elsevier's infrastructure. given university it departments current mania for outsourcing everything to "the cloud" this would be trivial to arrange. they've already done it to institutional repositories. elsevier would then be able to, for example, use a microsoft-like "embrace, extend and extinguish" strategy to exploit its control over researcher's pods. open access advocates point to the rise in the proportion of papers that are freely accessible. they don't point to the rise in payments to the major publishers, the added costs to universities of dealing with the fragmented system, the highly restrictive licenses that allow "free access" in many cases, the frequency with which author processing charges are paid without resulting in free access, and all the other ills that the "slow ais" have visited upon scholarly communication in the pursuit of profit growth. what people mean by saying "the web is centralized" is that it is dominated by a small number of extremely powerful "slow ais", the faangs (facebook, apple, amazon, netflix, google) and the big telcos. none of the discussion of the decentralized web i've seen is about how to displace them, its all about building a mousetrap network infrastructure so much better along some favored axes that, magically, the world will beat a path to their door. this is so not going to happen. for example, you could build a decentralized, open source social network system. in fact, people did. it is called diaspora and it launched in a blaze of geeky enthusiasm in . diaspora is one of the eight decentralization initiatives studied by the mit media lab's defending internet freedom through decentralization (mr) report: the alpha release of the diaspora software was deeply problematic, riddled with basic security errors in the code. at the same time, the founders of the project received a lot of pressure from silicon valley venture capitalists to “pivot” the project to a more profitable business model. eventually the core team fell apart and the diaspora platform was handed over to the open source community, who has done a nice job of building out a support website to facilitate new users in signing up for the service. today it supports just under , active participants, but the platform remains very niche and turnover of new users is high. facebook has . * daily users, so it is about , times bigger than diaspora. even assuming diaspora was as good as facebook, an impossible goal for a small group of eben moglen's students, no-one had any idea how to motivate the other . % of facebook users to abandon the network where all their friends were and restart building their social graph from scratch. the fact that after years diaspora has k active users is impressive for an open source project, but it is orders of magnitude away from the scale needed to be a threat to facebook. we can see this because facebook hasn't bothered to react to it. suppose the team of students had been inspired, and built something so much better than facebook along axes that the mass of facebook users cared about (which don't include federation, censorship resistance, open source, etc.) that they started to migrate. facebook's "slow ai" would have reacted in one of two ways. either the team would have been made a financial offer they couldn't refuse, which wouldn't have made a dent in the almost $ b in cash and short-term investments on facebook's balance sheet. or facebook would have tasked a few of their more than engineers to replicate the better system. they'd have had an easy job because (a) they'd be adding to an existing system rather than building from scratch, and (b) because their system would be centralized, so wouldn't have to deal with the additional costs of decentralization. almost certainly facebook would have done both. replicating an open source project in-house is very easy and very fast. doing so would reduce the price they needed to pay to buy the startup. hiring people good enough to build something better than the existing product is a big problem for the faangs. the easiest way to do it is to spot their startup early and buy it. the faangs have been doing this so effectively that it no longer makes sense to do a startup in the valley with the goal of ipo-ing it; the goal is to get bought by a faang. lets see what happens when one of the faangs actually does see something as a threat. last january lina m. kahn of the open markets team at the new america foundation published amazon's antitrust paradox (mr) in the yale law review. her , -word piece got a lot of well-deserved attention for describing how platforms evade antitrust scrutiny. in august, barry lynn, kahn's boss and the entire open markets team were ejected from the new america foundation. apparently, the reason was this press release commenting favorably on google's € . billion loss in an antitrust case in the eu. lynn claims that: hours after his press release went online, [new america ceo] slaughter called him up and said: “i just got off the phone with eric schmidt and he is pulling all of his money,” the faangs' "slow ais" understand that antitrust is a serious threat. € . billion checks get their attention, even if they are small compared to their cash hoards. the pr blowback from defenestrating the open markets team was a small price to pay for getting the message out that advocating for effective antitrust enforcement carried serious career risks. this was a faang reacting to a law journal article and a press release. "all of his money" had averaged about $ m/yr over two decades. imagine how faangs would react to losing significant numbers of users to a decentralized alternative! kahn argued that: the current framework in antitrust—specifically its pegging competition to “consumer welfare,” defined as short-term price effects—is unequipped to capture the architecture of market power in the modern economy. we cannot cognize the potential harms to competition posed by amazon’s dominance if we measure competition primarily through price and output. specifically, current doctrine underappreciates the risk of predatory pricing and how integration across distinct business lines may prove anticompetitive. these concerns are heightened in the context of online platforms for two reasons. first, the economics of platform markets create incentives for a company to pursue growth over profits, a strategy that investors have rewarded. under these conditions, predatory pricing becomes highly rational—even as existing doctrine treats it as irrational and therefore implausible. second, because online platforms serve as critical intermediaries, integrating across business lines positions these platforms to control the essential infrastructure on which their rivals depend. this dual role also enables a platform to exploit information collected on companies using its services to undermine them as competitors. in the s antitrust was aimed at preserving a healthy market by eliminating excessive concentration of market power. but: due to a change in legal thinking and practice in the s and s, antitrust law now assesses competition largely with an eye to the short-term interests of consumers, not producers or the health of the market as a whole; antitrust doctrine views low consumer prices, alone, to be evidence of sound competition. by this measure, amazon has excelled; it has evaded government scrutiny in part through fervently devoting its business strategy and rhetoric to reducing prices for consumers. shop, ikebukuro, tokyo the focus on low prices for "consumers" rather than "customers" is especially relevant for google and facebook; it is impossible to get monetary prices lower than those they charge "consumers". the prices they charge the "customers" who buy ad space from them are another matter, but they don't appear to be a consideration for current antitrust law. nor is the non-monetary price "consumers" pay for the services of google and facebook in terms of the loss of privacy, the spam, the fake news, the malvertising and the waste of time. perhaps the reason for google's dramatic reaction to the open markets team was that they were part of a swelling chorus of calls for antitrust action against the faangs from both the right and the left. roger mcnamee (previously) was an early investor in facebook and friend of zuckerberg's, but in how to fix facebook — before it fixes us (mr) even he voices deep concern about facebook's effects on society. he and ethicist tristan harris provide an eight-point prescription for mitigating them: ban bots. block further acquisitions. "be transparent about who is behind political and issues-based communication" "be more transparent about their algorithms" "have a more equitable contractual relationship with users" impose "a limit on the commercial exploitation of consumer data by internet platforms" "consumers, not the platforms, should own their own data" why would the facebook "slow ai" do any of these things when they're guaranteed to decrease its stock price? the eighth is straight out of lina kahn: we should consider that the time has come to revive the country’s traditional approach to monopoly. since the reagan era, antitrust law has operated under the principle that monopoly is not a problem so long as it doesn’t result in higher prices for consumers. under that framework, facebook and google have been allowed to dominate several industries—not just search and social media but also email, video, photos, and digital ad sales, among others—increasing their monopolies by buying potential rivals like youtube and instagram. while superficially appealing, this approach ignores costs that don’t show up in a price tag. addiction to facebook, youtube, and other platforms has a cost. election manipulation has a cost. reduced innovation and shrinkage of the entrepreneurial economy has a cost. all of these costs are evident today. we can quantify them well enough to appreciate that the costs to consumers of concentration on the internet are unacceptably high. mcnamee understands that the only way to get facebook to change its ways is the force of antitrust law. another of the initiatives studied by the mit media lab's defending internet freedom through decentralization (mr) is solid. they describe the project's goal thus: ultimately, the goal of this project is to render platforms like facebook and twitter as merely “front-end” services that present a user’s data, rather than silos for millions of people’s personal data. to this end, solid aims to support users in controlling their own personal online datastore, or “pod,” where their personal information resides. applications would generally run on the client-side (browser or mobile phone) and access data in pods via apis based on http. in other words, to implement mcnamee's # prescription. why do you think mcnamee's # talks about the need to "revive the country’s traditional approach to monopoly"? he understands that having people's personal data under their control, not facebook's, would be viewed by facebook's "slow ai" as an existential threat. exclusive control over the biggest and best personal data of everyone on the planet, whether or not they have ever created an account, is the basis on which facebook's valuation rests. the media lab report at least understands that there is an issue here: the approach of solid towards promoting interoperability and platform-switching is admirable, but it begs the question: why would the incumbent “winners” of our current system, the facebooks and twitters of the world, ever opt to switch to this model of interacting with their users? doing so threatens the business model of these companies, which rely on uniquely collecting and monetizing user data. as such, this open, interoperable model is unlikely to gain traction with already successful large platforms. while a site like facebook might share content a user has created–especially if required to do so by legislation that mandates interoperability–it is harder to imagine them sharing data they have collected on a user, her tastes and online behaviors. without this data, likely useful for ad targeting, the large platforms may be at an insurmountable advantage in the contemporary advertising ecosystem. the report completely fails to understand the violence of the reaction solid will face from the faangs "slow ais" if it ever gets big enough for them to notice. note that the report fails to understand that you don't have to be a facebook user to have been extensively profiled. facebook's "slow ai" is definitely not going to let go of the proprietary data it has collected (and in many cases paid other data sources for) about a person. attempts to legislate this sharing in isolation would meet ferocious lobbying, and might well be unconstitutional. nor is it clear that, even if legislation passed, the data would be in a form usable by the person, or by other services. history tends to show that attempts to force interoperability upon unwilling partners are easily sabotaged by them. mcnamee points out that, even if sharing were forced upon facebook, it would likely do little to reduce their market power: consumers, not the platforms, should own their own data. in the case of facebook, this includes posts, friends, and events—in short, the entire social graph. users created this data, so they should have the right to export it to other social networks. given inertia and the convenience of facebook, i wouldn’t expect this reform to trigger a mass flight of users. instead, the likely outcome would be an explosion of innovation and entrepreneurship. facebook is so powerful that most new entrants would avoid head-on competition in favor of creating sustainable differentiation. start-ups and established players would build new products that incorporate people’s existing social graphs, forcing facebook to compete again. after all, allowing users to export their data from facebook doesn't prevent facebook maintaining a copy. and you don't need to be a facebook user for them to make money from data they acquire about you. note that, commendably, google has for many years allowed users to download the data they create in the various google systems (but not the data google collects about them) via the data liberation front, now google takeout. it hasn't caused their users to leave. no alternate social network can succeed without access to the data facebook currently holds. realistically, if this is to change, there will be some kind of negotiation. facebook's going-in position will be "no access". thus the going-in position for the other side needs to be something that facebook's "slow ai" will think is much worse than sharing the data. we may be starting to see what the something much worse might be. in contrast to the laissez-faire approach of us antitrust authorities, the eu has staked out a more aggressive position. it fined google the € . billion that got the open markets team fired. and, as cory doctorow reports (mr): back in , the eu passed the general data protection regulation, a far-reaching set of rules to protect the personal information and privacy of europeans that takes effect this coming may. doctorow explains that these regulations require that: under the new directive, every time a european's personal data is captured or shared, they have to give meaningful consent, after being informed about the purpose of the use with enough clarity that they can predict what will happen to it. every time your data is shared with someone, you should be given the name and contact details for an "information controller" at that entity. that's the baseline: when a company is collecting or sharing information about (or that could reveal!) your "racial or ethnic origin, political opinions, religious or philosophical beliefs, or trade union membership, … [and] data concerning health or data concerning a natural person’s sex life or sexual orientation," there's an even higher bar to hurdle. pagefair has a detailed explanation of what this granting of granular meaningful consent would have to look like. it is not a viable user interface to the current web advertising ecosystem of real-time auctions based on personal information. all of these companies need to get consent here is pagefair's example of what is needed to get consent from each of them. the start of a long, long chain of dialog boxes doctorow's take on the situation is: there is no obvious way the adtech industry in its current form can comply with these rules, and in the nearly two years they've had to adapt, they've done virtually nothing about it, seemingly betting that the eu will just blink and back away, rather than exercise its new statutory powers to hit companies for titanic fines, making high profile examples out of a few sacrificial companies until the rest come into line. but this is the same institution that just hit google with a $ . billion fine. they're spoiling for this kind of fight, and i wouldn't bet on them backing down. there's no consumer appetite for being spied on online ... and the companies involved are either tech giants that everyone hates (google, facebook), or creepy data-brokers no one's ever heard of and everyone hates on principle (acxiom). these companies have money, but not constituencies. meanwhile, publishers are generally at the mercy of the platforms, and i assume most of them are just crossing their fingers and hoping the platforms flick some kind of "comply with the rules without turning off the money-spigot" switch this may. pagefair's take is: websites, apps, and adtech vendors, should switch from using personal data to monetize direct and rtb advertising to “non-personal data”. using non-personal, rather than personal, data neutralizes the risks of the gdpr for advertisers, publishers, and adtech vendors. and it enables them to address the majority ( %- %) of the audience that will not give consent for rd party tracking across the web. the eu is saying "it is impractical to monetize personal information". since facebook's and google's business models depend on monetizing personal information, this is certainly looks like "something worse" than making it portable. i remember at esther dyson's conference listening to the ceo of american express explain how they used sophisticated marketing techniques to get almost all their customers to opt-in to information sharing. if i were facebook's or google's "slow ai" i'd be wondering if i could react to the gdpr by getting my users to opt-in to my data collection, and structuring things so they wouldn't opt-in to everyone else's. i would be able to use their personal information, but i wouldn't be able to share it with anyone else. that is a problem for everyone else, but for me its a competitive advantage. it is hard to see how this will all play out: the chinese government is enthusiastic about enabling companies to monetize personal information. that way the companies fund the government's surveillance infrastructure: wechat, the popular mobile application from tencent holdings, is set to become more indispensable in the daily lives of many chinese consumers under a project that turns it into an official electronic personal identification system. the us has enabled personal information to be monetized, but seems to be facing a backlash from both right and left. the eu seems determined to eliminate, or at least place strict limits on, monetizing of personal information. balkanization of the web seems more likely than decentralization. if a decentralized web doesn't achieve mass participation, nothing has really changed. if it does, someone will have figured out how to leverage antitrust to enable it. and someone will have designed a technical infrastructure that fit with and built on that discovery, not a technical infrastructure designed to scratch the itches of technologists. posted by david. at : am labels: advertising, amazon, distributed web, open access, scholarly communication, social networks comments: david. said... richard smith, ex-editor of the bmj, has figured out what elsevier's "slow ai" is doing: "the company recognises that science publishing will become a service that scientists will largely run themselves. in a sense, it always has been with scientists producing the science, editing the journals, peer reviewing the studies, and then reading the journals. ... elsevier have recognised the importance of this trend and are creating their own software platforms to speed up and make cheaper the process of publishing science. but how, i wondered, can elsevier continue to make such big profits from science publishing? now, i think i see. the company thinks that there will be one company supplying publishing services to scientists—just as there is one amazon, one google, and one facebook; and elsevier aims to be that company. but how can it make big profits from providing a cheap service? the answer lies in big data. ... elsevier will come to know more about the world’s scientists—their needs, demands, aspirations, weaknesses, and buying patterns—than any other organisation. the profits will come from those data and that knowledge. the users of facebook are both the customers and the product, and scientists will be both the customers and the product of elsevier." unfortunately, he's a bit late to the game. i realized the elsevier had figured this out almost a decade ago when i served on the jury for elsevier's grand challenge . in i wrote: "in i was on the jury for the elsevier grand challenge, a competition elsevier ran with a generous prize for the best idea of what could be done with access to their entire database of articles. this got a remarkable amount of attention from some senior managers. why did they sponsor the competition? they understood that, over time, their ability to charge simply for access to the raw text of scholarly articles will go away. their idea was to evolve to charging for services based on their database instead." january , at : am ruben verborgh said... i don't think decentralization or balkanization are the only options. being a strong believer and creator of decentralized technology, i still intend to maintain my facebook account for the years to come—while simultaneously also using decentralized storage and applications. there is room for both options to exist in parallel, and they would just have different usages. i see a strong potential for solid-like solutions in several business sectors, such as the legal and medical domains, where special data requirements make the decoupling of data and apps a strong advantage. we should stop seeing decentralized solutions as competitors to facebook, but rather as useful platforms in their own right, which already have relevant use cases. january , at : am david. said... ruben, if all you have are some applications in the "legal and medical domains", you haven't decentralized the web, have you? january , at : am david. said... "zuckerberg’s announcement on wednesday that he would be changing the facebook news feed to make it promote “meaningful interactions” does little to address the concerns i have with the platform." writes roger mcnamee. january , at : pm adrian cockcroft said... hi david, it would be good to catch up sometime and have a chat about this. i’m now at aws and though in some ways cloud is centralizing it’s also decentralizing in other ways. a lot of companies are shutting down entire data centers and moving core backend systems to a more distributed cloud based architecture. january , at : pm david. said... hi, adrian! decentralization and distribution are two different things. vitalik buterin identifies three axes of (de)centralization: - architectural (de)centralization — how many physical computers is a system made up of? how many of those computers can it tolerate breaking down at any single time? - political (de)centralization — how many individuals or organizations ultimately control the computers that the system is made up of? - logical (de)centralization— does the interface and data structures that the system presents and maintains look more like a single monolithic object, or an amorphous swarm? one simple heuristic is: if you cut the system in half, including both providers and users, will both halves continue to fully operate as independent units? as far as i can see, systems built on aws may be architecturally decentralized (but are more likely just distributed), but are politically and logically centralized so, even if decentralization delivered its promised advantages, they would get few of them. i believe that aws is better at running data centers than companies. whether they are enough better to outweigh the added risk that comes from correlations between the failure of my system and failures of other systems at aws that my system depends upon (say my supply chain partners) is an interesting question. as far as i can see the biggest selling point of aws is that it provides the in-house it organization some place to point fingers when things go wrong. its the modern equivalent of "no-one ever got fired for buying ibm". january , at : pm unknown said... thank you for putting this article together with great references. just to clarify (a minor point given everything that you've mentioned): the "solid approach" doesn't mandate that we must all have our domains and run our personal online datastores (pod) there. it is certainly one of the ways of going at it. the bigger story there is that our assets are access controlled working alongside a global authentication mechanism. the bottom line there is about where one places their trust, eg. if a researcher trusts their institution to take care of their pod, that's all good - as exemplified in herbert's talk. to support your argument towards how the "big players" are not particularly concerned - at least in public at this time - about decentralised initiatives, we can simply look at their involvement in standardisation efforts. namely speaking: the w c social web working group which went on a long journey in coming up with recommendations towards interoperable protocols and data exchange on the web, with emphasis on "social" stuff. the big players, while having w c member status did not participate in this wg. i think that speaks in volumes. january , at : am david. said... thanks for the clarification, sarven! my concern is, as i briefly mentioned in the post, that research institutions are in a desperate rush to outsource everything to do with it to "the cloud". there are a number of reasons, including the fact that they can't hire enough skilled people, and that outsourcing means that the cio has some place to point the finger when things go wrong. "the cloud" generally means aws, but in the case of researcher's "pods" it would definitely mean elsevier (who would probably run on aws). so the "pods" would all end up controlled by elsevier. which in the big picture would not be much of a change, and would probably end up being even more expensive than the system we have now. january , at : am unknown said... i agree with you on the likelihood of elsevier (or some other company) seamlessly offering pod services to institutions. while there are alarm bells all around that for never ending vendor lock-in - eg "embrace, extend and extinguish" - i suppose it might settle down on which core features of the system are non-proprietary. for instance, could institutions or researchers decide to pack up their bags and go to the next "hosting" provider, ie. continuous and ubiquitous offering of the research objects that are part of the commons? would the generated data be reusable by any application that conform to some open standards? if i was shopping around for a service or a tool, i'd check to see if it passes an acid test along the lines of https://linkedresearch.org/rfc#acid-test. from that angle, the possibility of "controlled by elsevier" could mean anything. is the web hosting provider for my domain controlling my pod? they may be mining the data (including all the interactions that go with it) but i suppose i can't be certain. to escape that, i'll have to run my own hosting box. i assume that most people agree that the major academic publishers will continue to milk the system. in the meantime, the best i think we (researchers and institutions) can do is embrace the feasibility of this lets-get-all-decentralised-bandwagon because the current path is not quite working in our favour. if we are lucky, the effect of this may be that the playing field will even out for new service/tool providers to participate, and maybe diversity in how one can participate - wishful thinking? just to contrast: orcid is a great service for many, but what's the end result or some of the consequences of such effort? new scholarly systems are being built that solely recognise identifiers that virtually include orcid's domain name. one is literally forbidden to participate / make their contributions to humanity - and be acknowledged by it - unless they create an orcid account. this is not orcid's fault but precisely what happens when we go all in on centralised systems, no matter how shiny or useful it may seem at first. its design has consequences. doi is precisely the same story. your "scholarly" article doesn't have a doi? it is considered to be equivalent to a random blogpost on the web regardless of what it says, or dare i say, a "preprint". again, this is not because orcid or doi are bad or wrong designs. the bigger question in my opinion is can we have different systems cooperate and be welcome to participate in the big scholarly bazaar? for our little corner in web science - but not exclusive to - i've tried to capture some our problems and steps towards a paradigm shift which you might be interested in having a look: http://csarven.ca/web-science-from- -to- . same old, same old :) january , at : am kingsley idehen said... hi david, i little correction regarding the section about solid in this insightful article: a webid (http uri that identifies an agen) and its associated webid-profile doc (collection of rdf sentences using various notations) only require users to possess read-write privileges over a folder (a/k/a ldpcontainer instance these days). fundamentally, domain ownership requirement is one of the problems with current storage models (e.g., systems where identification depends on ".well-known" pattern), in regards to a viable read-write web etc.. conventions outlined by the solid project enable pods (or personal data spaces) to be scoped to folders. thus, google drive, dropbox, onedrive, amazon aws, rackspace, box. are all viable launch points [ ]. [ ] https://medium.com/virtuoso-blog/taking-control-of-access-to-resources-stored-in-google-drive-dropbox-microsoft-onedrive-box-and-d ab dd d -- mounting various storage services january , at : pm pmcb said... hi david - fantastic article (i've an even bigger mr backlog now!). but i didn't understand your response to ruben. surely both centralised and decentralised systems have their place (e.g. facebook 'cos all my mates are there already, or facebook-decentralised 'cos they pay me per view, or allow non-facebookers to comment on or annotate my posts, or whatever). noone is suggesting *everything* must be decentralised for us to start 'decentralising the web', incrementally, as deemed appropriate by individuals, are they? january , at : pm david. said... pmcb, what people mean by "the web is centralized" is that it is dominated by the faangs. if you want to change that, you have to displace some or all of the faangs. otherwise the web will still be centralized i.e. dominated by the faangs. its fine to build something, e.g. diaspora, that attracts say k users. but that hasn't changed the big picture at all. what i'm trying to do is to get people to think about how to displace the faangs instead of thinking about the neat-oh technology they think it'd be cool to build for themselves and a few friends. decentralizing the web: it isn't about the technology. january , at : pm pmcb said... david - hhhmmm... ok, i think i see where you're coming from now, and it's a good point. but for whatever reason, one word popped into my head after reading your response - tesla. maybe it'll take a social-media breach on the scale of equifax to wake up the general populace to the consequences of facebook et al, but regardless, like autonomous or electric-only cars, i think (purely personal opinion) that a large proportion of the web will become decentralised (how 'large' is just a reflection of how naive you are i guess!), but my conviction is based on it simply being the more 'correct' thing to do - like electric cars (full disclosure: i'm a cyclist, and therefore inherently dislike all cars!). i know a counter-argument is 'but where's the money in decentralisation', but wasn't that exactly the argument years ago with electric cars too (which seems so utterly myopic and 'missing the whole point' today)?? january , at : pm david. said... prof. scott galloway of nyu argues for breaking up the faangs in his talk to the dld conference. january , at : am david. said... tesla delivered about k cars in the first half of . in the whole of general motors delivered m in the us, m in china. lets say tesla delivered k cars in . in just these two regions gm alone sold times as many. tesla is an amazing achievement, but its still a very small car company. its a little more than / the size of porsche. but the key point is that, right from the start with the roadster, elon musk had an explanation for why people would buy the cars that wasn't simply that they were electric. they were fast and fun to drive. "being the more 'correct' thing to do" is not an explanation for why people will migrate off the faangs. january , at : am david. said... with facebook and google’s surveillance capitalism model is in trouble, paul blumenthal joins in the calls for regulation. he too notices the looming impact of the gdpr: "for the most part, facebook and google prevent you from using their products if you decline to agree to their entire terms of service. you cannot pick and choose what to agree to and still use their free services. the gdpr changes that by requiring online companies, in some cases, to get permission from each individual user to collect, share and combine their data for use by advertisers. companies will not be allowed to ban people from their services who decline to share their data for advertising purposes. there are million eu residents who will soon be able to opt out of helping facebook and google make money. if companies do not comply with the new regulations they will face fines totaling four percent of their global revenues." january , at : pm david. said... anti-trust is having an effect: "google's bet against the commission cost it $ . billion; qualcomm's out more than a billion, and apple's got to pay $ . billion in evaded taxes. these - and -figure invoices have made deathbed converts out of big tech, who are now in a made scramble to comply with the gdpr before its fines of the greater of % of global total profit or million euros kick in this may." as everett dirksen said "a billion here, a billion there, pretty soon you're talking real money." january , at : am david. said... adam ludwin's a letter to jamie dimon is worth a read. even though i don't agree with all of it, he makes some very good points, such as: "since ethereum is a platform, its value is ultimately a function of the value of the applications built on top. in other words, we can ask if ethereum is useful by simply asking if anything that has been built on ethereum is useful. for example, do we need censorship resistant prediction markets? censorship resistant meme playing cards? censorship resistant versions of youtube or twitter? while it’s early, if none of the + decentralized apps built on ethereum so far seem useful, that may be telling. even in year of the web we had chat rooms, email, cat photos, and sports scores. what are the equivalent killer applications on ethereum today?" february , at : pm david. said... david dayen's tech companies are under pressure everywhere except where it matters looks at the pathetic state of anti-trust in the us, particularly at the federal trade commission, and contrasts it with all the political rhetoric about the tech behemoths. as usual with politicians, you need to look at what they do not what they say. february , at : pm david. said... "mining and oil companies exploit the physical environment; social media companies exploit the social environment. this is particularly nefarious because social media companies influence how people think and behave without them even being aware of it. this has far-reaching adverse consequences on the functioning of democracy, particularly on the integrity of elections. the distinguishing feature of internet platform companies is that they are networks and they enjoy rising marginal returns; that accounts for their phenomenal growth. the network effect is truly unprecedented and transformative, but it is also unsustainable. it took facebook eight and a half years to reach a billion users and half that time to reach the second billion. at this rate, facebook will run out of people to convert in less than years." this is from george soros' wide-ranging and very interesting remarks in davos. he goes on: "the exceptional profitability of these companies is largely a function of their avoiding responsibility for– and avoiding paying for– the content on their platforms. they claim they are merely distributing information. but the fact that they are near- monopoly distributors makes them public utilities and should subject them to more stringent regulations, aimed at preserving competition, innovation, and fair and open universal access." february , at : am david. said... prof. scott galloway continues his argument for breaking up the faangs in esquire. it is long but really worth reading: "why should we break up big tech? not because the four are evil and we’re good. it’s because we understand that the only way to ensure competition is to sometimes cut the tops off trees, just as we did with railroads and ma bell. this isn’t an indictment of the four, or retribution, but recognition that a key part of a healthy economic cycle is pruning firms when they become invasive, cause premature death, and won’t let other firms emerge. the breakup of big tech should and will happen, because we’re capitalists." february , at : am david. said... german court rules facebook use of personal data illegal by hans-edzard busemann & nadine schimroszik at reuters reveals that: "a court had found facebook’s use of personal data to be illegal because the u.s. social media platform did not adequately secure the informed consent of its users. the verdict, from a berlin regional court, comes as big tech faces increasing scrutiny in germany over its handling of sensitive personal data that enables it to micro-target online advertising. the federation of german consumer organisations (vzvb) said that facebook’s default settings and some of its terms of service were in breach of consumer law, and that the court had found parts of the consent to data usage to be invalid. " february , at : am david. said... chuck mcmanis points me to rowland manthorpe's google’s nemesis: meet the british couple who took on a giant, won... and cost it £ . billion. it is a fascinating account of the history behind the eu's anti-trust fine on google. february , at : am david. said... chris dixon's why decentralization matters is sort-of-half-right: "the question of whether decentralized or centralized systems will win the next era of the internet reduces to who will build the most compelling products, which in turn reduces to who will get more high quality developers and entrepreneurs on their side." the first part of the sentence is right, the second part is techno-optimism like most of the rest of the essay. february , at : pm david. said... sam d'amico argues that it is about the technology, in a sense. march , at : am david. said... what happens if you give an ai control over a corporation? is an interesting question. clive thompson points to a paper by ucla law professor lynn lopucki: "odds are high you'd see them emerge first in criminal enterprises, as ways of setting up entities that engage in nefarious activities but cannot be meaningfully punished (in human terms, anyway), even if they're caught, he argues. given their corporate personhood in the us, they'd enjoy the rights to own property, to enter into contracts, to legal counsel, to free speech, and to buy politicians -- so they could wreak a lot of havoc." not that the current "slow ais" can be "meaningfully punished" if they engage in "nefarious activities". march , at : am david. said... lina kahn's the supreme court case that could give tech giants more power reports: "but the decision in a case currently before the supreme court could block off that path, by effectively shielding big tech platforms from serious antitrust scrutiny. on monday the court heard ohio v. american express, a case centering on a technical but critical question about how to analyze harmful conduct by firms that serve multiple groups of users. though the case concerns the credit card industry, it could have sweeping ramifications for the way in which antitrust law gets applied generally, especially with regards to the tech giants." march , at : am david. said... shira ovide's how amazon’s bottomless appetite became corporate america’s nightmare concludes: "amazon is far from invulnerable. all the same old red flags are there—a puny . percent e-commerce profit in north america, massive outlays to establish delivery routes abroad—but few are paying attention. anyone buying a share of amazon stock today is agreeing to pay upfront for the next years of profit. by one measure, it’s generating far less cash than investors believe. and its biggest risk may be the fear of its power in washington, new york, and brussels, a possible prelude to regulatory crackdown." [my emphasis] march , at : pm david. said... "an index of tech growth shares pushed its advance to percent so far this year, giving the group an annualized return since early of percent. that frenzied pace tops the nasdaq composite index’s percent return in the final two years of the dot-com bubble." writes lu wang at bloomberg: "in addition to the quartet of facebook, amazon, netflix and google, the nyse index also includes apple, twitter, alibaba, baidu, nvidia and tesla. these companies have drawn money as investors bet that their dominance in areas from social media to e-commerce will foster faster growth. ... at times earnings, the companies in the nyse fang+ index are valued at a multiple that’s almost three times the broader gauge’s. that compared with . in march ." march , at : am david. said... cory doctorow points out that: "in , the guardian's victor keegan published "will myspace ever lose its monopoly?" in which he enumerated the unbridgeable moats and unscalable walls that "rupert murdoch's myspace" had erected around itself, evaluating all the contenders to replace myspace and finding them wanting." i could be wrong! but that was then and this is now. facebook is far bigger and more embedded than myspace ever was. march , at : am david. said... in the real villain behind our new gilded age eric posner and glen weyl reinforce the meme that the absence of effective antitrust policy is the cause of many of society's ills, including inequality, slow economic growth and the faangs: "the rise of the internet has bestowed enormous market power on the tech titans — google, facebook — that rely on networks to connect users. yet again, antitrust enforcers have not stopped these new robber barons from buying up their nascent competitors. facebook swallowed instagram and whatsapp; google swallowed doubleclick and waze. this has allowed these firms to achieve near-monopolies over new services based on user data, such as training machine learning and artificial intelligence systems. as a result, antitrust authorities allowed the creation of the world’s most powerful oligopoly and the rampant exploitation of user data." may , at : am david. said... people like convenience more than privacy – so no, blockchain will not 'decentralise the web' by matt asay makes the same point: "it's not that a blockchain-based web isn't possible. after all, the original web was decentralised, too, and came with the privacy guarantees that blockchain-based options today purport to deliver. no, the problem is people. as user interface designer brennan novak details, though the blockchain may solve the crypto crowd's privacy goals, it fails to offer something as secure and easy as a (yes) facebook or google login: 'the problem exists somewhere between the barrier to entry (user-interface design, technical difficulty to set up, and overall user experience) versus the perceived value of the tool, as seen by joe public and joe amateur techie.'" may , at : am david. said... cory doctorow's facebook is worth much less to its users than search and email, but it keeps a larger share of the value reports on an experiment by: " economists erik brynjolfsson, felix eggers and avinash gannamaneni have published an nber paper (sci-hub mirror) detailing an experiment where they offered americans varying sums to give up facebook, and then used a less-rigorous means to estimate much much americans valued other kinds of online services: maps, webmail, search, etc. they concluded that % of facebook's users value the service at less than a dollar a month, and at $ /month, half of facebook's users would quit. search is the most valued online service -- the typical american won't give up search for less than $ , /year -- while social media is the least valuable ($ )." may , at : am david. said... "in , john gilmore famously said that “the internet interprets censorship as damage and routes around it.” that was technically true when he said it but only because the routing structure of the internet was so distributed. as centralization increases, the internet loses that robustness, and censorship by governments and companies becomes easier." from censorship in the age of large cloud providers by bruce schneier. i believe that current efforts to decentralize the web won't be successful for economic reasons. bruce adds another reason, because governments everywhere prefer that it be centralized. but that doesn't mean either of us think decentralizing the web isn't important. june , at : am david. said... in his must-read intel and the danger of integration, ben thompson describes how intel became the latest large "slow ai" to get trapped defending its margins: "the company’s integrated model resulted in incredible margins for years, and every time there was the possibility of a change in approach intel’s executives chose to keep those margins." actually, intel's slow ai forced them to make that choice, just as sun's did. june , at : pm david. said... rachel m cohen's has the new america foundation lost its way? lays out the way "slow ais" took over the new america foundation and led to the defenestration of the open markets team i discussed above. june , at : am david. said... perhaps it is simpler to say that intel…was disrupted by steven sinofsky is fascinating commentary on ben thompson's intel and the danger of integration: "disruption is never one feature, but full set of *assumptions* that go into a business." that is another way of putting the "slow ai" concept. july , at : am david. said... felix salmon's the false tale of amazon's industry-conquering juggernaut makes the case that amazon hasn't been as successful at disrupting established industries (except book publishing) as the stock market thinks. but that doesn't mean startups shouldn't be very afraid; because amazon owns their computing infrastructure it has good visibility into which startups are worth buying or nuking. july , at : pm david. said... the eu fining google $ b elicits this response from duckduckgo: "up until just last year, it was impossible to add duckduckgo to chrome on android, and it is still impossible on chrome on ios. we are also not included in the default list of search options like we are in safari, even though we are among the top search engines in many countries. the google search widget is featured prominently on most android builds and is impossible to change the search provider. for a long time it was also impossible to even remove this widget without installing a launcher that effectively changed the whole way the os works. their anti-competitive search behavior isn't limited to android. every time we update our chrome browser extension, all of our users are faced with an official-looking dialogue asking them if they'd like to revert their search settings and disable the entire extension. google also owns http://duck.com and points it directly at google search, which consistently confuses duckduckgo users." july , at : pm david. said... senator elizabeth warren's accountable capitalism act is a thoughtful attempt to re-program us companies' "slow ais". it would force companies with more than $ b in revenue to obtain a federal rather than a state charter, eliminating the race to the bottom among states to charter companies. % of these companies' boards would have to be elected by workers. directors and officers could not sell shares within years of acquiring them, nor within years of any stock buyback. any political expenditures would require % votes of directors and shareholders. and, most importantly, it would permit the federal government to revoke a company's charter for "a history of egregious and repeated illegal conduct and has failed to take meaningful steps to address its problems" such as those, for example, at wells fargo. states (cough, cough, delaware) would never revoke a large company's charter for fear of sparking an exodus. hat tip "undercover blue" at digby's hullabaloo. august , at : am david. said... catalin cimpanu's senator blasts ftc for failing to crack down on google's ad fraud problems reports on senator mark warner third letter to the ftc about google and ad fraud: "google is directly profiting by letting ad fraud run rampant at the expense of the companies who buy or sell ads on its platform. however, warner is just as mad about the ftc as he is about google, claiming the ftc has failed to take action against the mountain view-based company for more than two years since he and new york democrat senator chuck schumer first wrote the agency about google's ad fraud problem." december , at : pm david. said... karl bode's facebook's use of smear merchants is the norm, not the exception is an example of facebook's "slow ai" in action, and how typical it is of oligopolists: " while facebook's decision to smear critics instead of owning their own obvious dysfunction is clearly idiotic, much of the backlash has operated under the odd belief that facebook's behavior is some kind of exception, not the norm. countless companies employ think tanks, consultants, bogus news ops, pr firms, academics, and countless other organizations to spread falsehoods, pollute the public discourse, and smear their critics on a daily basis. it's a massive industry. just ask the telecom sector. in the last decade alone broadband providers and firms far worse than definers have been caught paying minority groups to generate bunk support for bad policy, hijacking consumer identities to support bad policy, creating bogus consumer groups to generate fake support for bad policy, flooding the news wires endlessly with misleading op/eds without disclosing financial conflicts of interest, stocking public meetings with cardboard cutouts (so real people can't attend), or filling news comments sections and social media with bullshit criticism of corporate critics." december , at : pm david. said... roger mcnamee has expanded the thoughts referenced above into a book, zucked: waking up to the facebook catastrophe. the ideas laid out above are echoed in an extract published in time. january , at : pm david. said... jack balkin: "has been developing a theory of “information fiduciaries” for the past five years or so. the theory is motivated by the observation that ordinary people are enormously vulnerable to and dependent on the leading online platforms—facebook, google, twitter, uber, and the like. to mitigate this vulnerability and ensure these companies do not betray the trust people place in them, balkin urges that we draw on principles of fiduciary obligation." lina kahn and david pozen beg to differ. they seek to: "disrupt the emerging consensus by identifying a number of lurking tensions and ambiguities in the theory of information fiduciaries, as well as a number of reasons to doubt the theory’s capacity to resolve them satisfactorily. although we agree with balkin that the harms stemming from dominant online platforms call for legal intervention, we question whether the concept of information fiduciaries is an adequate or apt response to the problems of information insecurity that he stresses, much less to more fundamental problems associated with outsized market share and business models built on pervasive surveillance." march , at : pm david. said... david z. morris reports that vitalik buterin is embracing a new role: political theorist: "one of my theses here is that the cypherpunks’ attempts to get into the money business forced them to realize some other things along the way. and [one of those things] is that money is a fundamentally social thing in a much deeper way than, say, two-party encrypted communication, ... you have to start thinking about governance, social contracts … common shared expectations in this community, how do changes get made, how do we decide how changes get made, how do we discuss things … these are all very political things." good to know. march , at : pm david. said... scott galloway's take on the death stars of technology is a good read: "they started out benign, a group committed to peace and prosperity in the galaxy. slowly, they aggregated power and, wanting to maintain that power, turned to the dark side. they built a weapon km in diameter, staffed by hundreds of thousands of personnel and robots. of course i’m talking about the ds- orbital battle station commanded by governor tarkin — and also amazon, apple, facebook, google, and microsoft. the death stars of our economy can use bundling and economic power to add services and features that can destroy a promising business, at the speed of light." september , at : am david. said... john ryan's why decentralize? is a useful overview with links to many views of the issues. but for me it is still too focused on technology, not on the economic reason the decentralized technologies of the internet got assembled into large centralized systems. october , at : am post a comment newer post older post home subscribe to: post comments (atom) blog rules posts and comments are copyright of their respective authors who, by posting or commenting, license their work under a creative commons attribution-share alike . united states license. off-topic or unsuitable comments will be deleted. dshr dshr in anwr recent comments full comments blog archive ► ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ▼ ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ▼ january ( ) magical thinking at the new york times herbert van de sompel's paul evan peters award lec... web advertising and the shark, revisited (and upda... not really decentralized after all the internet society takes on digital preservation it isn't about the technology the $ b joke digital preservation declaration of shared values meltdown & spectre the box conspiracy ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► february ( ) ► january ( ) ► ( ) ► december ( ) ► november ( ) ► october ( ) ► september ( ) ► july ( ) ► june ( ) ► february ( ) ► ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► march ( ) ► january ( ) ► ( ) ► december ( ) ► october ( ) ► september ( ) ► august ( ) ► july ( ) ► june ( ) ► may ( ) ► april ( ) lockss system has permission to collect, preserve, and serve this archival unit. simple theme. powered by blogger. reflection part : my second year at gitlab and on becoming senior again – learning (lib)tech skip to content learning (lib)tech stories from my life as a technologist menu about me about this blog contact me twitter github linkedin flickr rss reflection part : my second year at gitlab and on becoming senior again this reflection is a direct continuation of part of my time at gitlab so far. if you haven’t, please read the first part before beginning this one. becoming an engineer ( months) the more time i spent working in support, the more i realized that the job was much more technical than i originally thought it to be. for gitlab.com (saas product), we’re the administrators of the instance, so we do get a lot of procedural questions around people’s accounts, and things like dormant namespace requests. at the same time, the more we integrated and promoted some of our features, the more questions we got on them. some areas that i’m still very much a beginner with but we often help with are kubernetes and package registries (especially maven and gradle). on the flip side, i’ve become an expert in ci, api, project imports, and saml. the hardest problems are the ones that potentially involve all parts of the setup (docker, kubernetes, certificates, etc.) because it’s hard to pin down the problem. what’s more, some of the issues we have, we can’t use the “typical” workaround, because it would involve changing the configuration of the gitlab instance, which we can’t do for gitlab.com. when i started the job, i knew it’d be technical. it was supporting gitlab after all. however, as time went on, questions started getting more complex. more than once, i’ve asked the managers to dig into why we weren’t always meeting our performance goals. the most interesting part was we were supposedly not getting more customer tickets. and yet, others and i felt like we were spending more time on tickets than ever before. our theory is the complexity of tickets, though that’s hard to measure. honestly, if you read the “support agent” job description, it sounds a lot of like any other generic support job, but unlike most other larger organizations, we don’t do tiered support. meaning, everyone works on all tickets. if you can’t take a ticket further, it is your responsibility to find someone who is willing to take it on. we encourage team members to find someone to work on it together, to learn how to troubleshoot or resolve similar issues in the future, but there’s no levels of support such that if it’s too technical for you, you’d push it up to the next tier. even as an “agent”, i was contributing code to gitlab projects. in recognition of all this, my manager worked hard to get approval and do all the work necessary to deprecate our position. this meant converting all our existing “support agents” to “support engineers” and that we would no longer hire for the agent role (which was actually really good, because we had too many applications who weren’t technical enough). that whole process was even more amazing to me than my promotion. having come from working in institutions with unions, i was skeptical that it could happen, let alone in a matter of months. in december , all the support agents officially became support engineers. becoming senior again ( years) my one regret with the move to engineer was that i lost the “senior” part of my title. the way it was mapped over was that a senior agent became an intermediate engineer, so my title became simply “support engineer”. it was a badge of honour, a recognition of the level of work, an indicator to others that they can rely on me to help, and more. nevertheless, my job itself didn’t really change. i kept doing what i was doing before and continued to try to improve myself, the team, and the company in whatever ways i could. while i became an expert in some topics, i wanted to make sure that the knowledge i gained wasn’t “bottled up”, but that everyone (including our users) could benefit from it. for the most part, i update the docs but sometimes a walkthrough is more useful so i’ll do a demo video. we have a joint okr (objective and key result) with docs this quarter, so based on everything i knew i put in a review merge request which in some ways probably went a little beyond our okr (since we weren’t supposed to be creating new pages). at the team level, i started getting more involved with the hiring process again (which was somewhat paused for me in making the transition to engineer). jerome uy, another team member, and i also took on revising the support portal (again). this time, we had a manager’s backing to help purchase a theme we could edit for our own use (thanks tom atkins). jerome did most of the coding work while i helped to compile the content we would use and the requirements (like changing the search bar to search our docs instead of the zendesk articles). in the end, we had a lot of content and didn’t want to overload the change, so we kept the content mostly the same as a first iteration. portal revision portal revision last year, when i started to feel like i didn’t know everyone on the team anymore, i started up a pairing challenge to make a conscious effort to get to know everyone on the team, and encourage everyone else to do the same by making a template. admittedly, even waking up early or staying late, i wouldn’t have completed it without having spent weeks in europe back in january. still, it was a great accomplishment that i fully completed earlier this year, and i’m still keeping it up with new support team members. at the company level, sometimes i wonder how much of an impact i have, but i do what i can. i was honoured to be part of the women of gitlab fireside chat at this year’s gitlab contribute. what’s a senior? shortly after the whole “agent” to “engineer” transition completed, my manager suggested we write up and submit a promotion document to get me promoted to senior again. i was super excited partly because we often see members with code-oriented contributions promoted and i didn’t think i was “qualified” to become one (thank imposter syndrome for that). in our discussions, he re-iterated that he considered what i do to be what a “senior” does. while working on my first promotion, we discussed how a senior is primarily distinguished from an intermediate by their focus. a junior or intermediate team member focuses on their work and getting good at it. on a side note, this isn’t a bad thing, and team members can be very successful at that level. a senior focuses on the team. in many ways, they do a lot of the same work. however, if answering tickets for example, instead of just answering a ticket, a senior will consider what they can do to radiate that knowledge they gained. generally, the default is a docs update, but if it’s not appropriate for a docs update or issue, they might still share it during a team meeting. generally, a senior will also think about how they can improve the team’s work, which can be manifested in many different ways. for non-technical contributions, it might involve helping to hire and train team members, improve or create processes, help with strategic work (such as a quarterly okr (objective and key result), or lead projects. even technical contributions, aside from helping improve the gitlab products, might be to improve the team efficacy. we’ve had team members build zendesk apps, browser plugins, and information gathering tools for troubleshooting. when you look over the engineering career development page, this starts to make even more sense. once at the senior level, you typically start thinking about whether you want to become “staff” or a manager (not that moving up is required), but to get to the senior level, you could be on the way to one or the other. in may, i became a senior again, this time as a senior support engineer. i've been promoted! first application focused senior support engineer @gitlab =d thanks to @lkozloff for seeing that i've been working at the senior level and getting the ball rolling, and ronnie (my current manager) for pushing the process through! plus #gitlab for its #values — cynthia "arty" ng (@therealarty) may , managerial change during this time, my then manager was making the transition to senior manager, and i was going to be moved over to a new manager. honestly, this news was met with some trepidation. i had had the same manager since i started at gitlab; and honestly, he was the best manager i had ever had. to be clear, this is not to say my previous managers were bad. instead, lyle has had a huge impact on my growth at gitlab. without his support and advice, i wouldn’t have as much of a positive impact as i do today. though not as directly, he continues to support and advise many of his previous direct reports, including myself, as our senior manager. thankfully, lyle put a lot of thought into the decision into who was moving over to which new manager. as part of the move, he was also moving any of the operation work that he did to my new manager. to give the new manager time to learn and understand how all that worked, only of us moved to our new manager. not only that, my one teammate in americas west working on gitlab.com was moving with me so that we would both continue to report to the same manager. keeping us together makes sense because we were both part of the early batch of gitlab.com hires ( of the left) and have a lot of “institutional knowledge”. having the same manager meant it was easier to coordinate some of our efforts. also, if we both raised the same concerns, our manager would know. i’m not sure that we “immediately hit it off”, but we certainly got along, and the more weeks that went by, the more i began to appreciate lyle’s decision. ronnie was open and friendly from the beginning, and the more we bonded over many shared stories and random food stuff (like furikake and bánh mì, in which i bought the book he recommended for pomax), the more i became willing to bring up the “hard stuff”. when my promotion document got a bit “stuck”, he took it upon himself to meet with me to finish it up and get it pushed through. when i was stressing out about work, in addition to helping me sort out my priorities, he set aside time during our one-on-one just to make sure i booked a day off. contribution graph from year showing more contributions starting january you can see that in recent months, i have empty contribution days where i obviously took a day off, and that the days i am working, there are more contributions than before; taking days off so that i can be more effective when i am working. the longer i’m at gitlab, the more i get pinged on things and the more responsibilities i have. in the past, i’ve always done a good job of prioritizing, but i’ve never been in support where there’s a constant balance between tickets and the rest of the work that needs to get done. having a manager who seems to know the right things to say and to help you tone down the stress level is invaluable. the support team i talk a lot about the support team and the way we work in my series on the gitlab values and culture, but that focuses on how we do our work. here, i have to express how working with the support team we have at gitlab is one of the big reasons that continuously makes me want to stay. every team member is smart, dedicated, helpful, and friendly. some team members seem to tend to keep to themselves more, but if you do a pairing session with them, it’s obvious that they’re simply “quieter” than others. most engineering teams are a dozen members or less, so support is by far the largest engineer team with over members, and one of the largest teams at gitlab. while there have certainly been some “growing pains”, most of them have been fairly typical. these include making sure everyone is aware of changes, and that we create or update processes that are scalable. communication and change management at scale are not new problems, so were fairly expected. one of the things i love is that we have this sense of belonging and camaraderie in support of being “one team”. despite the fact that we currently have two groups of people working on the two products, and we’re divided up by regions, when asked about splitting up for team dinner at contribute, the general consensus was to have everyone together. we frequently discuss making sure we’re not silo’ed by region. we are one team. we share our shortcomings and our successes together. while we’re not there yet, we’re also moving towards working as a single team even in tickets. nevertheless, we often exceed our performance goals, and consistently receive not only a high rate of positive satisfaction ratings, but also a high rate of users completing them. we’ve received praises from our executive team. in recognition of our awesome work, we’ve also received a pizza party or two. final thoughts a lot of what i got done and so much of what i’ve learnt was thanks to help from others: their feedback, knowledge, trust, and support, plus allowing me to not step on their toes. at years, it’s currently hard to imagine working anywhere else. thank you gitlab and the support team, with special thanks to my awesome managers. here’s to more amazing things in year ! thanks for reading! if you made it all the way to end, just want to say, thanks for reading! have a red panda as a reward =d i’m excited to keep exploring how i can continue to make a positive impact at gitlab. out for a morning walk.#mkezoo #redpanda pic.twitter.com/i s hmzncq — milwaukee county zoo (@milwaukeecozoo) june , twitter linkedin pocket more facebook tumblr telegram email print like this: like loading... author: cynthia technologist, librarian, metadata and technical services expert, educator, mentor, web developer, uxer, accessibility advocate, documentarian view all posts by cynthia author cynthiaposted on june , january , categories update, work culturetags gitlab, organizational culture, reflection leave a comment cancel reply enter your comment here... please log in using one of these methods to post your comment: email (required) (address never made public) name (required) website you are commenting using your wordpress.com account. ( log out / change ) you are commenting using your google account. ( log out / change ) you are commenting using your twitter account. ( log out / change ) you are commenting using your facebook account. ( log out / change ) cancel connecting to %s notify me of new posts via email. this site uses akismet to reduce spam. learn how your comment data is processed. post navigation previous previous post: reflection part : my first year at gitlab and becoming senior next next post: prioritization in support: tickets, slack, issues, and more cynthia technologist, librarian, metadata and technical services expert, educator, mentor, web developer, uxer, accessibility advocate, documentarian view full profile → follow us twitter linkedin github telegram search for: search categories events librarianship library academic public special tours methodology project work technology tools update web design work culture follow via email enter your email address to receive notifications of new posts by email. email address: follow about me about this blog contact me twitter github linkedin flickr rss learning (lib)tech add your thoughts here... (optional) post to cancel send to email address your name your email address cancel post was not sent - check your email addresses! email check failed, please try again sorry, your blog cannot share posts by email. %d bloggers like this: certbot a project of the electronic frontier foundation certbot instructions about certbot contribute to certbot hosting providers with https get help donate donate to eff ≡ home certbot instructions about certbot contribute to certbot hosting providers with https get help donate × get your site on https:// find out if your hosting provider has https built in — no certbot needed. see the list of providers see if your hosting provider offers https. or, run certbot once to automatically get free https certificates forever. get certbot instructions or, get instructions for certbot. what's your http website running on? my http website is running software apache nginx haproxy plesk web hosting product none of the above on system web hosting service snapd debian (stretch) debian (buster) debian testing/unstable ubuntu . ubuntu . ubuntu . lts (bionic) ubuntu . (xenial) gentoo arch linux fedora centos/rhel centos/rhel freebsd openbsd . + macos devuan ascii . devuan beowulf . devuan testing/unstable opensuse tumbleweed opensuse leap . other linux windows help, i'm not sure! no javascript? see all setup instructions here. read the full documentation here. use our instruction generator to find custom commands to get certbot on your server's environment. pick your server's software and system above. to use certbot, you'll need... comfort with the command line command line a command line is a way of interacting with a computer by typing text-based commands to it and receiving text-based replies. certbot is run from a command-line interface, usually on a unix-like server. in order to use certbot for most purposes, you’ll need to be able to install and run it on the command line of your web server, which is usually accessed over ssh. a command line is a way of interacting with a computer by typing text-based commands to it and receiving text-based replies. certbot is run from a command-line interface, usually on a unix-like server. in order to use certbot for most purposes, you’ll need to be able to install and run it on the command line of your web server, which is usually accessed over ssh. from our certbot glossary command line command line a command line is a way of interacting with a computer by typing text-based commands to it and receiving text-based replies. certbot is run from a command-line interface, usually on a unix-like server. in order to use certbot for most purposes, you’ll need to be able to install and run it on the command line of your web server, which is usually accessed over ssh. from our certbot glossary ...and an http website http http (hypertext transfer protocol) is the traditional, but insecure, method for web browsers to request the content of web pages and other online resources from web servers. it is an internet standard and normally used with tcp port . almost all websites in the world support http, but websites that have been configured with certbot or some other method of setting up https may automatically redirect users from the http version of the site to the https version. http (hypertext transfer protocol) is the traditional, but insecure, method for web browsers to request the content of web pages and other online resources from web servers. it is an internet standard and normally used with tcp port . almost all websites in the world support http, but websites that have been configured with certbot or some other method of setting up https may automatically redirect users from the http version of the site to the https version. from our certbot glossary http website http http (hypertext transfer protocol) is the traditional, but insecure, method for web browsers to request the content of web pages and other online resources from web servers. it is an internet standard and normally used with tcp port . almost all websites in the world support http, but websites that have been configured with certbot or some other method of setting up https may automatically redirect users from the http version of the site to the https version. from our certbot glossary that is already online website that’s already online certbot is usually meant to be used to switch an existing http site to work in https (and, afterward, to continue renewing the site’s https certificates whenever necessary). some certbot documentation assumes or recommends that you have a working web site that can already be accessed using http on port . that means, for example, that if you use a web browser to go to your domain using http://, your web server answers and some kind of content comes up (even if it’s just a default welcome page rather than the final version of your site). some methods of using certbot have this as a prerequisite, so you’ll have a smoother experience if you already have a site set up with http. (if your site can’t be accessed this way as a matter of policy, you’ll probably need to use dns validation in order to get a certificate with certbot.) certbot is usually meant to be used to switch an existing http site to work in https (and, afterward, to continue renewing the site’s https certificates whenever necessary). some certbot documentation assumes or recommends that you have a working web site that can already be accessed using http on port . that means, for example, that if you use a web browser to go to your domain using http://, your web server answers and some kind of content comes up (even if it’s just a default welcome page rather than the final version of your site). some methods of using... from our certbot glossary already online website that’s already online certbot is usually meant to be used to switch an existing http site to work in https (and, afterward, to continue renewing the site’s https certificates whenever necessary). some certbot documentation assumes or recommends that you have a working web site that can already be accessed using http on port . that means, for example, that if you use a web browser to go to your domain using http://, your web server answers and some kind of content comes up (even if it’s just a default welcome page rather than the final version of your site). some methods of using certbot have this as a prerequisite, so you’ll have a smoother experience if you already have a site set up with http. (if your site can’t be accessed this way as a matter of policy, you’ll probably need to use dns validation in order to get a certificate with certbot.) from our certbot glossary with an open port port different internet services are distinguished by using different tcp port numbers. unencrypted http normally uses tcp port , while encrypted https normally uses tcp port . to use certbot –webroot, certbot –apache, or certbot –nginx, you should have an existing http website that’s already online hosted on the server where you’re going to use certbot. this site should be available to the rest of the internet on port . to use certbot –standalone, you don’t need an existing site, but you have to make sure connections to port on your server are not blocked by a firewall, including a firewall that may be run by your internet service provider or web hosting provider. please check with your isp or hosting provider if you’re not sure. (using dns validation does not require let’s encrypt to make any inbound connection to your server, so with this method in particular it’s not necessary to have an existing http website or the ability to receive connections on port .) different internet services are distinguished by using different tcp port numbers. unencrypted http normally uses tcp port , while encrypted https normally uses tcp port . to use certbot –webroot, certbot –apache, or certbot –nginx, you should have an existing http website that’s already online hosted on the server where you’re going to use certbot. this site should be available to the rest of the internet on port . to use certbot –standalone, you don’t need an existing site, but you have to make sure connections to port on your server are not blocked by a firewall, including a... from our certbot glossary port port different internet services are distinguished by using different tcp port numbers. unencrypted http normally uses tcp port , while encrypted https normally uses tcp port . to use certbot –webroot, certbot –apache, or certbot –nginx, you should have an existing http website that’s already online hosted on the server where you’re going to use certbot. this site should be available to the rest of the internet on port . to use certbot –standalone, you don’t need an existing site, but you have to make sure connections to port on your server are not blocked by a firewall, including a firewall that may be run by your internet service provider or web hosting provider. please check with your isp or hosting provider if you’re not sure. (using dns validation does not require let’s encrypt to make any inbound connection to your server, so with this method in particular it’s not necessary to have an existing http website or the ability to receive connections on port .) from our certbot glossary ...which is hosted on a server server a server is a computer on the internet that provides a service, like a web site or an email service. most web site owners pay a hosting provider for the use of a server located in a data center and administered over the internet. this might be a physical dedicated server, a virtual private server (vps), or a shared server. other servers provide other parts of the internet infrastructure, such as dns servers. a server is a computer on the internet that provides a service, like a web site or an email service. most web site owners pay a hosting provider for the use of a server located in a data center and administered over the internet. this might be a physical dedicated server, a virtual private server (vps), or a shared server. other servers provide other parts of the internet infrastructure, such as dns servers. from our certbot glossary server server a server is a computer on the internet that provides a service, like a web site or an email service. most web site owners pay a hosting provider for the use of a server located in a data center and administered over the internet. this might be a physical dedicated server, a virtual private server (vps), or a shared server. other servers provide other parts of the internet infrastructure, such as dns servers. from our certbot glossary which you can access via ssh ssh ssh (which stands for “secure shell”) is a technology for connecting to a remote server and accessing a command line on that server, often in order to administer it. the administrator of a server can grant ssh access to others, and can also use ssh access directly in order to administer the server remotely. ssh is usually used to access servers running unix-like operating systems, but your own computer doesn’t have to be running unix in order to use ssh. you normally use ssh from your computer’s command line in a terminal by typing a command such as ssh username@example.com, especially if your own computer runs linux or macos. after logging in, you’ll have access to the server’s command line. if you use windows on your computer, you might also use a dedicated ssh application such as putty. most certbot users run certbot from a command prompt on a remote server over ssh. ssh (which stands for “secure shell”) is a technology for connecting to a remote server and accessing a command line on that server, often in order to administer it. the administrator of a server can grant ssh access to others, and can also use ssh access directly in order to administer the server remotely. ssh is usually used to access servers running unix-like operating systems, but your own computer doesn’t have to be running unix in order to use ssh. you normally use ssh from your computer’s command line in a terminal by typing a command such as ssh username@example.com,... from our certbot glossary ssh ssh ssh (which stands for “secure shell”) is a technology for connecting to a remote server and accessing a command line on that server, often in order to administer it. the administrator of a server can grant ssh access to others, and can also use ssh access directly in order to administer the server remotely. ssh is usually used to access servers running unix-like operating systems, but your own computer doesn’t have to be running unix in order to use ssh. you normally use ssh from your computer’s command line in a terminal by typing a command such as ssh username@example.com, especially if your own computer runs linux or macos. after logging in, you’ll have access to the server’s command line. if you use windows on your computer, you might also use a dedicated ssh application such as putty. most certbot users run certbot from a command prompt on a remote server over ssh. from our certbot glossary with the ability to sudo sudo sudo is the most common command on unix-like operating systems to run a specific command as root (the system administrator). if you’re logged in to your server as a user other than root, you’ll likely need to put sudo before your certbot commands so that they run as root (for example, sudo certbot instead of just certbot), especially if you’re using certbot’s integration with a web server like apache or nginx. (the certbot-auto script automatically runs sudo if it’s necessary and you didn’t specify it.) sudo is the most common command on unix-like operating systems to run a specific command as root (the system administrator). if you’re logged in to your server as a user other than root, you’ll likely need to put sudo before your certbot commands so that they run as root (for example, sudo certbot instead of just certbot), especially if you’re using certbot’s integration with a web server like apache or nginx. (the certbot-auto script automatically runs sudo if it’s necessary and you didn’t specify it.) from our certbot glossary sudo sudo sudo is the most common command on unix-like operating systems to run a specific command as root (the system administrator). if you’re logged in to your server as a user other than root, you’ll likely need to put sudo before your certbot commands so that they run as root (for example, sudo certbot instead of just certbot), especially if you’re using certbot’s integration with a web server like apache or nginx. (the certbot-auto script automatically runs sudo if it’s necessary and you didn’t specify it.) from our certbot glossary optional if you want a wildcard cert wildcard certificate a wildcard certificate is a certificate that includes one or more names starting with *.. browsers will accept any label in place of the asterisk (*). for example, a certificate for *.example.com will be valid for www.example.com, mail.example.com, hello.example.com, and goodbye.example.com. however, a wildcard certificate including only the name *.example.com will not be valid for example.com: the substituted label can not be empty. if you want the certificate to be valid for example.com, you also need to include example.com (i.e. without the *. part) on the certificate. additionally, the asterisk can only be substituted by a single label and not by multiple labels. for example, the name hello.goodbye.example.com will not be covered by a certificate including only the name *.example.com. it will be covered however, by *.goodbye.example.com. note that a wildcard name can not contain multiple asterisks. for example, *.*.example.com is not valid. a wildcard certificate is a certificate that includes one or more names starting with *.. browsers will accept any label in place of the asterisk (*). for example, a certificate for *.example.com will be valid for www.example.com, mail.example.com, hello.example.com, and goodbye.example.com. however, a wildcard certificate including only the name *.example.com will not be valid for example.com: the substituted label can not be empty. if you want the certificate to be valid for example.com, you also need to include example.com (i.e. without the *. part) on the certificate. additionally, the asterisk can only be substituted by a single label and not... from our certbot glossary wildcard cert wildcard certificate a wildcard certificate is a certificate that includes one or more names starting with *.. browsers will accept any label in place of the asterisk (*). for example, a certificate for *.example.com will be valid for www.example.com, mail.example.com, hello.example.com, and goodbye.example.com. however, a wildcard certificate including only the name *.example.com will not be valid for example.com: the substituted label can not be empty. if you want the certificate to be valid for example.com, you also need to include example.com (i.e. without the *. part) on the certificate. additionally, the asterisk can only be substituted by a single label and not by multiple labels. for example, the name hello.goodbye.example.com will not be covered by a certificate including only the name *.example.com. it will be covered however, by *.goodbye.example.com. note that a wildcard name can not contain multiple asterisks. for example, *.*.example.com is not valid. from our certbot glossary : dns credentials dns credentials dns credentials are a password or other kind of secret (such as an api key) that your dns provider lets you use to change the contents of your dns records. they are usually issued by your domain registrar (or by another dns provider, if your dns provider isn’t the same as your registrar). dns credentials are a sensitive kind of secret because they can be used to take over your site completely. you should never share these credentials publicly or with an unauthorized person. it can be ok to provide a copy of them to certbot to let it perform dns validation automatically, since it runs locally on your machine. dns credentials are a password or other kind of secret (such as an api key) that your dns provider lets you use to change the contents of your dns records. they are usually issued by your domain registrar (or by another dns provider, if your dns provider isn’t the same as your registrar). dns credentials are a sensitive kind of secret because they can be used to take over your site completely. you should never share these credentials publicly or with an unauthorized person. it can be ok to provide a copy of them to certbot to let it perform... from our certbot glossary dns credentials dns credentials dns credentials are a password or other kind of secret (such as an api key) that your dns provider lets you use to change the contents of your dns records. they are usually issued by your domain registrar (or by another dns provider, if your dns provider isn’t the same as your registrar). dns credentials are a sensitive kind of secret because they can be used to take over your site completely. you should never share these credentials publicly or with an unauthorized person. it can be ok to provide a copy of them to certbot to let it perform dns validation automatically, since it runs locally on your machine. from our certbot glossary don't have these requirements? not to worry! some hosting providers automate the https process. see the full list of hosting providers, or find out more about how to set up your system. want to learn more? see our explainers want to contribute to certbot? see opportunites to help certbot home about certbot certbot instructions hosting providers with https get help frequently asked questions certbot glossary certbot documentation community forum contact us contribute to certbot donate to eff privacy search | clir | aviary dlf forum program ndsa digipres program for program login resources collections about sign in × email password remember me forgot password system will deduct the location before submission. don't have an account? join now! join now × sign up for aviary! experience a new way of accessing, browsing and displaying av collections as a user or member of an organization. you must register first as an individual to create an organization account. first name last name email password password confirmation password must contain: at least uppercase letter at least lowercase letter at least one number at least characters at least special character (.@$!%*?&) can only have special character (.@$!%*?&) password must match: password and password confirmation must match username preferred language preferred language abkhazian afar afrikaans akan albanian amharic arabic aragonese armenian assamese avaric avestan aymara azerbaijani bambara bashkir basque belarusian bengali bihari languages bislama bosnian breton bulgarian burmese catalan, valencian central khmer chamorro chechen chichewa, chewa, nyanja chinese chinese (traditional) church slavonic, old bulgarian, old church slavonic chuvash cornish corsican cree croatian czech danish divehi, dhivehi, maldivian dutch, flemish dzongkha english esperanto estonian ewe faroese fijian finnish french fulah gaelic, scottish gaelic galician ganda georgian german gikuyu, kikuyu greek (modern) greenlandic, kalaallisut guarani gujarati haitian, haitian creole hausa hebrew herero hindi hiri motu hungarian icelandic ido igbo indonesian interlingua (international auxiliary language association) interlingue inuktitut inupiaq irish italian japanese javanese kannada kanuri kashmiri kazakh kinyarwanda komi kongo korean kwanyama, kuanyama kurdish kyrgyz lao latin latvian letzeburgesch, luxembourgish limburgish, limburgan, limburger lingala lithuanian luba-katanga macedonian malagasy malay malayalam maltese manx maori marathi marshallese moldovan, moldavian, romanian mongolian nauru navajo, navaho northern ndebele ndonga nepali northern sami norwegian norwegian bokmål norwegian nynorsk nuosu, sichuan yi occitan (post ) ojibwa oriya oromo ossetian, ossetic pali panjabi, punjabi pashto, pushto persian polish portuguese quechua romansh rundi russian samoan sango sanskrit sardinian serbian shona sindhi sinhala, sinhalese slovak slovenian somali sotho, southern south ndebele spanish spanish (latin america) sundanese swahili swati swedish tagalog tahitian tajik tamil tatar telugu thai tibetan tigrinya tonga (tonga islands) tsonga tswana turkish turkmen twi uighur, uyghur ukrainian urdu uzbek venda vietnamese volap_k walloon welsh western frisian wolof xhosa yiddish yoruba zhuang, chuang zulu system will deduct the location before submission. i agree to the terms of service have an account? sign in forgot password × please enter the email you used to register and we will email you a link to reset your password. have an account? sign in … search in any field resource name resource description indexes transcript collection title search add new criteria sort results by relevance title (z-a) title (a-z) date added clear search terms and or not any field resource name resource description indexes transcript collection title contains ends with starts with exact english russian hebrew arabic assamese burmese chinese czech farsi french georgian german gilaki greek hindi italian japanese kannada korean nigerian polish spanish swedish thai turkish ukrainian urdu simple search advanced search | clear search terms | start over filter reset collection title dlf forum ndsa digital preservation dlf working groups for events how-to date - date format must be yyyy-mm-dd - to select a range, use space-hyphen-space " - " between two valid dates, e.g., - - - - - . date range begin resource language english primary has transcript yes has index no agent courtney mumma alex kinnaman tricia patterson vicky steeves alex green amy rudersdorf andrew berger andrew french angela fritz ashley blewer bethany scott bradley daigle brenda burk carol kussmann christopher prom corey davis david cirella david underdown david wilcox dean smith deb verhoff elizabeth england eric kaltman felicity dykas grant hurley greer martin grete graf hannah merwood james tuttle jes neal jessica knight julia kim karl-rainer blumenthal krista oldham lauren work leah prescott leslie johnston mike shallcross nathan tallman nick krabbenhoeft peggy griesinger ruby lorraine martinez sally debauche shira peltzman sibyl schaefer stephen abrams winnie schwaid-lindner yinlin chen zhiwu xie access public resource duration duration range begin to duration range end h m to h m sort by: relevance relevance title (z-a) title (a-z) date added - of search constraints search results but don't panic! you may be early - the video will be available at the scheduled time. be sure to check the schedule. think you're in the right spot at the right time? be sure you're logged in. if all else fails, contact forum@diglib.org or the #dlf-village channel on slack. request access : : digipres closing plenary clir ndsa digital preservation agent: tricia patterson ...more agent: tricia patterson date: - - ...more date: - - but don't panic! you may be early - the video will be available at the scheduled time. be sure to check the schedule. think you're in the right spot at the right time? be sure you're logged in. if all else fails, contact forum@diglib.org or the #dlf-village channel on slack. request access : : digipres lightning talks clir ndsa digital preservation agent: jessica knight, alex kinnaman, angela fritz, greer martin, andrew french, david wilcox, dean smith, david cirella, grete graf, winnie schwaid-lindner, andrew berger, ashley blewer, eric kaltman, nick krabbenhoeft, vicky steeves, david cirella ...more agent: jessica knight, alex kinnaman, angela fritz, greer martin, andrew french, david wilcox, dean smith, david cirella, grete graf, winnie schwaid-lindner, andrew berger, ashley blewer, eric kaltman, nick krabbenhoeft, vicky steeves, david cirella date: - - ...more date: - - but don't panic! you may be early - the video will be available at the scheduled time. be sure to check the schedule. think you're in the right spot at the right time? be sure you're logged in. if all else fails, contact forum@diglib.org or the #dlf-village channel on slack. request access : : combo session: #sustainability #technology clir ndsa digital preservation agent: yinlin chen, james tuttle, alex kinnaman, bethany scott, zhiwu xie ...more agent: yinlin chen, james tuttle, alex kinnaman, bethany scott, zhiwu xie date: - - ...more date: - - but don't panic! you may be early - the video will be available at the scheduled time. be sure to check the schedule. think you're in the right spot at the right time? be sure you're logged in. if all else fails, contact forum@diglib.org or the #dlf-village channel on slack. request access : : panel: #assessment #action the right stuff (over time): collaborative approaches to selection for digital preservation clir ndsa digital preservation agent: lauren work, nathan tallman, mike shallcross, courtney mumma ...more agent: lauren work, nathan tallman, mike shallcross, courtney mumma date: - - ...more date: - - but don't panic! you may be early - the video will be available at the scheduled time. be sure to check the schedule. think you're in the right spot at the right time? be sure you're logged in. if all else fails, contact forum@diglib.org or the #dlf-village channel on slack. request access : : panel: #sustainability #technology thinking outside the inbox: new developments in email archiving clir ndsa digital preservation agent: tricia patterson, sally debauche, christopher prom, ruby lorraine martinez ...more agent: tricia patterson, sally debauche, christopher prom, ruby lorraine martinez date: - - ...more date: - - but don't panic! you may be early - the video will be available at the scheduled time. be sure to check the schedule. think you're in the right spot at the right time? be sure you're logged in. if all else fails, contact forum@diglib.org or the #dlf-village channel on slack. request access : : panel: #policy #practice what’s wrong with digital stewardship: evaluating the organization of digital preservation programs from practitioners’ perspectives clir ndsa digital preservation agent: karl-rainer blumenthal, peggy griesinger, julia kim, shira peltzman, vicky steeves ...more agent: karl-rainer blumenthal, peggy griesinger, julia kim, shira peltzman, vicky steeves date: - - ...more date: - - but don't panic! you may be early - the video will be available at the scheduled time. be sure to check the schedule. think you're in the right spot at the right time? be sure you're logged in. if all else fails, contact forum@diglib.org or the #dlf-village channel on slack. request access : : combo session: #policy #practice clir ndsa digital preservation agent: amy rudersdorf, stephen abrams, sibyl schaefer, courtney mumma ...more agent: amy rudersdorf, stephen abrams, sibyl schaefer, courtney mumma date: - - ...more date: - - but don't panic! you may be early - the video will be available at the scheduled time. be sure to check the schedule. think you're in the right spot at the right time? be sure you're logged in. if all else fails, contact forum@diglib.org or the #dlf-village channel on slack. request access : : combo session: #assessment #action clir ndsa digital preservation agent: elizabeth england, leslie johnston, david underdown, hannah merwood, alex green, grant hurley, corey davis ...more agent: elizabeth england, leslie johnston, david underdown, hannah merwood, alex green, grant hurley, corey davis date: - - ...more date: - - but don't panic! you may be early - the video will be available at the scheduled time. be sure to check the schedule. think you're in the right spot at the right time? be sure you're logged in. if all else fails, contact forum@diglib.org or the #dlf-village channel on slack. request access : : ndsa innovation awards clir ndsa digital preservation date: - - ...more date: - - but don't panic! you may be early - the video will be available at the scheduled time. be sure to check the schedule. think you're in the right spot at the right time? be sure you're logged in. if all else fails, contact forum@diglib.org or the #dlf-village channel on slack. request access : : get active with the ndsa brunch clir ndsa digital preservation agent: courtney mumma, krista oldham, bradley daigle, leah prescott, jes neal, carol kussmann, deb verhoff, brenda burk, felicity dykas ...more agent: courtney mumma, krista oldham, bradley daigle, leah prescott, jes neal, carol kussmann, deb verhoff, brenda burk, felicity dykas date: - - ...more date: - - but don't panic! you may be early - the video will be available at the scheduled time. be sure to check the schedule. think you're in the right spot at the right time? be sure you're logged in. if all else fails, contact forum@diglib.org or the #dlf-village channel on slack. request access : : digipres opening plenary clir ndsa digital preservation date: - - ...more date: - - add to playlist - of bulk review select playlist selected resources id resource name close apply access request title reasons for requesting access reasons for requesting access is required. *required fields close access requested close ` clir council on library and information resources south clark street wework— th floor arlington, va unless otherwise indicated, content on this site is available for re-use under cc by-sa . license operation message × close close yes × no yes yes accessibility issues with this page? please click here to let us know. tweets to donald trump (@realdonaldtrump) - web archives for historical research group dataverse skip to main content toggle navigation search search find user guide support english english français log in please enable javascript in your browser. it is required to use most of the features of dataverse. web archives for historical research group dataverse scholars portal dataverse > web archives for historical research group dataverse > tweets to donald trump (@realdonaldtrump) version . ruest, nick, , "tweets to donald trump (@realdonaldtrump)", https://doi.org/ . /sp/ bavqm, scholars portal dataverse, v cite dataset endnote xml ris bibtex learn about data citation standards. access dataset download options download zip ( . gb) contact owner share dataset metrics , downloads description , , tweet ids for tweets directed at donald trump (@realdonaldtrump), collected with documenting the now's twarc. tweets can be “rehydrated” with documenting the now’s twarc, or hydrator. twarc hydrate to_realdonaldtrump_ _ids.txt > to_realdonaldtrump_ .jsonl. collection notes: tweets from may , - october , of the dataset used a combination of the filter (streaming) api and search api. the filter api failed on june , . from june , forward only the search api was used to collect. collection was done every days on a cron job, and periodically deduplicated. there is a data gap from tue jul : : + through thu aug : : + due to a collection error. this dataset also includes a number of derivative csv files from the original jsonl collected. this includes: a user csv file created with jq (see below). twut userinfo twut language twut times twut sources twut hashtags twut urls twut animatedgifurls twut imageurls twut mediaurls twut videourls user csv: jq -r '[.id_str, .created_at, .user.screen_name, .retweeted_status != null] | @csv' to_realdonaldtrump_ .jsonl > to_realdonaldtrump_ _users.jsonl ( - - ) subject computer and information science; social sciences keyword twitter, donald trump, politics, american politics ui-button files metadata terms versions search find filter by file type: all all archive ( ) unknown ( ) text ( ) access: all all public ( ) file tag: all all data ( ) sort name (a-z) name (z-a) newest oldest size type to of files download to_realdonaldtrump_ _ids.txt plain text - . gb - dec , - downloadsmd : beaec e fabf d f a a data download explore read text to_realdonaldtrump_ _ids.txt.tar.gz gzip archive - . mb - dec , - downloadsmd : ecde fa f d f de data download to_realdonaldtrump_ _ids.txt.gz gzip archive - . mb - feb , - downloadsmd : c e e e ed ba eced a b data download to_realdonaldtrump_ _ids.txt.gz gzip archive - . mb - jun , - downloadsmd : df b a e eae f dbe fe f data download to_realdonaldtrump_ _ids.txt.gz gzip archive - . mb - jul , - downloadsmd : b c ae a d d f data download to_realdonaldtrump_ _ids.txt.gz gzip archive - . mb - oct , - downloadsmd : c b f acd cfefeeb bada data download to_realdonaldtrump_ _ids.txt.gz gzip archive - . mb - feb , - downloadsmd : dc d da d eecaa cf b data download to_realdonaldtrump_ _users.csv.gz gzip archive - . gb - jan , - downloadsmd : e f b d f fb f d data download to_realdonaldtrump_ _ids.txt.gz gzip archive - . mb - may , - downloadsmd : cafa fb dec a e dda b dd data download to_realdonaldtrump_ _ids.txt.gz gzip archive - . gb - mar , - downloadsmd : e f cf a e a b d a data download fp nefiles per page rows per page export metadata dublin core ddi datacite ddi html codebook json oai_ore openaire schema.org json-ld citation metadata dataset persistent id doi: . /sp/ bavqm publication date - - title tweets to donald trump (@realdonaldtrump) author ruest, nick (york university) - orcid: - - - contact use email button above to contact. ruest, nick (york university) description , , tweet ids for tweets directed at donald trump (@realdonaldtrump), collected with documenting the now's twarc. tweets can be “rehydrated” with documenting the now’s twarc, or hydrator. twarc hydrate to_realdonaldtrump_ _ids.txt > to_realdonaldtrump_ .jsonl. collection notes: tweets from may , - october , of the dataset used a combination of the filter (streaming) api and search api. the filter api failed on june , . from june , forward only the search api was used to collect. collection was done every days on a cron job, and periodically deduplicated. there is a data gap from tue jul : : + through thu aug : : + due to a collection error. this dataset also includes a number of derivative csv files from the original jsonl collected. this includes: a user csv file created with jq (see below). twut userinfo twut language twut times twut sources twut hashtags twut urls twut animatedgifurls twut imageurls twut mediaurls twut videourls user csv: jq -r '[.id_str, .created_at, .user.screen_name, .retweeted_status != null] | @csv' to_realdonaldtrump_ .jsonl > to_realdonaldtrump_ _users.jsonl ( - - ) subject computer and information science; social sciences keyword twitter donald trump politics american politics grant information social sciences and humanities research council: - - depositor ruest, nick deposit date - - time period covered start: - - ; end: - - date of collection start: - - ; end: - - kind of data line oriented twitter ids software twarc, version: . . twut, version: . . jq, version: . terms of use waiver our community norms as well as good scientific practices expect that proper credit is given via citation. please use the data citation above, generated by the dataverse. no waiver has been selected for this dataset. terms of use creative commons — attribution . canada — cc by . ca https://creativecommons.org/licenses/by/ . /ca/ guestbook guestbook no guestbook is assigned to this dataset, you will not be prompted to provide any information on file download. view differences direct dataset summary contributors published no records found. share dataset share this dataset on your favorite social media networks. close dataset citations citations for this dataset are retrieved from crossref via datacite using make data count standards. for more information about dataset metrics, please refer to the user guide. sorry, no citations were found. close restricted files selected the restricted file(s) selected may not be downloaded because you have not been granted access. close restricted files selected the restricted file(s) selected may not be downloaded because you have not been granted access. click continue to download the files you have access to download. continue cancel restricted files selected the restricted file(s) selected may not be downloaded because you have not been granted access. click continue to download the files you have access to download. continue cancel delete dataset are you sure you want to delete this dataset and all of its files? you cannot undelete this dataset. continue cancel delete draft version are you sure you want to delete this draft version? files will be reverted to the most recently published version. you cannot undelete this draft. continue cancel unpublished dataset private url private url can only be used with unpublished versions of datasets. cancel unpublished dataset private url are you sure you want to disable the private url? if you have shared the private url with others they will no longer be able to use it to access your unpublished dataset. yes, disable private url cancel delete files the file(s) will be deleted after you click on the delete button. files will not be removed from previously published versions of the dataset. delete cancel compute this dataset contains restricted files you may not compute on because you have not been granted access. close deaccession dataset are you sure you want to deaccession? the selected version(s) will no longer be viewable by the public. no deaccession dataset are you sure you want to deaccession this dataset? it will no longer be viewable by the public. no version differences details please select two versions to view the differences. close version differences details version: last updated: version: last updated: done select file(s) please select a file or files to be downloaded. close select file(s) please select a file or files for access request. close select file(s) please select a file or files to be deleted. close select file(s) please select unrestricted file(s) to be restricted. close select file(s) please select restricted file(s) to be unrestricted. close select file(s) please select a file or files to be edited. close select file(s) please select a file or files to be edited. close edit tags select existing file tags or create new tags to describe your files. each file can have more than one tag. save changescancel request access you need to sign up or log in to request access to this file. close data not published in order to map your data with worldmap, your data must be published. please publish this dataset, then retry the map data feature. close continue cancel dataset terms please confirm and/or complete the information needed below in order to continue. terms of use creative commons — attribution . canada — cc by . ca https://creativecommons.org/licenses/by/ . /ca/ accept cancel preview guestbook upon downloading files the guestbook asks for the following information. guestbook name collected data account information close package file download use the download url in a wget command or a download manager to download this package file. download via web browser is not recommended. user guide - downloading a dataverse package via url - - : download url https://dataverse.scholarsportal.info/api/access/datafile/ close request access please confirm and/or complete the information needed below in order to request access to files in this dataset. terms of use creative commons — attribution . canada — cc by . ca https://creativecommons.org/licenses/by/ . /ca/ accept cancel compute batch clear batchui-button dataset dataset persistent id compute batch cancel file restrictions terms of access request access enable access request continue cancel submit for review you will not be able to make changes to this dataset while it is in review. submit cancel publish dataset are you sure you want to republish this dataset? select if this is a minor or major version update. minor release ( . ) major release ( . ) continue cancel publish dataset this dataset cannot be published until web archives for historical research group dataverse is published by its administrator. close publish dataset this dataset cannot be published until web archives for historical research group dataverse and scholars portal dataverse are published. close return to author return this dataset to contributor for modification. continue cancel copyright © powered by v. v . . -sp- contact scholars portal dataverse support to scholars portal dataverse support from subject message please fill this out to prove you are not a robot. + = send message cancel r developer page r developer page this site is intended as an intermediate repository for more or less finalized ideas and plans for the r statistical system. most parts of the site are open to the public, and we welcome discussions on the ideas, but please do not take them for more than that, in particular there is no commitment to actually carry out the plans in finite time unless expressedly stated. the site also includes a collection of technical papers about r and the r blog. for general information about r see the r project's homepage. this page (and most of its links) are updated hourly and so may not be absolutely current. the up-to-the-minute version can be seen at https://svn.r-project.org/r-dev-web/trunk/index.html. release plans version . . ("bunny-wunnies freak out") was released on - - . the wrap-up release of the r- . .x series was . . ("holding the windsock") on - - . the release of . . ("lost library book") is scheduled for monday - - . release candidate tarballs will be made available during the week up to the release. please refer to the generic checklist for details friday - - : start ( . . beta) monday - - : code freeze ( . . rc) monday - - : release ( . . ) the overall release schedule is to have annual x.y. releases in spring, with patch releases happening on an as-needed basis. it is intended to have a final patch release of the previous version shortly before the next x.y. release. pointers the latest news file, which has a section for the current development version of r, and below that for the patches in the current r-patched (part of the previous series). rss feeds based on changes to the news file. subversion log from the start of to today. this log is only updated daily. to get absolutely current information, use subversion, e.g. for all changes in the last week, svn log -v -r head:\{`date +%y-%m-%d -d' days ago'`\} https://svn.r-project.org/r for older logs in text format, try changing the filename, e.g. to "r_svnlog_ ". updating packages for changes in r r . . package check summaries: cran bioconductor building and checking r source packages for windows: http://win-builder.r-project.org/ subversion tips -- notably for r-core development guidelines todo lists windows version of r. tcltk package. list from brian ripley. list from kurt hornik. list from martin mächler. list from paul murrell. list from duncan temple lang. rfcs and writeups a redesign of how to customise a grob in 'grid'. guidelines for *.rd files. translating r messages: notes for translation teams. list of translation teams: contact these people if you want to help with translations, or contact r-core if you want to add your language to the list. resources for portable programming. using valgrind on the r memory manager memory profiling how s methods work by john chambers (on how s method dispatch has been working since r version . . ). a brief description of new support for raster images in the graphics engine. an rfc on a proposal to make srgb the internal colorspace for r graphics. a description of the parse_rd() parser for rd files. this document also includes a draft description of the new facility for executing r code within rd man pages. how to write model-fitting functions in r, and especially on how to enable all the safety features. a list of things to consider for a possible re-design of the r homepage. a brief writeup on how to tag r source files for use with emacs. some notes on reference counting. copy of the noweb sources here, needed for configuring r in maintainer mode. older material material in this section is at least months' old, and may or may not still be relevant. in particular, some of the proposals here have been implemented in slightly different ways, or since been extended. subversion/cvs logs for , , , , , , , , and updating packages for changes in r r . .x r . .x r . .x r . .x r . .x r . .x r . .x r . .x r . .x notes on encodings and r, including the current implementation planned for . . . open bugs in the repository (original correspondence truncated to lines per message). no longer updated. cvs tips rfcs and writeups rfc on sparse matrices in r an embeddable r library some early notes on name spaces, and the name space implementation for r . (also available in pdf format). some notes on references, external objects, or mutable state for r and a simple implementation of external references and finalization added to . . a simple regular expression package provides an example of using this facility. an outline of a preliminary weak reference mechanism is available. details are likely to change after more experience is gained with this in some internal uses. some notes on the generational collector and its write barrier some very preliminary notes on byte code compilation for r luke's notes on threading, gui, and asynchronous io issues. some of the reference links are messed up; the full references are in the bibtex file. duncan also has a set of notes on threading. a simple alternate experimental implementation of tcltk as a proof of concept for an approach that uses a separate gui server process with inter-process message passing (which may be easier to get to work across unix/windows/macos). a preliminary experimental implementation of exception handling for r. also a simple mechanism for managing dynamic state. some notes on possible enhancements to the package installation procedure. a description of a new way of resolving c and fortran routines in .c, .call and .fortran calls. notes and example package of the new, extensible mechanism for converting arguments in .c() calls. developers notes on the methods and classes package. related pages discuss compatibility issues for formal vs old-style classes and methods, and other questions of strategy for future work, including areas where we are beginning to extend the api in programming with data. notes on some recent changes to functions get, assign, and their relatives, to allow consistent use of environments and other cleanups. dataeditor restructuring ideas (outdated) thoughts on embedding tk in r (outdated) notes on the = operator in assignments. some notes on a new save/load format and a serialization mechanism for r. an rfc on changes to r base graphics and the graphics device drivers. notes and examples on callbacks that are invoked at the end of each successfully completed top-level expression. a document-in-progress describing the changes to the graphics system from . . to . . a proposal for changing the specification of fonts in r and another one extending the discussion to line end/joins and alpha transparency specifications in r. a list of r devices, including device maintainers and pointers to where the code resides. a package that provides a null graphics device; includes a vignette, "devnull", that documents how to create a new graphics device as an add on package. description of the nonstandard evaluation rules in r . . and some suggestions. (updated). also an r function and docn for making model frames from multiple formulas. the ideas list r and databases area material mainly for the core team the core team can access the source files for this site by svn checkout https://svn.r-project.org/r-dev-web/trunk r-dev-web any commit to this area will be reflected in the web pages at the next (daily) update. the current version can be seen at https://svn.r-project.org/r-dev-web/trunk/index.html. texts to use when entering r into software archives such as freshmeat: short long r page at gnu.org release statement for . . release management details: release procedures and checklist script for building r releases script for building prereleases (alpha/beta/rc) script for setting and committing the version file script for setting up the branches at start of release procedures sample crontab (for . . ) release letter skeleton setting up and signing gpg keys advertising material the r logo in several resolutions. howto modify www.r-project.org. choosing not to go into management (again) – learning (lib)tech skip to content learning (lib)tech stories from my life as a technologist menu about me about this blog contact me twitter github linkedin flickr rss choosing not to go into management (again) often, to move up and get a higher pay, you have to become a manager, but not everyone is suited to become a manager, and sometimes given the preference, it’s not what someone wants to do. thankfully at gitlab, in every engineering team including support, we have two tracks: technical (individual contributor), and management. progression basically the progression is: intermediate senior staff or manager the management track does continue, and some engineering teams have levels beyond staff for the technical track. however, the idea is once at the senior level, an individual needs to think about which track they want to move to (if they want to move up). of course, there is no pressure to keep moving up. much like in any position at any organization, some people don’t want the added responsibilities that comes with moving up, and so prefer to stay in their current position. when there’s a lack of an individual contributor track personally, one of my biggest frustrations was not having a choice. in many organizations and industries, if you want to move up, you have to become a manager. while pay it a part of it, so much of the time, if you want to have more impact, you have to become a manager. but why? in small organizations, you can only hire so many people and someone has to be a manager, so it’s not possible for them to separate the roles, but so often even in organizations that have enough people to separate the work, a manager is expected to do a lot more than just manage people, coordinate work, meet quarterly goals, and figure out how to meet those goals. for example, why would a manager also be the systems administrator for many or all technology in an organization? if there’s an incident, that manager’s time would be taken up by the incident. who is then looking after their direct reports if something happens to one of them at the same time? i’ve also seen big projects being managed by a people manager. first, people managers aren’t necessarily great project managers, and vice versa. secondly, both types of managers often have people clamouring for or requiring their attention, and then the manager is forced to decide between neglecting projects or people, or trying to do both and likely fail at satisfying either. i’ve worked at and seen organizations where the people management and non-people management are mostly separated, and i have typically seen those organizations do better overall. to recognize that both are important for their organization to thrive, the positions also paid similarly (with the “project manager” paid more if they include other responsibilities that are organization-wide). technology companies are probably the best at recognizing that great technical individuals can have as much impact and importance as managers. i just wish we saw more of this outside of engineering teams and tech companies. but back to the topic at hand… deciding not to be a manager, the first time there were a lot of different reasons for me not to apply to be a manager, when looking for a job, after leaving my last position. for one, i didn’t think i had enough management experience (only . years) to be seriously considered at another organization as a manager. but the biggest reason was the simple fact that while i think i can be a good people manager, my strength really lies in solving problems and getting things done. i also knew that going remote, i was likely to have to change industries. so, as much as i disliked the idea of starting from “square one,” realistically, i didn’t have what i needed on paper to get hired into a higher position immediately. despite the lower pay, i was happy to go back to being an individual contributor and being able to focus on that. it helps that i’ve had a manager who recognized the work i did and got be promoted to senior (twice). deciding not to be a manager, the second time since i’ve been working as a senior for over two years now, my manager brought up becoming staff. at first, i wasn’t sure what that would mean. we’ve only ever had staff support engineers, and only promoted through the current process. i’m grateful to one of our senior managers, who was willing to sit down and talk to me about becoming a staff support engineer. recently, i’ve also had a lot of conversations with team members that have made me consider whether i should become a manager again. i’ve had engineers and managers tell me they think i’d make a good manager, which i’m certainly happy to hear. moving up to the manager level would also mean having a bigger impact. there’s also a ton of satisfaction from knowing you’re making people feel good about their work. at the same time, becoming a manager means less time to focus on technical things. it also means less time for self-learning and training on technical topics, and i don’t feel like i’m done yet. i’ve grown so much technically, and i’m just not ready to stop. there’s still a lot i feel like i can learn and that i can do for the team as an individual contributor. in the past couple of weeks, i’m also finally getting a better sense of how a bunch of my work is at the staff level, and not work others are doing. in the end i will definitely be working on a promotion, but until i feel like i could do more for the team by becoming a manager instead of staying an individual contributor, i’d rather stay a senior than become a manager before i feel it’s time. have a bunny if you made it to the end, thank you for reading, and have a great bun day! twitter linkedin pocket more facebook tumblr telegram email print like this: like loading... author: cynthia technologist, librarian, metadata and technical services expert, educator, mentor, web developer, uxer, accessibility advocate, documentarian view all posts by cynthia author cynthiaposted on february , february , categories work culturetags career growth, management, reflection leave a comment cancel reply enter your comment here... please log in using one of these methods to post your comment: email (required) (address never made public) name (required) website you are commenting using your wordpress.com account. ( log out / change ) you are commenting using your google account. ( log out / change ) you are commenting using your twitter account. ( log out / change ) you are commenting using your facebook account. ( log out / change ) cancel connecting to %s notify me of new posts via email. this site uses akismet to reduce spam. learn how your comment data is processed. post navigation previous previous post: prioritization in support: tickets, slack, issues, and more cynthia technologist, librarian, metadata and technical services expert, educator, mentor, web developer, uxer, accessibility advocate, documentarian view full profile → follow us twitter linkedin github telegram search for: search categories events librarianship library academic public special tours methodology project work technology tools update web design work culture follow via email enter your email address to receive notifications of new posts by email. email address: follow about me about this blog contact me twitter github linkedin flickr rss learning (lib)tech send to email address your name your email address cancel post was not sent - check your email addresses! email check failed, please try again sorry, your blog cannot share posts by email. %d bloggers like this: scaling your dyno formation | heroku dev center skip navigationshow navheroku dev center get started documentation changelog search get started node.js ruby on rails ruby python java php go scala clojure documentation changelog moreadditional resources home elements products pricing careers help status events podcasts compliance center heroku blogheroku blog find out what's new with heroku on our blog. visit blog log inorsign up view categories categories heroku architecture dynos (app containers) stacks (operating system images) networking & dns platform policies platform principles command line deployment deploying with git deploying with docker deployment integrations continuous delivery continuous integration language support node.js ruby working with bundler rails support python working with django background jobs in python java working with maven java database operations working with the play framework java advanced topics working with spring boot php go go dependency management scala clojure databases & data management heroku postgres postgres basics postgres performance postgres data transfer & preservation postgres availability postgres special topics heroku redis apache kafka on heroku other data stores monitoring & metrics logging app performance add-ons all add-ons collaboration security app security identities & authentication compliance heroku enterprise private spaces infrastructure networking enterprise accounts enterprise teams heroku connect (salesforce sync) single sign-on (sso) patterns & best practices extending heroku platform api app webhooks heroku labs building add-ons add-on development tasks add-on apis add-on guidelines & requirements building cli plugins developing buildpacks dev center accounts & billing troubleshooting & support heroku architecture dynos (app containers) scaling your dyno formation scaling your dyno formation english — 日本語に切り替える last updated december table of contents manual scaling autoscaling scaling limits dyno formation listing dynos introspection understanding concurrency heroku apps can be scaled to run on multiple dynos simultaneously (except on free or hobby dynos). you can scale your app’s dyno formation up and down manually from the heroku dashboard or cli. you can also configure heroku autoscaling for performance-tier dynos, and for dynos running in private spaces. threshold autoscaling adds or removes web dynos from your app automatically based on current request latency. dynos are prorated to the second, so if you want to experiment with different scale configurations, you can do so and only be billed for actual seconds used. remember, it’s your responsibility to set the correct number of dynos and workers for your app. manual scaling before you can scale your app’s dyno formation, make sure your app is running on professional-tier dynos: from the heroku dashboard, select the app you want to scale from your apps list. navigate to the resources tab. above your list of dynos, click change dyno type. select the professional (standard/performance) dyno type. click save. scaling from the dashboard to scale the number of dynos for a particular process type: select the app you want to scale from your apps list. navigate to the resources tab. in the app’s list of dynos, click the edit button (looks like a pencil) next to the process type you want to scale. drag the slider to the number of dynos you want to scale to. click confirm. to change the dyno type used for a particular process type: click the hexagon icon next to the process type you want to modify. choose a new dyno type from the drop-down menu (standard- x, standard- x, performance-m, or performance-l). click confirm. scaling from the cli scaling the number of dynos you scale your dyno formation from the heroku cli with the ps:scale command: $ heroku ps:scale web= scaling dynos... done, now running web at :standard- x the command above scales an app’s web process type to dynos. you can scale multiple process types with a single command, like so: $ heroku ps:scale web= worker= scaling dynos... done, now running web at :standard- x, worker at :standard- x you can specify a dyno quantity as an absolute number (like the examples above), or as an amount to add or subtract from the current number of dynos, like so: $ heroku ps:scale web+ scaling dynos... done, now running web at :standard- x. if you want to stop running a particular process type entirely, simply scale it to : $ heroku ps:scale worker= scaling dynos... done, now running web at :standard- x. changing dyno types to move a process type from standard- x dynos up to standard- x dynos for increased memory and cpu share: $ heroku ps:scale web= :standard- x scaling dynos... done, now running web at :standard- x. note that when changing dyno types, you must still specify a dyno quantity (such as in the example above). moving back down to standard- x dynos works the same way: $ heroku ps:scale web= :standard- x scaling dynos... done, now running web at :standard- x see the documentation on dyno types for more information on dyno types and their characteristics. autoscaling autoscaling is currently available only for performance-tier dynos and dynos running in private spaces. heroku’s auto-scaling uses response time which relies on your application to have very small variance in response time. if your application does not, then you may want to consider a third-party add-on such as rails auto scale which scales based on queuing time instead of overall response time. if autoscaling doesn’t cover your needs or is not working as expected for your apps, we recommend trying rails auto scale add-on or adept scale add-on. autoscaling lets you scale your web dyno quantity up and down automatically based on one or more application performance characteristics. configuration a small number of customers have observed a race condition when using heroku web autoscaling in conjunction with a third-party worker autoscaling utility like hirefire. this race condition results in the unexpected disabling of heroku’s autoscaling. to prevent this scenario, do not use these two autoscaling tools in conjunction. autoscaling is configured from your app’s resources tab on the heroku dashboard: click the enable autoscaling button next to your web dyno details. the autoscaling configuration options appear: use the slider or text boxes to specify your app’s allowed autoscaling range. the cost range associated with the specified range is shown directly below the slider. your dyno count is never scaled to a quantity outside the range you specify. note that the minimum dyno limit cannot be less than . next, set your app’s desired p response time. the autoscaling engine uses this value to determine how to scale your dyno count (see below). a recommended p response time is provided. enable email notifications if you’d like all app collaborators (or team members if you’re using heroku teams) to be notified when your web dyno count reaches the range’s upper limit. at most one notification email is sent per day. when you’ve configured autoscaling to your liking, click confirm. note that if your new autoscaling dyno range setting does not include the process type’s current dyno count, the current dyno count is immediately adjusted to conform to the range. autoscaling logic the dyno manager uses your app’s desired p response time to determine when to scale your app. the autoscaling algorithm uses data from the past hour to calculate the minimum number of web dynos required to achieve the desired response time for % of incoming requests at your current request throughput. the autoscaling algorithm does not include websocket traffic in its calculations. every time an autoscaling event occurs, a single web dyno is either added or removed from your app. autoscaling events always occur at least minute apart. the autoscaling algorithm scales down less aggressively than it scales up. this protects against a situation where substantial downscaling from a temporary lull in requests results in high latency if demand subsequently spikes upward. if your app experiences no request throughput for minutes, its web dynos scale down at -minute intervals until throughput resumes. sometimes, slow requests are caused by downstream bottlenecks, not web resources. in these cases, scaling up the number of web dynos can have a minimal (or even negative) impact on latency. to address these scenarios, autoscaling will cease if the percentage of failed requests is % or more. you can monitor the failed request metric using threshold alerting. monitoring autoscaling events from the heroku dashboard autoscaling events appear alongside manual scale events in the events chart. in event details they are currently identified as having been initiated by “dyno autoscaling”. in addition, enabling, disabling and changes to autoscaling are shown. if a series of autoscaling events occur in a time interval rollup, only the step where the scaling changed direction is shown. for example, in the events chart below “scaled up to of performance-m” is an intermediate step to the peak of performance-m dynos, and is not shown. with webhooks you can subscribe to webhook notifications that are sent whenever your app’s dyno formation changes. webhook notifications related to dyno formation have the api:formation type. read app webhooks for more information on subscribing to webhook notifications. disabling autoscaling disable autoscaling by clicking the disable autoscaling button on your app’s resources tab. then, specify a fixed web dyno count and click confirm. manually scaling through the cli, or otherwise making a call to ps:scale via the api to instruct it to manually scale (e.g. via a third-party autoscaling tool) will disable autoscaling. known issues & limitations as with any autoscaling utility, there are certain application health scenarios for which autoscaling might not help. you might also need to tune your postgres connection pool, worker count, or add-on plan(s) to accommodate changes in web dyno formation. the mechanism to throttle autoscaling based on a request throughput error rate of % or more was designed for the scenario where the bottleneck occurs in downstream components. please see understanding concurrency for additional details. we strongly recommend that you simulate the production experience with load testing, and use threshold alerting in conjunction with autoscaling to monitor your app’s end-user experience. please refer to our load testing guidelines for heroku support notification requirements. scaling limits different dyno types have different limits to which they can be scaled. see dyno types to learn about the scaling limits. dyno formation the term dyno formation refers to the layout of your app’s dynos at a given time. the default formation for simple apps will be a single web dyno, whereas more demanding applications may consist of web, worker, clock, etc… process types. in the examples above, the formation was first changed to two web dynos, then two web dynos and a worker. the scale command affects only process types named in the command. for example, if the app already has a dyno formation of two web dynos, and you run heroku ps:scale worker= , you will now have a total of four dynos (two web, two worker). listing dynos the current dyno formation can been seen by using the heroku ps command: $ heroku ps === web (free): `bundle exec unicorn -p $port -c ./config/unicorn.rb` web. : up for h web. : up for m === worker (free): `bundle exec stalk worker.rb` worker. : up for m the unix watch utility can be very handy in combination with the ps command. run watch heroku ps in one terminal while you add or remove dynos, deploy, or restart your app. introspection any changes to the dyno formation are logged: $ heroku logs | grep scale - - t : : + : heroku[api]: scale to web= , worker= by adam@example.com note that the logged message includes the full dyno formation, not just dynos mentioned in the scale command. understanding concurrency singleton process types, such as a clock/scheduler process type or a process type to consume the twitter streaming api, should never be scaled beyond a single dyno. these process types don’t benefit from additional concurrency and in fact they will create duplicate records or events in your system as each tries to do the same work at the same time. scaling up a process type provides additional concurrency for performing the work handled by that process type. for example, adding more web dynos allows you to handle more concurrent http requests, and therefore higher volumes of traffic. adding more worker dynos lets you process more jobs in parallel, and therefore a higher total volume of jobs. importantly, however, there are scenarios where adding dynos to to a process type won’t immediately improve app performance: backing service bottlenecks sometimes, your app’s performance is limited by a bottleneck created by a backing service, most commonly the database. if your database is currently a performance bottleneck, adding more dynos might only make the problem worse. instead, try some or all of the following: optimize your database queries upgrade to a larger database implement caching to reduce database load switch to a sharded configuration, or scale reads using followers long-running jobs dyno concurrency doesn’t help with large, monolothic http requests, such as a report with a database query that takes seconds, or a job to email out your newsletter to , subscribers. concurrency gives you horizontal scale, which means it applies best to work that is easily subdivided. the solution to the slow report might be to move the report’s calculation into the background and cache the results in memcache for later display. for the long job, the answer is to subdivide the work: create a single job that in turn puts , jobs (one for each newsletter to send) onto the queue. a single worker can consume all these jobs in sequence, or you can scale up to multiple workers to consume these jobs more quickly. the more workers you add, the more quickly the entire batch will finish. the request timeout article has more information on the effects of concurrency on request queueing efficiency. keep reading dynos (app containers) feedback log in to submit feedback. information & support getting started documentation changelog compliance center training & education blog podcasts support channels status language reference node.js ruby java php python go scala clojure other resources careers elements products pricing subscribe to our monthly newsletter your email address: rss dev center articles dev center changelog heroku blog heroku news blog heroku engineering blog heroku podcasts twitter dev center articles dev center changelog heroku heroku status facebook instagram github linkedin youtube heroku is acompany © salesforce.com heroku.com terms of service privacy cookies rails autoscale | heroku dev center skip navigationshow navheroku dev center get started documentation changelog search get started node.js ruby on rails ruby python java php go scala clojure documentation changelog moreadditional resources home elements products pricing careers help status events podcasts compliance center heroku blogheroku blog find out what's new with heroku on our blog. visit blog log inorsign up view categories categories heroku architecture dynos (app containers) stacks (operating system images) networking & dns platform policies platform principles command line deployment deploying with git deploying with docker deployment integrations continuous delivery continuous integration language support node.js ruby working with bundler rails support python working with django background jobs in python java working with maven java database operations working with the play framework java advanced topics working with spring boot php go go dependency management scala clojure databases & data management heroku postgres postgres basics postgres performance postgres data transfer & preservation postgres availability postgres special topics heroku redis apache kafka on heroku other data stores monitoring & metrics logging app performance add-ons all add-ons collaboration security app security identities & authentication compliance heroku enterprise private spaces infrastructure networking enterprise accounts enterprise teams heroku connect (salesforce sync) single sign-on (sso) patterns & best practices extending heroku platform api app webhooks heroku labs building add-ons add-on development tasks add-on apis add-on guidelines & requirements building cli plugins developing buildpacks dev center accounts & billing troubleshooting & support add-ons all add-ons rails autoscale this add-on is operated by rails autoscale simple, reliable queue-based autoscaling for your rails app. rails autoscale last updated june table of contents full documentation how is this different from heroku’s own autoscaling? installation adjust your settings in the rails autoscale dashboard worker autoscaling troubleshooting migrating between plans removing the add-on support rails autoscale is an add-on that automatically scales web and worker dynos for rails applications. autoscaling ensures that your app can gracefully handle increased traffic while only paying for the dynos you need. full documentation this documentation page is a high-level summary. see the official documentation site for more. how is this different from heroku’s own autoscaling? heroku offers a native autoscaling solution that’s worth a try if you run performance dynos and you only need to autoscale web dynos. rails autoscale goes beyond this by autoscaling based on request queue times, autoscaling worker dynos, and working with all dyno types. for more info, check out the common questions doc. installation i’ll walk you through the full installation and setup process in the -minute video below. additional instructions are in the getting started guide. rails autoscale can be installed from railsautoscale.com, the heroku elements marketplace, or the via cli: check out rails autoscale on heroku elements for pricing and plans. $ heroku addons:create rails-autoscale -----> adding rails_autoscale_url to sharp-mountain- ... done, v (free) after rails autoscale is provisioned, a rails_autoscale_url config var is available in the app configuration. you don’t have to do anything with it. this setting is used by the rails_autoscale_agent gem when communicating with the rails autoscale service. rails autoscale supports rails . + and ruby . . +. if you run into any trouble with the initial setup, check out the official getting started docs or the troubleshooting guide. adjust your settings in the rails autoscale dashboard after you install the gem and deploy your application, you’ll begin to see activity in your rails autoscale dashboard. access the dashboard via the cli: $ heroku addons:open rails-autoscale opening rails_autoscale_url for sharp-mountain- you can also access the dashboard from your app’s heroku dashboard: the rails autoscale dashboard documentation is here. request queue time threshold rails autoscale triggers scale events by monitoring the request queue time for every request. this is the time between heroku accepting a request and your application beginning processing. queue time will increase when your application servers are too busy to handle all incoming requests. for a thorough explanation of queue time, read nate berkopec’s excellent article “scaling ruby apps to requests per minute - a beginner’s guide”. rails autoscale tracks the th percentile queue time, which for most applications will hover well below the default threshold of ms. a breach of this threshold will trigger an upscale for your application, adding one web dyno. after minutes of measurements below the downscale threshold ( ms by default), a downscale event is triggered. rails autoscale’s queue time metrics will not match apm tools like scout and new relic. those tools typically show an average, while rails autoscale tracks the th percentile. settings page rails autoscale provides default settings that work for most apps, but some apps may need a higher or lower threshold. these settings are described in detail in the official docs. worker autoscaling rails autoscale supports autoscaling worker dynos that are using sidekiq, resque, delayed job, or que. it’s as easy as autoscaling your web dynos, and the settings page will walk you through the setup. see the launch announcment for a video walkthrough. troubleshooting check out the official troubleshooting guide if you run into any problems. if you’re still stuck, please email help@railsautoscale.com. migrating between plans use the heroku addons:upgrade command to migrate to a new plan. $ heroku addons:upgrade rails-autoscale:newplan -----> upgrading rails-autoscale:newplan to sharp-mountain- ... done, v ($ /mo) your plan has been updated to: rails-autoscale:newplan removing the add-on rails autoscale can be removed via the cli. this will destroy all associated data and cannot be undone! $ heroku addons:destroy rails-autoscale -----> removing rails-autoscale from sharp-mountain- ... done, v (free) support all rails autoscale support and runtime issues should be submitted via heroku’s support channels or emailed to help@railsautoscale.com. keep reading all add-ons feedback log in to submit feedback. ziggeo raygun crash reporting information & support getting started documentation changelog compliance center training & education blog podcasts support channels status language reference node.js ruby java php python go scala clojure other resources careers elements products pricing subscribe to our monthly newsletter your email address: rss dev center articles dev center changelog heroku blog heroku news blog heroku engineering blog heroku podcasts twitter dev center articles dev center changelog heroku heroku status facebook instagram github linkedin youtube heroku is acompany © salesforce.com heroku.com terms of service privacy cookies reflection part : my first year at gitlab and becoming senior – learning (lib)tech skip to content learning (lib)tech stories from my life as a technologist menu about me about this blog contact me twitter github linkedin flickr rss reflection part : my first year at gitlab and becoming senior about a year ago, i wrote a reflection on summit and contribute, our all staff events, and later that year, wrote a series of posts on the gitlab values and culture from my own perspective. there is a lot that i mention in the blog post series and i’ll try not to repeat myself (too much), but i realize i never wrote a general reflection at year , so i’ve decided to write about both years now but split into parts. getting hired overall, it was really nice to have the general process and everything laid out on the website. the process i went through was very straightforward. apply. take home test. screening call. interview with manager. interview with (interim/stand in) director. interview with vp. references check. at gitlab, the time till hire is meant to be fairly short so as not to lose great candidates simply due to the length of the process. admittedly, one of the reasons i accepted at gitlab was because they offered before the ones i was still interviewing with. while i knew gitlab did not fit into my experience as well as some of the others, i was very intrigued by the values and the way they seemed to be applied. of course, i was particularly attracted to the transparency of seemingly everything. getting started onboarding anywhere can be overwhelming and tedious at the same time. gitlab in some ways isn’t any different. you still have a ton of account setups and things to check off, but there’s a nicely laid out template and provisioning of accounts is mostly done by other people. you only need to deal with it if for some reason it was missed. coffee chats as part of onboarding, you’re encouraged to have coffee chats with people in other departments and outside your geographical area. one of the great things of being in a fully remote company is that you’re not location bound (though you are timezone bound). but it’s not only during onboarding! we’re encouraged to continue having coffee chats with random people within gitlab. we have a donut bot that randomly pairs people (who are in that channel) every weeks. it completely ignores timezones but sometimes people will do the calls outside of “standard” work hours, so it works out. it’s a great way to meet people you might not normally and hear about what’s happening in other parts of the company. sometimes you end up communicating or working with people you might not expect to and it’s great when you see a familiar face. learning gitlab ( months) i was hired at gitlab as a support agent, like everyone else newly hired to the team focused on the saas product. we had a couple of support engineers on the team, but the hiring process for that position was geared towards people who would work on the self-managed product, so we never actually hired any new ones. my first manager, lyle kozloff, was hired specifically to build out a team to work on gitlab.com (saas), and with the first hire in march, when i joined in june, the team was still quite small and new. a lot of processes weren’t written or weren’t quite fleshed out yet. even then, gitlab is a large product and at the time, there were close to company members, so there was a lot to learn both on the technological side and the company/culture side. the culture shock and adjustment learning technology is usually the “easy” part. learning to work within a company is usually the hard part, and often what sets them apart. while i’ve never worked in a “tech startup”, i’ve worked in small, flexible teams, and the co-op somewhat worked like a startup. i also know quite a few people who have or do work in startups, so i had heard a lot of stories. i’m not sure anything though could have truly prepared me for gitlab’s culture. little did i know that not only do gitlab team members follow the values, but that many of them will call others out on not following them, publicly. additionally, it was done in a way that followed our values too. our ceo, sid, very much leads by example in this area, but victor wu, a former product manager, is the one i probably remember most especially in my early months of doing this in places that was visible to me. i learnt a lot about our values from his messages, so i am still grateful that i had a chance to work with him. everyone that i met and began working with were welcoming and ready to share their knowledge. i constantly felt like i had crammed twice as much knowledge than i normally did in the same amount of time, but it was a great experience to be learning about technology that i had only heard of. nevertheless, i wasn’t sure how i could best give back and start making a positive impact. when i spoke with my manager about it, he completely changed my work life. he told me that i didn’t need to ask for permission on everything and that we trust you to contribute. having worked at many public institutions where procedures can be quite entrenched and literally yelled at for asking the “wrong” questions a couple of times (let alone doing something wrong), i didn’t realize how much my fear of stepping on people’s toes was hobbling my growth and impact at gitlab. at gitlab though, part of our values is having “short” toes, and assuming positive intent. added to that, the level of trust that we have across team members and teams is amazing. it was quite an adjustment for me, but i embraced it. contribution graph from year showing more contributions starting september becoming senior ( year) my fear of making mistakes has never disappeared, but with my manager’s encouragement, i started to take on work that i thought would benefit the team without asking ahead of time. to help coordinate or advertise that i’m working on something, i typically create an issue in the appropriate project tracker and assign myself. i also started digging into some of the common support topics we got. in particular, i looked at, started troubleshooting, filing issues, and once even getting a patch exception on project imports. then i took it further by proposing we should either have language to clearly explain why we won’t import projects for customers (then point them to our migration services team) or come up with a process where we could import one or two projects that might be blocking them from fully migrating. in the end, it took more than months to work out the whole process and then i ended up doing a second, more complete security review about months ago, but i’m happy to report that we’ve done over imports since then. (most were actually early on because our development teams have made a lot of improvements to imports in recent months.) as i got to know more people and discussing issues with others more in-depth, i eventually started collaborating with other teams including with the vpat (web accessibility evaluation). this internal “consultation” turned into my taking on a counterpart role with the ux team. later on, when we started formalizing the counterpart role, no surprised to anyone who knows my penchant for documentation and my self-labelling of being a documentarian, i also became one of the docs counterparts (and until more recently, the main one). i would say i know the docs team almost as well as the support team and am considered an honourary docs team member. i also get pinged on issues and in slack in some areas where we lack an existing counterpart or where i’ve become an expert. while it was only a small improvement, i even decided to brush off my web skills and help make updates to our support portal: portal original portal revision in february , less than months after i started at gitlab, i became the first senior support agent. my manager said it a recognition of the work that i was doing. it was awesome. when he first suggested finalizing the promotion document, i was floored. never had i imagined i could go anywhere and get promoted so quickly. thanks @gitlab for building a #work #culture where leadership and the credit #values are recognized & allow team members to move "up" without necessarily becoming managers. i'm proud and amazed to become the first senior support agent at #gitlab https://t.co/egii iw jg — cynthia "arty" ng (@therealarty) february , unsurprisingly, my job didn’t really look different after my promotion. i continued to work the way i thought i should, which meant not only working on tickets, but helping others, updating documentation, continually learning (meaning always working on a training “course”), and taking on whatever projects i could. the engineering team i want to wrap up this first part by talking about being part of the engineering team. which department the support team is part of differs quite broadly depending on the organization, often being in product or customers success. i don’t know the full history, but at gitlab, the first support team members were support engineers, and would not only be expected to troubleshoot any issue with gitlab, but also determine if and which piece connected or related to gitlab might be the issue. the product is what we support, but gitlab and the runner can be installed on almost any combination of hardware and os, any part of which might affect gitlab itself. at rare times, support might walk a customer through how to patch their instance. in most cases, we file an issue, or contribute a fix that will make its way up to all users. i think being part of engineering has been a big plus. we get any department-wide updates including major updates on how infrastructure and development are improving our largest instance of gitlab (gitlab.com), changes to development guidelines, productivity, and hiring. i believe there’s also a certain amount of lower mental barrier when reaching out, since it’s all “engineering” and not reaching across departments. i think there are less barriers at gitlab, but i know that before i adjusted to the culture, reaching out to another department involved internal mental resistance nonetheless, and i know i’m not the only one. our ux (user experience) team is part of engineering as well, and technical writing (docs) team got moved to engineering last year. it consolidated all the “teams” within the company that were regularly making changes to the repository (which includes documentation). especially with the way gitlab works, i don’t think it would have been bad for support to be elsewhere (there were rumours floating around at some point), but being in engineering implies that you contribute to the gitlab projects, and that’s a good expectation to have. part coming soon! here’s a group of capybaras to say show the sense of camaraderie and collaboration, also as a reward for reading through part . part coming soon! hello pic.twitter.com/bwygjjrgfc — capybaras (@capybaracountry) june , twitter linkedin pocket more facebook tumblr telegram email print like this: like loading... author: cynthia technologist, librarian, metadata and technical services expert, educator, mentor, web developer, uxer, accessibility advocate, documentarian view all posts by cynthia author cynthiaposted on june , january , categories update, work culturetags gitlab, organizational culture, reflection leave a comment cancel reply enter your comment here... please log in using one of these methods to post your comment: email (required) (address never made public) name (required) website you are commenting using your wordpress.com account. ( log out / change ) you are commenting using your google account. ( log out / change ) you are commenting using your twitter account. ( log out / change ) you are commenting using your facebook account. ( log out / change ) cancel connecting to %s notify me of new posts via email. this site uses akismet to reduce spam. learn how your comment data is processed. post navigation previous previous post: is blog reading dead? next next post: reflection part : my second year at gitlab and on becoming senior again cynthia technologist, librarian, metadata and technical services expert, educator, mentor, web developer, uxer, accessibility advocate, documentarian view full profile → follow us twitter linkedin github telegram search for: search categories events librarianship library academic public special tours methodology project work technology tools update web design work culture follow via email enter your email address to receive notifications of new posts by email. email address: follow about me about this blog contact me twitter github linkedin flickr rss learning (lib)tech add your thoughts here... (optional) post to cancel send to email address your name your email address cancel post was not sent - check your email addresses! email check failed, please try again sorry, your blog cannot share posts by email. %d bloggers like this: tether is “too big to fail” — the entire cryptocurrency industry utterly depends on it – attack of the foot blockchain skip to content attack of the foot blockchain blockchain and cryptocurrency news and analysis by david gerard about the author attack of the foot blockchain: the book book extras business bafflegab, but on the blockchain buterin’s quantum quest dogecoin ethereum smart contracts in practice icos: magic beans and bubble machines imogen heap: “tiny human”. total sales: $ . index libra shrugged: how facebook tried to take over the money my cryptocurrency and blockchain commentary and writing for others press coverage: attack of the foot blockchain press coverage: libra shrugged table of contents the conspiracy theory economics of bitcoin the dao: the steadfast iron will of unstoppable code search for: main menu tether is “too big to fail” — the entire cryptocurrency industry utterly depends on it th december th december - by david gerard - comments. tether is a us dollar substitute token, issued by tether inc., an associate of cryptocurrency exchange bitfinex. tether is a favourite of the crypto trading markets — it moves at the speed of crypto, without all that tedious regulation and monitoring that actual dollars attract. also, exchanges that are too dodgy to get dollar banking can use tethers instead. bryce weiner has written a good overview of how tether works in relation to the cryptocurrency industry. his piece says nothing that any regular reader of this blog won’t already know, but he says it quite well. [medium] weiner’s thesis: the whole crypto industry depends on tether staying up. tether is too big to fail. the purpose of the crypto industry, and all its little service sub-industries, is to generate a narrative that will maintain and enhance the flow of actual dollars from suckers, and keep the party going. increasing quantities of tethers are required to make this happen. we just topped twenty billion alleged dollars’ worth of tethers, sixteen billion of those just since march . if you think this is sustainable, you’re a fool. bitcoin-tether volumes are now magnitudes greater than bitcoin-dollar volumes. #btc #usdt pic.twitter.com/zizueitkwr — kaiko (@kaikodata) december , pump it! does crypto really need tether? ask the trading market — stablecoins are overwhelmingly tethers by volume. i suspect the other stablecoins are just a bit too regulated for the gamblers. tether is functionally unregulated. in fact, the whole crypto market is overwhelmingly tethers by volume — tether has more trading volume than the next three coins, bitcoin, ethereum and xrp, combined. [decrypt] in march, when the price of bitcoin crashed, a pile of exchanges and whales (large holders) put money, or maybe bitcoins, into tether to keep the system afloat. at least some percentage of the backing for tethers might exist! (all are now complicit in a manner that would be discoverable in court.) this is why crypto is so up in arms about the proposed stable act, which would require stablecoin issuers to become banks — the act would take out tether immediately, given tether’s extensive and judicially-recognised connections to new york state, and there is no way on earth that a company that comports itself in the manner of tether is getting a banking charter. (the tether printer was quiet for about a week after the stable act came out — and the price of bitcoin slid down about $ , .) the bitcoin price is visibly pumped by releases of tethers — particularly on weekends. i find it strangely difficult to believe real-money institutional investors are so keen to get on the phone to tether on a saturday. a snapshot of coinbase btc/usd this weekend. spot when tethers were deployed on binance or huobi. clap if you believe in tether but it’s far wider than the traders. every person making money from crypto — not just bitcoin, but everything else touching crypto — is painfully aware they need tether to keep the market pumped. all the little service sub-industries are vested in making crypto look real — and not just handwaving nonsense held up by a hilariously obvious fraud. so tether must be propped up, at all cost — in the face of behaviour that, at any real financial institution, would have had the air-raid sirens going off long ago. the big question is: what are all those tethers backed by? tether used to confidently claim that every tether was backed by a dollar held in a bank account. this turned out not to be the case — so now tethers are backed by dollars, or perhaps bitcoins, or loans, or maybe hot air. various unfortunately gullible journalists have embarrassed themselves by taking what tether tells them at face value. matthew leising from bloomberg confidently declared in december , based on documents supplied to him by tether, that tether seemed fully backed and solvent! [bloomberg] then four months later, the new york attorney general revealed that tether had admitted to them that tethers were no more than % backed, and the backing had failed in october — tether had snowed leising. i don’t recall leising ever speaking of this again, even to walk back his claim. larry cermak from the block fell for the same trick recently, from tether and from people working closely with tether. [twitter] bitfinex/tether was supposed to give the nyag a pile of documents by now. the nyag is talking to the companies about document production, and just what documents they do and don’t have — so proceedings have been delayed until january . [letter, pdf] the end game dan davies’ book lying for money (uk, us) talks about the life cycle of frauds. a fraud may start small — but it has to keep growing, to cover up the earlier fraud. so a fraud will grow until it can’t. i did a chat with the ft alphaville unofficial telegram a few weeks ago. someone asked a great question: “who’s going to make out like a bandit when/if bitcoin collapses?” most scam collapses involve someone taking all the money. in the case of bitcoin and tether, i think the answer is … nobody. a whole lot of imaginary value just vanishes, like morning dew. i can’t think of a way for tether or the whales to exit scam with a large pile of actual dollars — because a shortage of actual dollars is crypto’s whole problem. i mean, i’m sure someone will do well. but there’s no locked-up pile of money to plunder. crypto “market cap” is a marketing fiction — there’s very little realisable money there. what was the “market cap” of beanie babies in ? imaginary, that’s what. so how does this end? the main ways out i see are nyag or the cftc finally getting around to doing something. either of those are a long way off — because regulators move at the speed of regulators. even the nyag proceeding is just an investigation at this stage, not a case as such. everyone in crypto’s service industries has a job that’s backed by the whales. perhaps the whales will keep funding stuff? i’m not counting on it, given all the redundancies and shut-down projects over and . this will keep going until it can’t. remember that it took seventeen years to take down bernie madoff. he got institutional buyers in, too. past asset bubble veteran inch the inchworm, his ty slightly askew, hitting the sauce after seeing his crypto portfolio your subscriptions keep this site going. sign up today! share this: click to share on twitter (opens in new window) click to share on facebook (opens in new window) click to share on linkedin (opens in new window) click to share on reddit (opens in new window) click to share on telegram (opens in new window) click to share on hacker news (opens in new window) click to email this to a friend (opens in new window) taggedbryce weinerlarry cermakmatthew leisingnew yorktether post navigation previous article facebook’s libra is now diem; stable act says stablecoins must get banking licenses next article news: coinbase key holders leave, ethereum . slashing, libra may not become diem, regulatory clarity in france comments on “tether is “too big to fail” — the entire cryptocurrency industry utterly depends on it” brendan says: th december at : pm tether’s design is so flawed it has to fail and when it does it will take btc and the rest of crypto with it. i predicted this over years ago and got out of crypto and into bsv – the real bitcoin which is not dependent on scammy fraudsters running a counterfeit usd operation to pump the price. reply david gerard says: th december at : am they had us in the second half meme reply eloi says: th december at : am bsv is crypto… reply mark chamberlain says: th december at : am it would be interesting to know if there are any institutional or hedge fund players who understand this enough to be short crypto. plus how would it work? if you borrow the currency to sell it short and the price goes to zero and stops trading, how do you close your position? reply dylan says: th december at : pm this depends on a reliable way of shorting crypto, and i wouldn’t trust any of the existing exchanges enough to give money to them (unless i was ok losing that money). and i trust the “trustless” smart contract tools for doing this sort of thing even less. plus, you’d have to be confident about when the crypto market explodes, which is difficult to call given how manipulated it is. the market can stay irrational longer than you can stay solvent. reply david gerard says: th december at : pm yeah, in crypto the platforms themselves are part of the threat model. i expect you could deal with, say, cme in confidence. reply mark chamberlain says: th december at : pm the news today is about how the shorts were sold out in this new move. the better idea might be to short the new index fund bitw (if i was crazy enough to try). it’s now trading at an insane premium to nav, when you can instead just buy bitcoin itself through paypal. the kind of thing that happens at blow off tops…. reply greg allen says: th december at : pm not really – see articles on alphaville etc post-lehman, on how the move to mandate exchange settlement of derivatives just moved the counterparty risk from other banks to a single centralized exchange. i wouldn’t assume the exchange is sufficiently capitalized to help, and i wouldn’t even expect it to be on the hook for the other side of your futures contract reply massimo says: th january at : am interesting thesis, could you deepen the subject please? mark bloomfield says: th january at : pm please keep on at this: the message needs to get out. reply sesang says: th january at : am nice read. ty reply david says: th january at : pm excellent article. there’s no worst blind that those who don’t want to see. reply leave a reply cancel reply your email address will not be published. required fields are marked * comment name * email * website notify me of follow-up comments by email. notify me of new posts by email. this site uses akismet to reduce spam. learn how your comment data is processed. search for: click here to get signed copies of the books! get blog posts by email! email address subscribe support this site on patreon! hack through the blockchain bafflegab: $ /month for early access to works in progress! $ /month for early access and even greater support! buy the books! libra shrugged us paperback uk/europe paperback isbn- : kindle: uk, us, australia, canada (and all other kindle stores) — no drm google play books (pdf) apple books kobo smashwords other e-book stores attack of the foot blockchain us paperback uk/europe paperback isbn- : kindle: uk, us, australia, canada (and all other kindle stores) — no drm google play books (pdf) apple books kobo smashwords other e-book stores available worldwide rss - posts rss - comments recent blog posts tether printer go brrrrr — cryptocurrency’s substitute dollar problem news: craig wright copyright claims, grayscale, mt. gox, reddit on ethereum, the french bitcoin neo-nazi blockchain debate podcast: diem is a glorified paypal (david gerard vs. bryce weiner) stablecoins through history — michigan bank commissioners report, news: twitter on the blockchain, the us coup on the blockchain, gensler in at the sec, ripple ex-cto loses bitcoin keys excerpts from the book table of contents the conspiracy theory economics of bitcoin dogecoin buterin’s quantum quest icos: magic beans and bubble machines ethereum smart contracts in practice the dao: the steadfast iron will of unstoppable code business bafflegab, but on the blockchain imogen heap: “tiny human”. total sales: $ . index about press coverage for attack of the foot blockchain press coverage for libra shrugged my cryptocurrency and blockchain press commentary and writing facebook author page about the author contact the content of this site is journalism and personal opinion. nothing contained on this site is, or should be construed as providing or offering, investment, legal, accounting, tax or other advice. do not act on any opinion expressed here without consulting a qualified professional. i do not hold a position in any crypto asset or cryptocurrency or blockchain company. amazon product links on this site are affiliate links — as an amazon associate i earn from qualifying purchases. (this doesn’t cost you any extra.) copyright © – david gerard powered by wordpress and hitmag. send to email address your name your email address cancel post was not sent - check your email addresses! email check failed, please try again sorry, your blog cannot share posts by email. scriptio continua scriptio continua thoughts on software development, digital humanities, the ancient world, and whatever else crosses my radar. all original content herein is licensed under a creative commons attribution license. reminder experiencing technical difficulties thank you dh data talk outside the tent missing dh first contact you did _what_? form-based xml editing how to apologize a spot of mansplaining tei in other formats; part the second: theory tei in other formats; part the first: html humanities data curation interfaces and models tei is a text modelling language i will never not ever type an angle bracket (or iwnnetaab for short) dh tea leaves that bug bit me #alt-ac careers: digital humanities developer addenda et corrigenda making a new numbers server for papyri.info #apa converting apis object artefact script stomping on innovation killers disruptive library technology jester skip links skip to primary navigation skip to content skip to footer disruptive library technology jester about resume toggle search toggle menu peter murray library technologist, open source advocate, striving to think globally while acting locally follow columbus, ohio email twitter keybase github linkedin stackoverflow orcid email recent posts user behavior access controls at a library proxy server are okay minute read earlier this month, my twitter timeline lit up with mentions of a half-day webinar called cybersecurity landscape - protecting the scholarly infrastructure. ... as a cog in the election system: reflections on my role as a precinct election official minute read i may nod off several times in composing this post the day after election day. hopefully, in reading it, you won’t. it is a story about one corner of democ... running an all-online conference with zoom [post removed] less than minute read this is an article draft that was accidentally published. i hope to work on a final version soon. if you really want to see it, i saved a copy on the interne... with gratitude for the niso ann marie cunningham service award minute read during the inaugural niso plus meeting at the end of february, i was surprised and proud to receive the ann marie cunningham service award. todd carpenter, ... tethering a ubiquity network to a mobile hotspot minute read i saw it happen. the cable-chewing device the contractor in the neighbor’s back yard with the ditch witch trencher ... previous … next enter your search term... twitter github feed © peter murray. powered by jekyll & minimal mistakes. islandora open meeting: february , - google docs javascript isn't enabled in your browser, so this file can't be opened. enable and reload. some word features can't be displayed in google docs and will be dropped if you make changes view details islandora open meeting: february , share sign in the version of the browser you are using is no longer supported. please upgrade to a supported browser.dismiss file edit view insert format tools add-ons help accessibility debug unsaved changes to drive see new changes image options image options replace image accessibility tether printer go brrrrr — cryptocurrency’s substitute dollar problem – attack of the foot blockchain skip to content attack of the foot blockchain blockchain and cryptocurrency news and analysis by david gerard about the author attack of the foot blockchain: the book book extras business bafflegab, but on the blockchain buterin’s quantum quest dogecoin ethereum smart contracts in practice icos: magic beans and bubble machines imogen heap: “tiny human”. total sales: $ . index libra shrugged: how facebook tried to take over the money my cryptocurrency and blockchain commentary and writing for others press coverage: attack of the foot blockchain press coverage: libra shrugged table of contents the conspiracy theory economics of bitcoin the dao: the steadfast iron will of unstoppable code search for: main menu tether printer go brrrrr — cryptocurrency’s substitute dollar problem rd february th february - by david gerard - comments. tether (usdt) is the cryptocurrency trading economy’s favourite substitute us dollar. most bitcoin trading is against usdt — not against actual money. there’s about billion tethers in circulation. do you need to move dollars internationally, without all that tedious “compliance” and “being allowed to have a bank account” that actual dollars would involve? tether is there for you! tether is owned by ifinex — a cluster of companies formed around bitcoin exchange bitfinex. the remarkable rate of tether printing, combined with fabulously inept public relations from ifinex principals, strongly suggests the tether business is at the “out of control” stage of spiraling outwards. i don’t know precisely what tiger ifinex is riding, but it clearly has teeth. animation by @flashtrader the problem with tethers in the early days, tether claimed that every tether was backed - by us dollars held in a bank account. these claims turned out to be false — and ifinex, tether’s parent company, admitted this to the new york attorney general’s office in . the tether reserve is apparently made of loans to others, loans to other ifinex companies — yeah, they literally counted putting dollars in their other pocket as a “loan” that was an “asset” — bitcoins, other cryptocurrencies they found down the back of the sofa, and maybe even some us dollars — though there’s no evidence for that last one. this is less than ideal. so the nyag wanted to know just what tether was backing these billions of tethers with. also, ifinex cfo giancarlo devasini seemed to think that if tether couldn’t be backed, then the price of bitcoin “could tank to below k.” (every time someone questions the link between tethers and the price of bitcoin, point out that the cfo of bitfinex sure seems to think they’re solidly linked.) [affirmation of brian m. whitehurst, nydfs, pdf, ] after two years’ messing about, ifinex have been told in no uncertain terms to cough up all the information the nyag asked for. this was going to be due by january , but has now been extended another days. [letter, pdf] man the printer! tethers were printed at a furious rate all through — as usdt in circulation went from billion to billion between march and december. the printing picked up in december, when the bitcoin price was pumped from $ , to an all-time peak of just over $ , . the printer stopped for a bit in early january, then started up again just after january. this looks very like printing fake dollars as fast as possible before an axe falls. (i think the axe is the proposed fincen rule on documentation required to move crypto into and out of money services businesses — because in crypto, even if you’re a relatively clean actor, you know damn well that dirty money is no more than one or two steps away from you.) public relations there is precisely one question about tether: “what is your reserve?” and ifinex absolutely won’t — or can’t — answer it. so instead, they’ve done podcasts! the tether pr initiatlve is being led by the cto, paolo ardoino — who is also the guy that presses the button on the tether printer — and ifinex general counsel stuart hoegner. neither of the people saying things about the reserve are the ceo, jean-louis van der velde, nor the cfo, giancarlo devasini — the people with direct personal responsibility for the reserve. ardoino and hoegner did a softball interview with peter mccormack of what bitcoin did, explaining tether to him. (tether helped pay mccormack’s legal bills when craig wright sued him.) the podcast didn’t quite alleviate anyone’s fears. [what bitcoin did; amy castor] the bank that tether will admit to using, deltec, in the bahamas, did their very first bank video podcast — in which they declared that deltec held “large” reserves of bitcoins. [youtube, : on; deltec] deltec’s deputy ceo, gregory pepin, did an interview on laura shin’s unchained podcast pepin assured everyone that tether critics don’t understand markets and that tether is fully backed, but spent half an hour not giving any details of what it was backed with. shin asked him direct questions, and he wouldn’t answer. [unchained] pepin disappeared from the deltec “about” page shortly after. this was apparently a “website error”, and he returned in short order. [twitter; twitter; deltec, archive] ardoino has claimed that “tether is under strict scrutiny of various regulators, especially american ones.” [youtube, : on — in italian] this is just nonsense — tether does not seem to be supervised by any regulator. pepin talked up tether’s registration with fincen — but fincen registers money services businesses, and doesn’t regulate their behaviour. a fincen registration is explicitly not any sort of endorsement that this business is on the level. (hoegner insists that fincen registration is a type of regulation.) [coindesk; twitter] handwaving and paper shuffling there’s clearly shenanigans going on with tether and its reserve. the company has never been audited — and hoegner has claimed it would not be possible to audit tether. [coindesk] auditors have run screaming from the company part way through an audit. ifinex companies can’t hold onto bank accounts. tether won’t admit what’s in the reserve — though they no longer claim the reserve is entirely us dollars. at any normal company, the air-raid sirens would have been going off long ago. the game appears to be: print tethers, release these to the large tether-based exchanges, use the tethers to buy bitcoins or to fuel margin bets on bitcoins, move the bitcoins to coinbase, sell them for dollars. (that’s extremely simplified. there are many other curlicues and fine details involved in the scam.) there are rumours i can’t confirm that when an exchange “buys” tethers, they get them at a discount — and the rest is accounted as a “loan.” this would keep the tethers flowing, and keep exchanges interested in accepting them. twitter user desogames posted an interesting hypothesis: that tether’s “dollar” backing is holdings of usdc stablecoins. tether can’t get reliable banking, but they can hold usdc in a wallet. this would account for total usdc rising precipitously over the past year. it would also suggest that the tether system is nearly out of actual dollars. [desogames] jeremy allaire of usdc issuers centre has also been an unlikely defender of tether. [coindesk] usdc loudly touts claims that it’s well-regulated, and implies that it’s audited. but usdc is not audited — accountants grant thornton sign a monthly attestation that centre have told them particular things, and that the paperwork shows the right numbers. an audit would show for sure whether usdc’s reserve was real money, deposited by known actors — and not just a barrel of nails with a thin layer of gold and silver on top supplied by dubious entities. but, y’know, it’s probably fine and you shouldn’t worry. the question is, though: why does tether maintain all these claims about keeping books properly and so on, when it won’t, or can’t, prove anything to anyone in a verifiable manner? a useful question to ask would be: who are tether trying to convince with all this? other tether fun on the peer-to-peer trading board on tether-based crypto exchange binance, you can sell your tethers for three us dollars! you send them a tether, and you get the money via famous money-transmitter brands such as skrill, perfect money, neteller, and of course home credit kazakhstan. the best hypothesis is that the usd is stolen, and this is the washing machine; the premium is for the risk of the payment being reversed. [reddit] bennett tomlin writes up raphael nicolle, the founder of bitfinex and the cluster of companies that became ifinex. i don’t know about you, but i’d think that a crypto exchange founded by an aspiring ponzi schemer will surely be good and trustworthy. nicolle disappeared from public view shortly after bitfinex was hacked in august . [bennett tomlin] tether is getting more coverage in the mainstream press — most recently, the new republic and the wall street journal. new republic headline: “is tether just a scam to enrich bitcoin investors?” [new republic; wsj] as always, i recommend patrick mckenzie’s piece as the definitive text on tether: “tether is the internal accounting system for the largest fraud since madoff.” [kalzumeus] for every rohan buck i issue, i write a promissory note for that amount and put the note in my safe. which is why i'm definitely being transparent and not misleading when i say every rohan buck is backed % by reserves. wait what are you doing with those handcuffs? — rohan grey (@rohangrey) february , your subscriptions keep this site going. sign up today! share this: click to share on twitter (opens in new window) click to share on facebook (opens in new window) click to share on linkedin (opens in new window) click to share on reddit (opens in new window) click to share on telegram (opens in new window) click to share on hacker news (opens in new window) click to email this to a friend (opens in new window) taggedcentredeltecgiancarlo devasinigregory pepinifinexjean-louis van der veldejeremy allairepaolo ardoinoraphael nicollestuart hoegnertetherusdc post navigation previous article news: craig wright copyright claims, grayscale, mt. gox, reddit on ethereum, the french bitcoin neo-nazi comments on “tether printer go brrrrr — cryptocurrency’s substitute dollar problem” james says: rd february at : pm where is the sec and fbi to shut this tether scam down ?? reply david gerard says: rd february at : pm i hear rumours of regulators and government agencies with no sense of humor coming around to have a word with the ifinex principals – but regulators move at the speed of regulators and don’t like to do anything that isn’t a slam dunk, and frankly i was expecting them to be shut down by december . so i’m absolutely not putting a date on anything. but it’s a known public fact that other regulators and agencies are looking closely at tether, and are likely to use anything deeply interesting collected by the nyag. so as always, patience, while people are being ripped off. reply jacek pilat says: th february at : pm what are you gonna do with people who have so much of tether in their wallets? reply mark says: th february at : am why does no one talk about the ginger haired french ‘ex producer and writer’ of kids cartoons, jean chalopin? the chairman and largest shareholder of deltec bank. how does he become a banker? when? find the connection between chalopin and ifinex and you may find a smoking gun….. reply leave a reply cancel reply your email address will not be published. required fields are marked * comment name * email * website notify me of follow-up comments by email. notify me of new posts by email. this site uses akismet to reduce spam. learn how your comment data is processed. search for: click here to get signed copies of the books! get blog posts by email! email address subscribe support this site on patreon! hack through the blockchain bafflegab: $ /month for early access to works in progress! $ /month for early access and even greater support! buy the books! libra shrugged us paperback uk/europe paperback isbn- : kindle: uk, us, australia, canada (and all other kindle stores) — no drm google play books (pdf) apple books kobo smashwords other e-book stores attack of the foot blockchain us paperback uk/europe paperback isbn- : kindle: uk, us, australia, canada (and all other kindle stores) — no drm google play books (pdf) apple books kobo smashwords other e-book stores available worldwide rss - posts rss - comments recent blog posts tether printer go brrrrr — cryptocurrency’s substitute dollar problem news: craig wright copyright claims, grayscale, mt. gox, reddit on ethereum, the french bitcoin neo-nazi blockchain debate podcast: diem is a glorified paypal (david gerard vs. bryce weiner) stablecoins through history — michigan bank commissioners report, news: twitter on the blockchain, the us coup on the blockchain, gensler in at the sec, ripple ex-cto loses bitcoin keys excerpts from the book table of contents the conspiracy theory economics of bitcoin dogecoin buterin’s quantum quest icos: magic beans and bubble machines ethereum smart contracts in practice the dao: the steadfast iron will of unstoppable code business bafflegab, but on the blockchain imogen heap: “tiny human”. total sales: $ . index about press coverage for attack of the foot blockchain press coverage for libra shrugged my cryptocurrency and blockchain press commentary and writing facebook author page about the author contact the content of this site is journalism and personal opinion. nothing contained on this site is, or should be construed as providing or offering, investment, legal, accounting, tax or other advice. do not act on any opinion expressed here without consulting a qualified professional. i do not hold a position in any crypto asset or cryptocurrency or blockchain company. amazon product links on this site are affiliate links — as an amazon associate i earn from qualifying purchases. (this doesn’t cost you any extra.) copyright © – david gerard powered by wordpress and hitmag. send to email address your name your email address cancel post was not sent - check your email addresses! email check failed, please try again sorry, your blog cannot share posts by email. ci/cd - wikipedia ci/cd from wikipedia, the free encyclopedia jump to navigation jump to search in software engineering, ci/cd or cicd generally refers to the combined practices of continuous integration and either continuous delivery or continuous deployment.[ ][ ][ ] ci/cd bridges the gaps between development and operation activities and teams by enforcing automation in building, testing and deployment of applications. modern day devops practices involve continuous development, continuous testing, continuous integration, continuous deployment and continuous monitoring of software applications throughout its development life cycle. the ci/cd practice or ci/cd pipeline forms the backbone of modern day devops operations. in other contexts[edit] in the context of corporate communication, ci/cd can also refer to the overall process of corporate identity and corporate design.[ ] references[edit] ^ irani, zubin ( - - ). " common pitfalls of cicd -- and how to avoid them". infoworld. retrieved - - . ^ heller, martin ( - - ). "continuous integration is not always the right answer. here's why". techbeacon. retrieved - - . ^ atlassian ( - - ). "continuous integration vs. continuous delivery vs. continuous deployment". atlassian. retrieved - - . ^ "corporate identity". ionos startupguide. retrieved - - . external links[edit] what is ci/cd - all you need to know by maciej manturewicz retrieved from "https://en.wikipedia.org/w/index.php?title=ci/cd&oldid= " categories: software development software engineering promotion and marketing communications navigation menu personal tools not logged in talk contributions create account log in namespaces article talk variants views read edit view history more search navigation main page contents current events random article about wikipedia contact us donate contribute help learn to edit community portal recent changes upload file tools what links here related changes upload file special pages permanent link page information cite this page wikidata item print/export download as pdf printable version languages العربية Čeština español فارسی עברית magyar 日本語 português Русский 中文 edit links this page was last edited on february , at : (utc). text is available under the creative commons attribution-sharealike license; additional terms may apply. by using this site, you agree to the terms of use and privacy policy. wikipedia® is a registered trademark of the wikimedia foundation, inc., a non-profit organization. privacy policy about wikipedia disclaimers contact wikipedia mobile view developers statistics cookie statement developer agreement and policy – twitter developers | twitter developer developer terms developer agreement and policy developer agreement effective: march , this twitter developer agreement (“agreement”) is made between you (either an individual or an entity, referred to herein as “you”) and twitter (as defined below) and governs your access to and use of the licensed material (as defined below).your use of twitter’s websites, sms, apis, email notifications, applications, buttons, embeds, ads, and our other covered services is governed by our general terms of service and privacy policy. please read the terms and conditions of this agreement carefully, including any linked terms referenced below, which are part of this license agreement. by using the licensed material, you are agreeing that you have read, and that you agree to comply with and to be bound by the terms and conditions of this agreement and all applicable laws and regulations in their entirety without limitation or qualification. if you do not agree to be bound by this agreement, then you may not access or otherwise use the licensed material. this agreement is effective as of the first date that you use the licensed material (“effective date”). if you are an individual representing an entity, you acknowledge that you have the appropriate authority to accept this agreement on behalf of such entity. you may not use the licensed material and may not accept this agreement if you are not of legal age to form a binding contract with twitter, or you are barred from using or receiving the licensed material under applicable law. i. twitter api and twitter content a. definitions broadcast id - a unique identification number generated for each periscope broadcast. developer site ‒ twitter’s developer site located at https://developer.twitter.com end users ‒ users of your services. licensed material ‒ a collective term for the twitter api, twitter content and twitter marks. periscope broadcast - a live or on-demand video stream that is publicly displayed on twitter applications and is generated by a user via twitter’s periscope producer feature (as set forth at https://help.periscope.tv/customer/en/portal/articles/ ). services ‒ your services, websites, applications and other offerings (including research) that display twitter content or otherwise use the licensed material. tweet id ‒ a unique identification number generated for each tweet. tweet ‒ a posting made on twitter applications. twitter content ‒ tweets, tweet ids, twitter end user profile information, periscope broadcasts, broadcast ids and any other data and information made available to you through the twitter api or by any other means authorized by twitter, and any copies and derivative works thereof. “twitter” means twitter, inc., with an office located at market street, suite , san francisco, ca, , usa. if you enter into this agreement or an order outside of the united states, canada or latin america, twitter international company with its registered offices at one cumberland place, fenian street, dublin , d ax , ireland (“tic”) is the contracting entity. direct message - a message that is privately sent on twitter applications by one end user to one or more specific end user(s) using twitter’s direct message function. twitter api ‒ the twitter application programming interface (“api”), software development kit (“sdk”) and/or the related documentation, data, code, and other materials provided by twitter with the api, as updated from time to time, including without limitation through the developer site. twitter marks ‒ the twitter name, trademarks, and logos that twitter makes available to you, including via the developer site. twitter applications ‒ twitter’s consumer facing products, services, applications, websites, web pages, platforms, and other offerings, including without limitation, those offered via https://twitter.com and twitter’s mobile applications. b. license from twitter. subject to the terms and conditions in this agreement and the developer policy (as a condition to the grant below), twitter hereby grants you and you accept a non-exclusive, royalty free, non-transferable, non-sublicensable, revocable license solely to: use the twitter api to integrate twitter content into your services or conduct analysis of such twitter content, as explicitly approved by twitter; copy a reasonable amount of and display the twitter content on and through your services to end users, as permitted by this agreement; modify twitter content only to format it for display on your services; and use and display twitter marks, solely to attribute twitter’s offerings as the source of the twitter content, as set forth herein. c. license to twitter you hereby grant twitter and twitter accepts a non-exclusive, royalty free, non-transferable, non-sublicensable revocable license to access, index, and cache by any means, including web spiders and/or crawlers, any webpage or applications on which you display twitter content using embedded tweets or embedded timelines. d. incorporated terms. your use of the licensed material is further subject to and governed by the following terms and conditions: the twitter developer policy; the api restricted use rules; the twitter rules; as it relates to your display of any of the twitter content, the display requirements; as it relates to your use and display of the twitter marks, the twitter brand resources; as it relates to taking automated actions on your account, the automation rules; as it relates to your use of periscope, the periscope community guidelines, and the periscope trademark guidelines. the developer policy, api restricted use rules, twitter rules, display requirements, brand resources, automation rules, periscope community guidelines, and periscope trademark guidelines are collectively referred to herein as the “incorporated developer terms”. you agree to the incorporated developer terms, which are hereby incorporated by reference and are available in hardcopy upon request to twitter. in the event of a conflict between the incorporated developer terms and this agreement, this agreement shall control. none of the incorporated developer terms expand or extend the license to the twitter api, twitter content or twitter marks granted in this agreement. ii. restrictions on use of licensed materials a. reverse engineering and other restrictions. you will not or attempt to (and will not allow others to) a) reverse engineer, decompile, disassemble or translate the twitter api, or otherwise attempt to derive source code, trade secrets or know-how in or underlying any twitter api or any portion thereof; b) interfere with, modify, disrupt or disable features or functionality of the twitter api, including without limitation any such mechanism used to restrict or control the functionality, or defeat, avoid, bypass, remove, deactivate or otherwise circumvent any software protection or monitoring mechanisms of the twitter api; b) sell, rent, lease, sublicense, distribute, redistribute, syndicate, create derivative works of, assign or otherwise transfer or provide access to, in whole or in part, the licensed material to any third party except as expressly permitted herein; d) provide use of the twitter api on a service bureau, rental or managed services basis or permit other individuals or entities to create links to the twitter api or "frame" or "mirror" the twitter api on any other server, or wireless or internet-based device, or otherwise make available to a third party, any token, key, password or other login credentials to the twitter api; or e) use the licensed material for any illegal, unauthorized or other improper purposes, including without limitation to store or transmit infringing, libelous, or otherwise unlawful or tortious material, to store or transmit malicious code, or to store or transmit material in violation of third-party privacy rights; (f) utilize the licensed material to derive or obtain non-public information of individual twitter users, including without limitation a user’s location; (g) interfere with or disrupt the integrity or performance of the twitter applications, twitter api or twitter content contained therein, including by disrupting the ability of any other person to use or enjoy the twitter applications, twitter api or twitter content, or attempt to gain unauthorized access to the twitter applications, twitter api, twitter content or related systems or networks; (h) remove or alter any proprietary notices or marks on the twitter content; or (i) use twitter content, by itself or bundled with third party data, or derivative analysis therefrom, to target users with advertising outside of the twitter applications, including without limitation on other advertising networks, via data brokers, or through any other advertising or monetization services. b. commercial use restrictions. if your project (located in your twitter developer portal) is designated as ‘non-commercial’ you shall not make commercial use of the licensed materials. commercial use restrictions may not apply to those officially registered: ( ) non-profits or ( ) ngos. “commercial use” means any use of the licensed materials (which includes access to the twitter api): (i) by (or on behalf of) a business (i.e. entity whose primary purpose is to earn revenue through a product or service) or (ii) as part of a product or service that is monetized (including but not limited to website advertising, licensing fees, in-app promotions, and sponsorships). c. no monitoring or measuring. despite any other provision herein, you may only use the following information for non-commercial, internal purposes (e.g., to improve the functionality of the services): (a) aggregate twitter applications user metrics, such as number of active users or accounts on twitter applications; (b) the responsiveness of twitter applications; and (c) results, usage statistics, data or other information (in the aggregate or otherwise) derived from analyzing, using, or regarding the performance of the twitter api. all such information is twitter’s confidential information. d. rate limits. you will not attempt to exceed or circumvent limitations on access, calls and use of the twitter api ("rate limits"), or otherwise use the twitter api in a manner that exceeds reasonable request volume, constitutes excessive or abusive usage, or otherwise fails to comply or is inconsistent with any part of this agreement. if you exceed or twitter reasonably believes that you have attempted to circumvent rate limits, controls to limit use of the twitter apis or the terms and conditions of this agreement, then your ability to use the licensed materials may be temporarily suspended or permanently blocked. twitter may monitor your use of the twitter api to improve the licensed materials and twitter applications and to ensure your compliance with this agreement and incorporated developer terms. e. location data. you will not (and you will not allow others to) aggregate, cache, or store location data and other geographic information contained in the twitter content, except in conjunction with the twitter content to which it is attached. you may only use such location data and geographic information to identify the location tagged by the twitter content. you may not use location data or geographic information on a standalone basis. f. use of twitter marks. the twitter marks may not be included in or as part of your registered corporate name, any of your logos, or any of your service or product names. moreover, you may not create any derivative works of the twitter marks or use the twitter marks in a manner that creates or reasonably implies an inaccurate sense of endorsement, sponsorship, or association with twitter. you will not otherwise use business names and/or logos in a manner that can mislead, confuse, or deceive users of your services. all use of the twitter marks and all goodwill arising out of such use, will inure to twitter's benefit. you shall not use the twitter marks except as expressly authorized herein without twitter's prior consent. you will not remove or alter any proprietary notices or twitter marks on the licensed material. g. security. you will maintain the security of the twitter api and will not make available to a third party, any token, key, password or other login credentials to the twitter api. you will use industry standard security measures to prevent unauthorized access or use of any of the features and functionality of the twitter api, including access by viruses, worms, or any other harmful code or material. additionally, you will keep twitter content (including, where applicable, personal data) confidential and secure from unauthorized access by using industry-standard organizational and technical safeguards for such data, and with no less care than it uses in connection with securing similar data you store. you will immediately notify twitter consult and cooperate with investigations, assist with any required notices, and provide any information reasonably requested by twitter if you know of or suspects any breach of security or potential vulnerability related to the licensed material and will promptly remedy such breach or potential vulnerability resulting from your access to the licensed material. iii. updates and removals a. updates. you acknowledge that twitter may update or modify the licensed materials from time to time, and at its sole discretion (in each instance, an “update”). you are required to implement and use the most current version of the licensed materials and to make any changes to your services that are required as a result of such update, at your sole cost and expense. updates may adversely affect the manner in which your services access or communicate with the twitter api or display twitter content. your continued access or use of the licensed materials following an update or modification will constitute binding acceptance of the update. b. removals. if twitter content is deleted, gains protected status, or is otherwise suspended, withheld, modified, or removed from the twitter applications (including removal of location information), you will make all reasonable efforts to delete or modify such twitter content (as applicable) as soon as possible, and in any case within hours after a written request to do so by twitter or by a twitter user with regard to their twitter content, unless prohibited by applicable law or regulation and with the express written permission of twitter. iv. ownership and feedback a. ownership. the licensed material is licensed, not sold, and twitter retains and reserves all rights not expressly granted in this agreement. you expressly acknowledge that twitter, its licensors and its end users retain all worldwide right, title and interest in and to the licensed material, including all rights in patents, trademarks, trade names, copyrights, trade secrets, know-how, data (including all applications therefor), and all proprietary rights under the laws of the united states, any other jurisdiction or any treaty ("ip rights"). you agree not to do anything inconsistent with such ownership, including without limitation, challenging twitter’s ownership of the twitter marks, challenging the validity of the licenses granted herein, or otherwise copying or exploiting the twitter marks during or after the termination of this agreement, except as specifically authorized herein. if you acquire any rights in the twitter marks or any confusingly similar marks, by operation of law or otherwise, you will, at no expense to twitter, immediately assign such rights to twitter. b. feedback. you may provide twitter with comments concerning the licensed material, twitter applications or your evaluation and use thereof (collectively, "feedback"). you hereby grant twitter all rights, title and ownership of such feedback (including all intellectual property rights therein), and twitter may use the feedback for any and all commercial and non-commercial purposes with no obligation of any kind to you. v. termination. twitter may immediately terminate or suspend this agreement, any rights granted herein, and/or your license to the licensed materials, at its sole discretion at any time, for any reason by providing notice to you. you may terminate this agreement at any time by ceasing your access to the twitter api and use of all twitter content. upon termination of this agreement, (a) all licenses granted herein immediately expire and you must cease use of all licensed materials; and (b) you shall permanently delete all licensed material and twitter marks in all forms and types of media, and copies thereof, in your possession. the parties to this agreement will not be liable to each other for any damages resulting solely from termination of this agreement as permitted under this agreement. sections ii (restrictions on use of licensed materials), iv (ownership and feedback), v (termination), vi (confidentiality), vii (compliance audit), viii (warranty disclaimer), ix (indemnification), x (limitation of liability) and xii (miscellaneous) of this agreement will survive the termination of this agreement. vi. confidentiality. you may be given access to certain non-public information, software, and specifications relating to the licensed material (“confidential information”), which is confidential and proprietary to twitter. you may use this confidential information only as necessary in exercising your rights granted in this agreement. you may not disclose any of this confidential information to any third party without twitter’s prior written consent. you agree that you will protect this confidential information from unauthorized use, access, or disclosure in the same manner that you would use to protect your own confidential and proprietary information of a similar nature and in no event with less than a reasonable degree of care. vii. compliance audit. twitter, or a third party agent subject to obligations of confidentiality, shall be entitled to inspect and audit any records or activity related to your access to the licensed material for the purpose of verifying compliance with this agreement. twitter may exercise its audit right at any time upon notice. you will provide your full cooperation and assistance with such audit and provide access to all licensed material in your possession or control, applicable agreements and records. without limiting the generality of the foregoing, as part of the audit, twitter may request, and you agree to provide, a written report, signed by an authorized representative, listing your then-current deployment of the licensed material and twitter content. the rights and requirements of this section will survive for one ( ) year following the termination of this agreement. viii. warranty disclaimer. the licensed material is provided to you “as is”, “where is”, with all faults and each party disclaims all warranties, whether express, implied, statutory, or otherwise, including without limitation warranties of merchantability, noninfringement, fitness for a particular purpose, and any warranties or conditions arising out of this agreement, course of dealing or usage of trade. twitter does not warrant that the licensed material or any other twitter product or service provided hereunder will meet any of your requirements or that use of such licensed material or other products or services will be error-free, uninterrupted, virus-free or secure. this disclaimer of warranty may not be valid in some jurisdictions and you may have warranty rights under law which may not be waived or disclaimed. any such warranty extends only for thirty ( ) days from the effective date of this agreement (unless such law provides otherwise). ix. indemnification. you shall defend twitter against any and all actions, demands, claims and suits (including without limitation product liability claims), and indemnify and hold twitter harmless from any and all liabilities, damages and costs (including without limitation reasonable attorneys' fees) to the extent arising out of: (i) your use of the licensed material in any manner that is inconsistent with this agreement; or (ii) the performance, promotion, sale or distribution of your services. in the event twitter seeks indemnification or defense from you under this provision, twitter will promptly notify you in writing of the claim(s) brought against twitter for which it seeks indemnification or defense. twitter reserves the right, at its option and sole discretion, to assume full control of the defense of claims with legal counsel of its choice. you may not enter into any third party agreement, which would, in any manner whatsoever, affect the rights of twitter, constitute an admission of fault by twitter or bind twitter in any manner, without the prior written consent of twitter. in the event twitter assumes control of the defense of such claim, twitter shall not settle any such claim requiring payment from you without your prior written approval. x. limitation of liability. in no event will twitter be liable to you or any end users for any indirect, special, incidental, exemplary, punitive or consequential damages or any loss of or damage to use, data, business, goodwill or profits arising out of or in connection with this agreement. in any case, twitter's aggregate liability for any and all claims under this agreement will not exceed $ . usd. the foregoing limitations, exclusions and disclaimers shall apply regardless of whether such liability arises from any claim based upon contract, warranty, tort (including negligence), strict liability or otherwise, and whether or not twitter has been advised of the possibility of such loss or damage. insofar as applicable law prohibits any limitation on liability herein, the parties agree that such limitation will be automatically modified, but only to the extent so as to make the limitation compliant with applicable law. the parties agree that the limitations on liabilities set forth herein are agreed allocations of risk and such limitations will apply notwithstanding the failure of essential purpose of any limited remedy. xi. agreement updates. twitter may update or modify this agreement or any of the incorporated developer terms from time to time at its sole discretion by posting the changes on this site or by otherwise notifying you (such notice may be via email). you acknowledge that these updates and modifications may adversely affect how your service accesses or communicates with the twitter api. if any change is unacceptable to you, your only recourse is to cease all use of the licensed material. your continued access or use of the licensed material will constitute binding acceptance of such updates and modifications. xii. miscellaneous. a. assignment. you may not assign any of the rights or obligations granted hereunder, in whole or in part, whether voluntarily or by operation of law, contract, merger (whether you are the surviving or disappearing entity), stock or asset sale, consolidation, dissolution, through government action or otherwise, except with the prior written consent of twitter, inc. twitter, inc. is authorized to sign modifications and consents on behalf of twitter international company, an irish company responsible for the information of twitter users who live outside the united states. any attempted assignment in violation of this paragraph is null and void, and twitter may terminate this agreement. b. user protection. unless explicitly approved otherwise by twitter in writing, you may not use, or knowingly display, distribute, or otherwise make twitter content, or information derived from twitter content, available to any entity for the purpose of: (a) conducting or providing surveillance or gathering intelligence, including but not limited to investigating or tracking twitter users or twitter content; (b) conducting or providing analysis or research for any unlawful or discriminatory purpose, or in a manner that would be inconsistent with twitter users' reasonable expectations of privacy; (c) monitoring sensitive events (including but not limited to protests, rallies, or community organizing meetings); or (d) targeting, segmenting, or profiling individuals based on sensitive personal information, including their health (e.g., pregnancy), negative financial status or condition, political affiliation or beliefs, racial or ethnic origin, religious or philosophical affiliation or beliefs, sex life or sexual orientation, trade union membership, twitter content relating to any alleged or actual commission of a crime, or any other sensitive categories of personal information prohibited by law. c. government use. if you will display, distribute, or otherwise make available any twitter content to end users that are, or that act on behalf of, any government-related entity (each a “government end user”), you will identify all such government end users when submitting that use case for review to twitter and will thereafter notify twitter in writing of any new government end users, and any new use cases with existing government end users, prior to the services displaying, distributing, or otherwise making available any twitter content to a government end user or for any new use case, and twitter will have the right at anytime to prohibit you from making twitter content available to any new government end user. in no event shall your use, or knowingly display, distribute, or otherwise make twitter content, or information derived from twitter content, available to any government end user whose primary function or mission includes conducting surveillance or gathering intelligence. if law enforcement personnel request information about twitter or its users for the purposes of an ongoing investigation, you may refer them to twitter’s guidelines for law enforcement located at https://t.co/le. the twitter api and twitter content are "commercial items" as that term is defined at c.f.r. . , consisting of "commercial computer software" and "commercial computer software documentation" as such terms are used in c.f.r. . . any use, modification, derivative, reproduction, release, performance, display, disclosure or distribution of the twitter api or twitter content by any government entity is prohibited, except as expressly permitted by the terms of this agreement. additionally, any use by u.s. government entities must be in accordance with c.f.r. . and c.f.r. . - through . - . if you use the twitter api or twitter content in your official capacity as an employee or representative of a u.s., state or local government entity and you are legally unable to accept the indemnity, jurisdiction, venue or other clauses herein, then those clauses do not apply to such entity, but only to the extent as required by applicable law. contractor/manufacturer is twitter, inc. market street, suite , san francisco, california . d. compliance with laws; export and import. each party will comply with all applicable foreign, federal, state, and local laws, rules and regulations, including without limitation, all applicable laws relating to bribery and/or corruption. the licensed material is subject to u.s. export laws and may be subject to import and use laws of the country where it is delivered or used. you agree to abide by these laws. under these laws, the licensed material may not be sold, leased, downloaded, moved, exported, re-exported, or transferred across borders without a license, or approval from the relevant government authority, to any country or to any foreign national restricted by these laws, including countries embargoed by the u.s. government (currently cuba, iran, north korea, northern sudan and syria); or to any restricted or denied end-user including, but not limited to, any person or entity prohibited by the u.s. office of foreign assets control; or for any restricted end-use. you will maintain all rights and licenses that are required with respect to your services. e. data protection addendum. twitter international company (“tic”), an irish registered company, controls some of the twitter content, as set forth in the twitter privacy policy, and has authorized twitter to license such twitter content under this agreement (such twitter content is “twitter european data”). to the extent that you receive twitter european data, you agree that in addition to this agreement, the twitter controller-to-controller data protection addendum located at https://gdpr.twitter.com/en/controller-to-controller-transfers.html shall apply to twitter european data and is hereby incorporated by reference. f. governing law; dispute resolution. this agreement will be governed by and construed in accordance with the laws of the state of california, without regard to or application of conflicts of law rules or principles. any dispute, claim or controversy arising out of or relating to this agreement or the breach, termination, enforcement, interpretation or validity thereof, including the determination of the scope or applicability of this agreement to arbitrate, shall be determined by arbitration in san francisco, ca before a single arbitrator. the arbitration shall be administered by jams pursuant to its comprehensive arbitration rules and procedures. judgment on the award may be entered in any court having jurisdiction. you and twitter hereby expressly waive trial by jury. as an alternative, you may bring your claim in your local "small claims" court, if permitted by that small claims court's rules. you may bring claims only on your own behalf, and unless twitter agrees, the arbitrator may not consolidate more than one person's claims. despite the foregoing, you agree that money damages would be an inadequate remedy for twitter in the event of a breach or threatened breach of a provision of this agreement protecting twitter's intellectual property or confidential information, and that in the event of such a breach or threat, twitter, in addition to any other remedies to which it is entitled, is entitled to such preliminary or injunctive relief (including an order prohibiting company from taking actions in breach of such provisions), without the need for posting bond, and specific performance as may be appropriate. the parties agree that neither the united nations convention on contracts for the international sale of goods, nor the uniform computer information transaction act (ucita) shall apply to this agreement, regardless of the states in which the parties do business or are incorporated. no waiver by twitter of any covenant or right under this agreement will be effective unless memorialized in a writing duly authorized by twitter. g. severability. if any part of this agreement is determined to be invalid or unenforceable by a court of competent jurisdiction, that provision will be enforced to the maximum extent permissible and the remaining provisions of this agreement will remain in full force and effect. h. entire agreement. this agreement constitutes the entire agreement among the parties with respect to the subject matter and supersedes and merges all prior proposals, understandings and contemporaneous communications. any modification to this agreement must be in a writing signed by both you and twitter, inc. this agreement does not create or imply any partnership, agency or joint venture. developer policy twitter + developers twitter loves developers. we’re delighted and amazed by the tools and services this community creates by harnessing the power of twitter data. as part of our commitment to this community, we aim to provide data access that is open and fair for developers, safe for people on twitter, and beneficial for the twitter platform as a whole. to further these goals we’ve crafted the developer policy as a guide to help people understand our rules and expectations about appropriate api and twitter content usage. this developer policy (“policy”) provides rules and guidelines for developers who interact with twitter’s ecosystem of applications, services, website, web pages and content. it is part of your contract with twitter governing access to and use of the twitter api and twitter content (either as part of the developer agreement or other written agreement with twitter). policy violations are considered violations of your agreement. this policy may be changed from time to time without notice. capitalized terms used in this policy, which are not defined in this policy, will have the respective meanings ascribed to them in the developer agreement or the master license agreement. using this policy we’ve structured this policy to make it as easy to follow as possible. please keep information from the following policy sections top of mind as you use the twitter api and twitter content: set yourself up for success - you are responsible for complying with all twitter policies. it’s important that you review and understand this policy, as well as the policies we link to in this document, before you access the twitter api and twitter content. the time spent reviewing our policies may save you hours of rework down the road. privacy and control are essential - protecting and defending the privacy of people on twitter is built into the core dna of our company. as such, we prohibit the use of twitter data in any way that would be inconsistent with people’s reasonable expectations of privacy. by building on the twitter api or accessing twitter content, you have a special role to play in safeguarding this commitment, most importantly by respecting people’s privacy and providing them with transparency and control over how their data is used. follow the platform usage guidelines - getting approved to access the twitter api and twitter content is just the first step. our platform usage guidelines should be your first stop anytime you have questions about how to ensure policy compliance for your planned use of the twitter platform. we’ve provided a lot more detail on what each of these three key sections mean below. please review them carefully to ensure that your usage of the twitter api and twitter content is consistent with our policies. if we believe you are in violation of this policy (or any other twitter policy), we may suspend or permanently revoke your access to the twitter api and twitter content. if this happens to you, do not apply for or register additional api keys. instead, contact us via the api policy support form. finally, please note that twitter may monitor your use of the twitter api to improve the twitter applications, to examine any commercial use, and to ensure your compliance with your approved use case and this policy. thanks for reading, and thank you for building with us! we look forward to seeing what you create! set yourself up for success you can avoid many potential pitfalls while using the twitter api by ensuring that your service has been built the right way from day . this section of the developer policy contains rules that all developers must follow before using the twitter api or twitter content. we review all proposed uses of the twitter developer platform to verify policy compliance — so you’re required to disclose (and update, as applicable) your planned use of the twitter api and twitter content in order to be granted and to maintain access. all new developers must apply for a developer account to access the twitter api. current developers without an approved developer account must apply for one as directed to do so by twitter. as part of this process, you’ll need to provide us with a written description of your intended uses of the twitter api and twitter content. your use case description is binding on you, and any substantive deviation from it may constitute a violation of our rules and result in enforcement action. you must notify us of any substantive modification to your use case and receive approval before you may begin using twitter content for that new purpose. failure to do so may result in suspension and termination of your api and data access. you can update your use case by visiting our api policy support form, selecting i need to update my developer use case, or as otherwise agreed by twitter. by building on the twitter api or accessing twitter content, you must comply with all twitter policies. these include this developer policy, the automation rules, the display requirements, the api restricted uses rules, the twitter rules, the twitter brand resources, the periscope community guidelines, and the periscope trademark guidelines, as well as any other agreements you enter into with twitter relating to your use of the twitter api or twitter content, including but not limited to the developer agreement or a master licensing agreement or order (as applicable). you must also comply with any modifications to these policies and any new policies launched by twitter. it is your responsibility to monitor the use of your service and to design your service to prevent violations of twitter policy by people who use it. failure to do so may result in suspension or termination of your api and twitter content access. you may not register multiple applications for a single use case or substantially similar or overlapping use cases. in this context, a “use case” is a consistent set of analyses, displays, or actions performed via an application. please note that providing the same service or application to different people (including “white label” versions of a tool or service) counts as a single use case. as a single exception to these rules, you may create and use a maximum of applications for development, staging, and production instances of the same service. these apps must be registered to a single account, and should be clearly identified (in the name and description) as dev, staging, and prod instances of a single service. you may not use development or staging applications for production purposes. you must keep all api keys or other access credentials private. you may not use, and may not encourage or facilitate others to use, api keys or other access credentials owned by others. your license agreement with twitter limits your use of the twitter api and twitter content. among other things, the twitter api has rate limits which help to ensure fair data usage and to combat spam on the platform. you may not exceed or circumvent rate limits, or any other limitations or restrictions described in this policy or your agreement with twitter, listed on the developer site, or communicated to you by twitter. you may not remove or alter any proprietary notices or marks on twitter content received via the twitter api. this helps to make sure that people know where twitter content is coming from, and who it belongs to. for data integrity and platform health reasons, you may not interfere with, intercept, disrupt, or disable any features of the twitter api or the twitter service. in other words, use the apis as intended and documented on developer.twitter.com. refer to our hackerone guidelines for more details about acceptable use. privacy and control are essential twitter takes privacy seriously, and we expect everyone using twitter content and the twitter api to do the same. any use of the twitter developer platform, twitter api, or twitter content in a manner that is inconsistent with peoples’ reasonable expectations of privacy may be subject to enforcement action, which can include suspension and termination of api and twitter content access. your commitment to privacy and control must extend to all uses of twitter content and all aspects of the service that you build using our api. to that end, the people using your service must understand and consent to how you use their data, and how you access twitter on their behalf. this can be accomplished through providing people with a clear, comprehensive, and transparent privacy policy, as well as ensuring that you get express and informed consent from each person using your service before taking any action on their behalf. please note that a person authenticating into your service does not by itself constitute consent. consent & permissions in particular, you must get express and informed consent from people before doing any of the following: taking any actions on their behalf. this includes (but is not limited to): posting content to twitter following/unfollowing accounts modifying profile or account information starting a periscope broadcast adding hashtags or any other content to tweets republishing content accessed by means other than via the twitter api or other twitter tools using someone’s twitter content to promote a product or service storing non-public content such as direct messages (dms), or any other private or confidential information sharing or publishing protected content, or any other private or confidential information if your service allows people to post content to twitter you must do the following before publishing: show exactly what will be published make it clear to people using your service what geo information (if any) will be added to the content if your service allows people to post content to both your service and twitter, you must do the following before publishing: obtain permission to post the content explain where you will post the content you must respect the protected and blocked status of all twitter content. you may not serve content obtained using one person’s authentication token to a different person who is not authorized to view that content. protected accounts: a protected account’s content is only available to people who have been approved by the owner to follow that account. so, if you run a service that accesses protected accounts, you may only do so to serve such content to the specific people with permission to view that content. blocked accounts: people on twitter are able to block access to their accounts for any reason they choose. commingling information obtained from tokens (or any other api-based action) to bypass this choice is not permitted. as direct messages (dms) are non-public in nature, services that provide dm features must take extra steps to safeguard personal privacy. you may not serve dm content to people who are not authorized to view that content. if your service provides dm functionality you must also: notify people if you send read receipt events for dms. you can do this by providing a notice directly in your service, or by displaying read receipts from other participants in a conversation. get consent before configuring media to be sent in a dm as "shared" (i.e. reusable across multiple dms). if you do allow media in a dm to be “shared,” you must provide a clear notice that this content will be accessible to anyone with the media’s url. content compliance if you store twitter content offline, you must keep it up to date with the current state of that content on twitter. specifically, you must delete or modify any content you have if it is deleted or modified on twitter. this must be done as soon as reasonably possible, or within hours after receiving a request to do so by twitter or the applicable twitter account owner, or as otherwise required by your agreement with twitter or applicable law. this must be done unless otherwise prohibited by law, and only then with the express written permission of twitter. modified content can take various forms. this includes (but is not limited to): content that has been made private or gained protected status content that has been suspended from the platform content that has had geotags removed from it content that has been withheld or removed from twitter off-twitter matching we limit the circumstances under which you may match a person on twitter to information obtained or stored off-twitter. off-twitter matching involves associating twitter content, including a twitter @handle or user id, with a person, household, device, browser, or other off-twitter identifier. you may only do this if you have express opt-in consent from the person before making the association, or as described below. in situations in which you don’t have a person’s express, opt-in consent to link their twitter identity to an off-twitter identifier, we require that any connection you draw be based only on information that someone would reasonably expect to be used for that purpose. in addition, absent a person’s express opt-in consent you may only attempt to match your records about someone to a twitter identity based on: information provided directly to you by the person. note that records about individuals with whom you have no prior relationship, including data about individuals obtained from third parties, do not meet this standard; and/or public data. “public data” in this context refers to: information about a person that you obtained from a public, generally-available resource (such as a directory of members of a professional association) information on twitter about a person that is publicly available, including: tweets profile information, including an account bio and publicly-stated location display name and @handle your privacy policy you must display your service’s privacy policy to people before they are permitted to download, install, or sign up to your service. it must disclose at least the following information: the information that you collect from people who use your service how you use and share that information (including with twitter) how people can contact you with inquiries and requests regarding their information your privacy policy must be consistent with all applicable laws, and be no less protective of people than twitter’s privacy policy and the privacy policy of our other services and corporate affiliates. you must cease your access to the twitter api and the use of all twitter content if you are unable to comply with your and/or twitter’s privacy policy. using geo-data use of geo data comes with additional restrictions due to the sensitive nature of this information. if your service adds location information to tweets or periscope broadcasts, you must disclose to people: when you add location information whether you add location information as a geotag or annotations data whether your location information is listed as a place, or as geographic coordinates if your application allows people to tweet with their location you must comply with twitter’s geo guidelines in full. any use of location data or geographic information on a standalone basis is prohibited. you may not (and may not permit others to) store, aggregate, or cache location data and other geographic information contained in twitter content, except as part of a tweet or periscope broadcast. for example, you may not separate location data or geographic information out from tweets to show where individuals have been over time. heat maps and related tools that show aggregated geo activity (e.g.: the number of people in a city using a hashtag) are permitted. twitter passwords you may not store twitter passwords, or request that people provide their twitter password, account credentials, or developer application information (including consumer key) to you directly. we suggest the use of sign-in with twitter as the authentication tool to link your service and people on twitter. platform usage guidelines have you taken care to review twitter’s policies and set up your api access the right way? does your service follow twitter’s privacy and control guidelines? if you can answer yes to these two questions, then you are ready to start using the twitter api and twitter content. twitter’s platform usage guidelines provide the assistance needed to ensure that your use of twitter content is compliant from day throughout the lifecycle of your service. we suggest reviewing these rules on a regular basis to make sure that your integration is operating in a way that is safe and beneficial to people on twitter and the twitter platform as a whole. spam, bots, and automation the use of the twitter api and developer products to create spam, or engage in any form of platform manipulation, is prohibited. you should review the twitter rules on platform manipulation and spam, and ensure that your service does not, and does not enable people to, violate our policies. services that perform write actions, including posting tweets, following accounts, or sending direct messages, must follow the automation rules. in particular, you should: always get explicit consent before sending people automated replies or direct messages immediately respect requests to opt-out of being contacted by you never perform bulk, aggressive, or spammy actions, including bulk following never post identical or substantially similar content across multiple accounts if you’re operating an api-based bot account you must clearly indicate what the account is and who is responsible for it. you should never mislead or confuse people about whether your account is or is not a bot. a good way to do this is by including a statement that the account is a bot in the profile bio. twitter performance benchmarking you may not use the twitter api to measure the availability, performance, functionality, or usage of twitter for benchmarking, competitive, or commercial purposes. for example, you should never use the twitter api to: calculate aggregate twitter metrics, such as the total number of monthly actives (mas) or daily actives (das) calculate aggregate periscope metrics, such as total number of broadcast views calculate aggregate twitter tweet metrics, such as the total number of tweets posted per day, or the number of account engagements measure or analyze the responsiveness of twitter measure or analyze spam or security on twitter, except as permitted below we support research that helps improve conversational health on twitter. you may use the twitter api and twitter content to measure and analyze topics like spam, abuse, or other platform health-related topics for non-commercial research purposes. you may not develop, create, or offer commercial services using the twitter api or twitter content that measure, analyze, or attempt to identify behaviors or content which violate twitter policies without express written permission from twitter. if you have questions about whether your use case qualifies as non-commercial research for this purpose please submit a request via the api policy support form. public display of tweets you must maintain the integrity of all twitter content that you display publicly or to people who use your service. if you don’t use twitter for websites to display content, then you must use the twitter api to retrieve the most current version available for display. if displayed content ceases to be available through the twitter api, then you must remove it from your service as soon as reasonably possible, or within hours after the receipt of a removal request from twitter, or the applicable twitter account owner, or as otherwise required by applicable law. there are specific rules you must follow if you display twitter content offline. follow the guidelines for using tweets in broadcast if you display tweets offline. follow the guidelines for using periscope broadcasts if you display periscope broadcasts offline. if you embed or display tweets, you must contact us about your twitter api access if your site exceeds million daily impressions. twitter reserves the right to require additional terms as a condition to your use of the twitter api. additional restrictions on twitter for websites developer use include: embedded tweets and/or embedded timelines you must provide people with legally sufficient notice that fully discloses twitter’s collection and use of data about browsing activities on your website, including for interest-based advertising and personalization. you must also obtain legally sufficient consent from people for such collection and use you must provide legally sufficient instructions on how people can opt out of twitter’s interest-based advertising and personalization as described here twitter for websites widgets you must ensure that people are provided with clear and comprehensive information about, and consent to, the storing and accessing of cookies or other information on their devices as described in twitter’s cookie use, where providing such information and obtaining such consent is required by law services targeted to children under services targeted to children under must opt out of tailoring twitter in any embedded tweet and/or embedded timelines by setting the opt-out parameter to be ‘true’ as described here content redistribution the best place to get twitter content is directly from twitter. consequently, we restrict the redistribution of twitter content to third parties. if you provide twitter content to third parties, including downloadable datasets or via an api, you may only distribute tweet ids, direct message ids, and/or user ids (except as described below). we also grant special permissions to academic researchers sharing tweet ids and user ids for non-commercial research purposes. in total, you may not distribute more than , , tweet ids to any entity (inclusive of multiple individuals associated with a single entity) within any day period unless you have received written permission from twitter. in addition, all developers may provide up to , public tweets objects and/or user objects to each person who uses your service on a daily basis if this is done via non-automated means (e.g., download of spreadsheets or pdfs). academic researchers are permitted to distribute an unlimited number of tweet ids and/or user ids if they are doing so on behalf of an academic institution and for the sole purpose of non-commercial research. for example, you are permitted to share an unlimited number of tweet ids for the purpose of enabling peer review or validation of your research. if you have questions about whether your use case qualifies under this category please submit a request via the api policy support form. any twitter content provided to third parties remains subject to this policy, and those third parties must agree to the twitter terms of service, privacy policy, developer agreement, and developer policy before receiving such downloads. you may not enable any entity to circumvent any other limitations or restrictions on the distribution of twitter content as contained in this policy, the developer agreement, or any other agreement with twitter. replicating the twitter experience the best place to experience twitter is on twitter owned and operated (too) products. as such, we discourage developers from building services that replicate twitter’s core experience or features. if you create a service that replicates twitter’s core experience or features you will be subject to additional rules beyond what is already included in the developer policy. in particular, you must: obtain our permission to have more than , tokens. you may be subject to additional terms if this request is approved use the twitter api for functionalities in your service that are substantially similar to twitter features. this includes, but is not limited to, tweets, follows, unfollows, retweets, likes, comments, and replies. display a prominent link or button in your service that directs new people to twitter’s account creation page if you create a service that replicates twitter’s core experience or features you may not do any of the following: pay, or offer to pay, third parties for distribution. this includes offering compensation for downloads (other than transactional fees) or other mechanisms of traffic acquisition arrange for your service to be pre-installed on any other device, promoted as a "zero-rated" service, or marketed as part of a specialized data plan use twitter content or other data collected from people to create or maintain a separate conversational platform, social network, status update, private messaging or live broadcasting database or service pay to engage your service shouldn’t compensate people to take actions on twitter, as that results in inauthentic engagement that degrades the health of the platform. as you use the twitter api you may not sell or receive monetary or virtual compensation for any twitter or periscope actions. this includes, but is not limited to, tweets, follows, unfollows, retweets, likes, comments, and replies. service authenticity you must clearly identify your service so that people can understand its source and purpose. don’t use names, logos, or urls that mask your service’s identity and features, or that falsely imply an affiliation with twitter or third parties. note that creating applications for the purpose of selling names, or to prevent others from using names, is prohibited. you may not use any url (including shortened urls) for your service that directs people to: a site that is unrelated to your service a spam or malware site a site that encourages people to violate twitter policy twitter name, logo, and likeness you may only use and display the twitter name and logo to identify twitter as the source of twitter content. you should never use the twitter name and logo, the twitter official partner program badge, or any other similar marks or names in a manner that creates a false sense of endorsement, sponsorship, or association with twitter. the twitter brand resources contain detailed information to help you use the twitter brand in the right way. you may only use the twitter verified account badge and any other enhanced account categorization as it is reported to you by twitter through the api. this helps people know that the content your service displays is equivalent to that shown on twitter. advertising on twitter there are restrictions regarding how and where you are allowed to advertise around twitter content. to start, your advertisements can’t resemble or reasonably be confused by people as a tweet or periscope broadcast. other rules on advertising include: there must be a clear separation between twitter content and your advertisements. you may not place any advertisements within the twitter timeline or on or within periscope broadcasts on your service other than twitter ads or advertisements made available through the official twitter kit integration with mopub. twitter reserves the right to serve advertising via the twitter api. if you decide to serve twitter ads once we start delivering them via the api, we will share a portion of advertising revenue with you in accordance with the relevant terms and conditions. you may not use twitter content, or information obtained from the twitter api to target people with advertising outside of the twitter platform. you must contact us if you find that your service will require more than million tokens. services that require more than million tokens may be subject to additional terms regarding twitter api access. the following additional rules apply for any use of the twitter services or features listed below: twitter login you must present people with easy to find options to log into and out of twitter, for example via the oauth protocol. the sign in with twitter option must be displayed at least as prominently as any other sign-up or sign-in feature on your service. you must also provide people without a twitter account the opportunity to create one via twitter. once someone on your service authenticates via sign in with twitter you must clearly display their twitter identity. twitter identity includes the person’s current twitter @handle, avatar, and twitter logo. any display of someone’s twitter followers on your service must clearly show that the relationship is associated with twitter. twitter cards to ensure a quality experience you must develop your card to render across all platforms where cards are displayed. additional rules that you must follow when using cards include: you must mark your tweet as ‘true’ for sensitive media if you plan to display such media within a card you must use https for hosting all assets within your card. your card should never generate active mixed content browser warnings audio and video content should include stop or pause controls, and default to ‘sound off’ for videos that automatically play content you may not exceed or circumvent twitter’s limitations placed on any cards, including the card’s intended use. additional restrictions on cards use include: you may not place third-party sponsored content within cards without twitter’s approval you may not attach monetary incentives (including virtual currency) within your card or on twitter from your card you may not include content or actions within your card that are misleading or not contextually relevant, such as urls and media. you may only attach an app card to a tweet when someone is explicitly promoting or referring to the app in the tweet periscope producer you must contact us about your twitter api access if you expect your service to exceed million daily broadcasts. you may be subject to additional terms if you exceed this threshold. additional restrictions on periscope developer use include: you must provide a reasonable user-agent, as described in the periscope producer technical documentation, for your service when accessing the periscope api you must honor requests from people to log out of their periscope account on your service you may not provide tools in your service to allow people to circumvent technological protection measures developer agreement, policy & terms follow @twitterdev subscribe to developer news twitter platform twitter.com status card validator privacy center transparency center twitter, inc. about the company twitter for good company news brand toolkit jobs and internships investors help help center using twitter twitter media ads help center managing your account safety and security rules and policies contact us developer resources developer home documentation forums communities developer blog engineering blog developer terms business resources advertise twitter for business resources and guides twitter for marketers marketing insights brand inspiration twitter data twitter flight school © twitter, inc. cookies privacy terms and conditions language developer japanese french chinese (simplified) by using twitter’s services you agree to our cookies use. we use cookies for purposes including analytics, personalisation, and ads. ok this page and certain other twitter sites place and read third party cookies on your browser that are used for non-essential purposes including targeting of ads. through these cookies, google, linkedin and demandbase collect personal data about you for their own purposes. learn more. accept decline introduction to github actions - github docs github docs all products github actions quickstart for github actions guides about continuous integration setting up ci using templates building and testing node.js building and testing .net building and testing powershell building and testing python building and testing ruby building and testing java with maven building and testing java with gradle building and testing java with ant about packaging with github actions publishing node.js packages publishing java packages with maven publishing java packages with gradle publishing docker images storing workflow artifacts caching dependencies about service containers redis service containers postgresql service containers deploying to amazon elastic container service deploying to azure app service deploying to google kubernetes engine learn github actions introduction to github actions finding and customizing actions essential features managing complex workflows sharing workflows with your organization security hardening migrating from azure pipelines to github actions migrating from circleci to github actions migrating from gitlab ci/cd to github actions migrating from jenkins to github actions migrating from travis ci to github actions managing workflow runs creating actions hosting your own runners reference github docs explore by product github actions github.com enterprise administrators github discussions github actions github packages developers rest api graphql api github insights education github desktop github cli atom electron english english 简体中文 (simplified chinese) 日本語 (japanese) español (spanish) português do brasil (portuguese) article version: github.com github.com enterprise server . enterprise server . see all enterprise releases github actions learn github actions introduction to github actions article version: github.com github.com enterprise server . enterprise server . see all enterprise releases introduction to github actions learn about the core concepts and various components of github actions, and see an example that shows you how to add automation to your repository. in this article overview the components of github actions create an example workflow understanding the workflow file viewing the job's activity next steps contacting support overview github actions help you automate tasks within your software development life cycle. github actions are event-driven, meaning that you can run a series of commands after a specified event has occurred. for example, every time someone creates a pull request for a repository, you can automatically run a command that executes a software testing script. this diagram demonstrates how you can use github actions to automatically run your software testing scripts. an event automatically triggers the workflow, which contains a job. the job then uses steps to control the order in which actions are run. these actions are the commands that automate your software testing. the components of github actions below is a list of the multiple github actions components that work together to run jobs. you can see how these components interact with each other. workflows the workflow is an automated procedure that you add to your repository. workflows are made up of one or more jobs and can be scheduled or triggered by an event. the workflow can be used to build, test, package, release, or deploy a project on github. events an event is a specific activity that triggers a workflow. for example, activity can originate from github when someone pushes a commit to a repository or when an issue or pull request is created. you can also use the repository dispatch webhook to trigger a workflow when an external event occurs. for a complete list of events that can be used to trigger workflows, see events that trigger workflows. jobs a job is a set of steps that execute on the same runner. by default, a workflow with multiple jobs will run those jobs in parallel. you can also configure a workflow to run jobs sequentially. for example, a workflow can have two sequential jobs that build and test code, where the test job is dependent on the status of the build job. if the build job fails, the test job will not run. steps a step is an individual task that can run commands in a job. a step can be either an action or a shell command. each step in a job executes on the same runner, allowing the actions in that job to share data with each other. actions actions are standalone commands that are combined into steps to create a job. actions are the smallest portable building block of a workflow. you can create your own actions, or use actions created by the github community. to use an action in a workflow, you must include it as a step. runners a runner is a server that has the github actions runner application installed. you can use a runner hosted by github, or you can host your own. a runner listens for available jobs, runs one job at a time, and reports the progress, logs, and results back to github. for github-hosted runners, each job in a workflow runs in a fresh virtual environment. github-hosted runners are based on ubuntu linux, microsoft windows, and macos. for information on github-hosted runners, see "virtual environments for github-hosted runners." if you need a different operating system or require a specific hardware configuration, you can host your own runners. for information on self-hosted runners, see "hosting your own runners." create an example workflow github actions uses yaml syntax to define the events, jobs, and steps. these yaml files are stored in your code repository, in a directory called .github/workflows. you can create an example workflow in your repository that automatically triggers a series of commands whenever code is pushed. in this workflow, github actions checks out the pushed code, installs the software dependencies, and runs bats -v. in your repository, create the .github/workflows/ directory to store your workflow files. in the .github/workflows/ directory, create a new file called learn-github-actions.yml and add the following code.name: learn-github-actions on: [push] jobs: check-bats-version: runs-on: ubuntu-latest steps: - uses: actions/checkout@v - uses: actions/setup-node@v - run: npm install -g bats - run: bats -v commit these changes and push them to your github repository. your new github actions workflow file is now installed in your repository and will run automatically each time someone pushes a change to the repository. for details about a job's execution history, see "viewing the workflow's activity." understanding the workflow file to help you understand how yaml syntax is used to create a workflow file, this section explains each line of the introduction's example: name: learn-github-actions optional - the name of the workflow as it will appear in the actions tab of the github repository. on: [push] specify the event that automatically triggers the workflow file. this example uses the push event, so that the jobs run every time someone pushes a change to the repository. you can set up the workflow to only run on certain branches, paths, or tags. for syntax examples including or excluding branches, paths, or tags, see "workflow syntax for github actions." jobs: groups together all the jobs that run in the learn-github-actions workflow file. check-bats-version: defines the name of the check-bats-version job stored within the jobs section. runs-on: ubuntu-latest configures the job to run on an ubuntu linux runner. this means that the job will execute on a fresh virtual machine hosted by github. for syntax examples using other runners, see "workflow syntax for github actions." steps: groups together all the steps that run in the check-bats-version job. each item nested under this section is a separate action or shell command. - uses: actions/checkout@v the uses keyword tells the job to retrieve v of the community action named actions/checkout@v . this is an action that checks out your repository and downloads it to the runner, allowing you to run actions against your code (such as testing tools). you must use the checkout action any time your workflow will run against the repository's code or you are using an action defined in the repository. - uses: actions/setup-node@v this action installs the node software package on the runner, giving you access to the npm command. - run: npm install -g bats the run keyword tells the job to execute a command on the runner. in this case, you are using npm to install the bats software testing package. - run: bats -v finally, you'll run the bats command with a parameter that outputs the software version. visualizing the workflow file in this diagram, you can see the workflow file you just created and how the github actions components are organized in a hierarchy. each step executes a single action or shell command. steps and use prebuilt community actions. steps and run shell commands directly on the runner. to find more prebuilt actions for your workflows, see "finding and customizing actions." viewing the job's activity once your job has started running, you can see a visualization graph of the run's progress and view each step's activity on github. on github, navigate to the main page of the repository. under your repository name, click actions. in the left sidebar, click the workflow you want to see. under "workflow runs", click the name of the run you want to see. under jobs or in the visualization graph, click the job you want to see. view the results of each step. next steps to continue learning about github actions, see "finding and customizing actions." contacting support if you need help with anything related to workflow configuration, such as syntax, github-hosted runners, or building actions, look for an existing topic or start a new one in the github community support's github actions category. if you have feedback or feature requests for github actions, share those in the feedback form for github actions. contact github support or github premium support for any of the following, whether your use or intended use falls into the usage limit categories: if you believe your account has been incorrectly restricted if you encounter an unexpected error when executing one of your actions, for example: a unique id if you encounter a situation where existing behavior contradicts expected, but not always documented, behavior did this doc help you? privacy policy want to learn about new docs features and updates? sign up for updates! we're continually improving our docs. we'd love to hear how we can do better. what problem did you have? required choose an option information was unclear the content was confusing the article didn't answer my question other let us know what we can do better optional can we contact you if we have more questions? optional send thank you! your feedback has been submitted. help us make these docs great! all github docs are open source. see something that's wrong or unclear? submit a pull request. make a contribution or, learn how to contribute. still need help? ask the github community contact support © github, inc. terms privacy security status help contact github pricing developer api training about csv,conf,v - may - - online conference csv,conf,v home how we planned for a virtual csv,conf schedule and speakers faq watch talks code of conduct a community conference for data makers everywhere featuring stories about data sharing and data analysis from science, journalism, government, and open source. may - , , online. thank you for making csv,conf,v possible! look out for csv,conf,v in ! building community csv,conf brings diverse groups together to discuss data topics, and features stories about data sharing and data analysis from science, journalism, government, and open source. people who love data csv,conf is a non-profit community conference run by folks who really love data and sharing knowledge. if you are as passionate about data and its application to society as we are, then this is the conference for you. big and small csv,conf conferences aren't just about spreadsheets. we curate content on broader topics like advancing the art of data collaboration- from putting your data on github, to producing meaningful insight by running large scale distributed processing on a cluster. keynote speakers emily jacobi emily is the executive director and founder of digital democracy and is passionate about leveraging technology to empower marginalized communities. beginning her career as a youth journalist at the age of , she has led technology, media and research projects in latin america, west africa, southeast asia and the us. prior to founding digital democracy, she worked for internews network, allafrica.com and as assistant bureau director for y-press. emily has presented on the intersection of technology, civic engagement and human rights to us congress, the state department, the united nations, and numerous universities and technology conferences. devoted to protecting the environment and democratizing the design process, at digital democracy emily works with the staff and board, oversees strategic planning and development and works directly with grassroots partners to design programs and tools that empower marginalized communities to defend their rights. emily is based in oakland, ca, usa. watch emily jacobi's keynote sisi wei sisi wei is the director of programs at opennews, where she envisions and executes transformative initiatives for journalism, especially for journalists of color and local journalists. previously, she was assistant managing editor at propublica where she oversaw and edited news apps, graphics, visual reporting and investigations. watch sisi's keynote · sisi's slides csv,conf is a group effort to put on a non-profit, community conference with dozens of people putting in time, including but not limited to the organizing team: john chodacki, california digital library martin fenner, datacite elaine wong, canadian broadcasting corporation jessica hardwicke, code for science & society jonathan cain, university of oregon jo barratt, open knowledge foundation danielle robinson, code for science & society lilly winfree, open knowledge foundation paul walsh, datopian serah rono, the carpentries we strive to be a supportive and welcoming environment to all attendees. we encourage you to read the conf code of conduct and will be enforcing it. you can revisit csv,conf,v , csv,conf,v , csv,conf,v and csv,conf,v conference websites, or watch csv,conf talks on youtube. the code for this website is open source. view it on github. want to talk to us? join us on slack. filereader - web apis | mdn skip to main content skip to search technologies technologies overview html css javascript graphics http apis browser extensions mathml references & guides learn web development tutorials references developer guides accessibility game development ...more docs feedback send feedback contribute to mdn report a content issue 🌐 report a platform issue 🌐 search mdn filereader web technology for developers web apis filereader select your preferred language english (us) deutsch español français 日本語 한국어 português (do brasil) português (europeu) Русский svenska türkçe 中文 (简体) 正體中文 (繁體) change language jump to section jump to section constructor properties methods events specifications browser compatibility see also the filereader object lets web applications asynchronously read the contents of files (or raw data buffers) stored on the user's computer, using file or blob objects to specify the file or data to read. file objects may be obtained from a filelist object returned as a result of a user selecting files using the element, from a drag and drop operation's datatransfer object, or from the mozgetasfile() api on an htmlcanvaselement. important note: filereader is used to read file content from the user's (remote) system in secure ways only. it cannot be used to read a file by pathname from a file system. to read files by pathname in javascript, standard ajax solutions should be used to do server-side file reading, with cors permission if reading cross-domain. note: this feature is available in web workers. constructor filereader() returns a newly constructed filereader. see using files from web applications for details and examples. properties filereader.error read only a domexception representing the error that occurred while reading the file. filereader.readystate read only a number indicating the state of the filereader. this is one of the following: empty no data has been loaded yet. loading data is currently being loaded. done the entire read request has been completed. filereader.result read only the file's contents. this property is only valid after the read operation is complete, and the format of the data depends on which of the methods was used to initiate the read operation. event handlers filereader.onabort a handler for the abort event. this event is triggered each time the reading operation is aborted. filereader.onerror a handler for the error event. this event is triggered each time the reading operation encounter an error. filereader.onload a handler for the load event. this event is triggered each time the reading operation is successfully completed. filereader.onloadstart a handler for the loadstart event. this event is triggered each time the reading is starting. filereader.onloadend a handler for the loadend event. this event is triggered each time the reading operation is completed (either in success or failure). filereader.onprogress a handler for the progress event. this event is triggered while reading a blob content. as filereader inherits from eventtarget, all those events can also be listened for by using the addeventlistener method. methods filereader.abort() aborts the read operation. upon return, the readystate will be done. filereader.readasarraybuffer() starts reading the contents of the specified blob, once finished, the result attribute contains an arraybuffer representing the file's data. filereader.readasbinarystring() starts reading the contents of the specified blob, once finished, the result attribute contains the raw binary data from the file as a string. filereader.readasdataurl() starts reading the contents of the specified blob, once finished, the result attribute contains a data: url representing the file's data. filereader.readastext() starts reading the contents of the specified blob, once finished, the result attribute contains the contents of the file as a text string. an optional encoding name can be specified. events listen to these events using addeventlistener() or by assigning an event listener to the oneventname property of this interface. abort fired when a read has been aborted, for example because the program called filereader.abort(). also available via the onabort property. error fired when the read failed due to an error. also available via the onerror property. load fired when a read has completed successfully. also available via the onload property. loadend fired when a read has completed, successfully or not. also available via the onloadend property. loadstart fired when a read has started. also available via the onloadstart property. progress fired periodically as data is read. also available via the onprogress property. specifications specification status comment file api the definition of 'filereader' in that specification. working draft initial definition browser compatibility bcd tables only load in the browser see also using files from web applications file blob filereadersync related topics filereader constructor filereader() properties error onabort onload readystate result methods abort() readasarraybuffer() readasbinarystring() readasdataurl() readastext() events abort error load loadend loadstart progress inheritance: eventtarget related pages for file api blob file filelist filereadersync found a problem with this page? source on github report a problem with this content on github want to fix the problem yourself? see our contribution guide. last modified: dec , , by mdn contributors web technologies learn web development about mdn feedback about mdn web docs store contact us firefox mdn mozilla © - mozilla and individual contributors. content is available under these licenses. terms privacy cookies ajax (programming) - wikipedia ajax (programming) from wikipedia, the free encyclopedia jump to navigation jump to search "ajax" redirects here. for other uses, see ajax. group of interrelated web development techniques asynchronous javascript and xml first appeared march filename extensions .js file formats javascript influenced by javascript and xml ajax (also ajax /ˈeɪdʒæks/; short for "asynchronous javascript and xml")[ ][ ] is a set of web development techniques using many web technologies on the client-side to create asynchronous web applications. with ajax, web applications can send and retrieve data from a server asynchronously (in the background) without interfering with the display and behaviour of the existing page. by decoupling the data interchange layer from the presentation layer, ajax allows web pages and, by extension, web applications, to change content dynamically without the need to reload the entire page.[ ] in practice, modern implementations commonly utilize json instead of xml. ajax is not a single technology, but rather a group of technologies. html and css can be used in combination to mark up and style information. the webpage can then be modified by javascript to dynamically display—and allow the user to interact with—the new information. the built-in xmlhttprequest object, or since the new "fetch()" function within javascript, is commonly used to execute ajax on webpages, allowing websites to load content onto the screen without refreshing the page. ajax is not a new technology, or a different language, just existing technologies used in new ways. contents history technologies drawbacks examples . javascript example . jquery example . fetch example see also references external links history[edit] in the early-to-mid s, most web sites were based on complete html pages. each user action required that a complete new page be loaded from the server. this process was inefficient, as reflected by the user experience: all page content disappeared, then the new page appeared. each time the browser reloaded a page because of a partial change, all of the content had to be re-sent, even though only some of the information had changed. this placed additional load on the server and made bandwidth a limiting factor on performance. in , the iframe tag was introduced by internet explorer; like the object element, it can load or fetch content asynchronously. in , the microsoft outlook web access team developed the concept behind the xmlhttprequest scripting object.[ ] it appeared as xmlhttp in the second version of the msxml library,[ ][ ] which shipped with internet explorer . in march .[ ] the functionality of the windows xmlhttp activex control in ie was later implemented by mozilla, safari, opera and other browsers as the xmlhttprequest javascript object.[ ] microsoft adopted the native xmlhttprequest model as of internet explorer . the activex version is still supported in internet explorer, but not in microsoft edge. the utility of these background http requests and asynchronous web technologies remained fairly obscure until it started appearing in large scale online applications such as outlook web access ( )[ ] and oddpost ( ). google made a wide deployment of standards-compliant, cross browser ajax with gmail ( ) and google maps ( ).[ ] in october kayak.com's public beta release was among the first large-scale e-commerce uses of what their developers at that time called "the xml http thing".[ ] this increased interest in ajax among web program developers. the term ajax was publicly used on february by jesse james garrett in an article titled ajax: a new approach to web applications, based on techniques used on google pages.[ ] on april , the world wide web consortium (w c) released the first draft specification for the xmlhttprequest object in an attempt to create an official web standard.[ ] the latest draft of the xmlhttprequest object was published on october ,[ ] and the xmlhttprequest specification is now a living standard.[ ] technologies[edit] the conventional model for a web application versus an application using ajax the term ajax has come to represent a broad group of web technologies that can be used to implement a web application that communicates with a server in the background, without interfering with the current state of the page. in the article that coined the term ajax,[ ][ ] jesse james garrett explained that the following technologies are incorporated: html (or xhtml) and css for presentation the document object model (dom) for dynamic display of and interaction with data json or xml for the interchange of data, and xslt for xml manipulation the xmlhttprequest object for asynchronous communication javascript to bring these technologies together since then, however, there have been a number of developments in the technologies used in an ajax application, and in the definition of the term ajax itself. xml is no longer required for data interchange and, therefore, xslt is no longer required for the manipulation of data. javascript object notation (json) is often used as an alternative format for data interchange,[ ] although other formats such as preformatted html or plain text can also be used.[ ] a variety of popular javascript libraries, including jquery, include abstractions to assist in executing ajax requests. drawbacks[edit] any user whose browser does not support javascript or xmlhttprequest, or has this functionality disabled, will not be able to properly use pages that depend on ajax. the only way to let the user carry out functionality is to fall back to non-javascript methods. this can be achieved by making sure links and forms can be resolved properly and not relying solely on ajax.[ ] similarly, some web applications that use ajax are built in a way that cannot be read by screen-reading technologies, such as jaws. the wai-aria standards provide a way to provide hints in such a case.[ ] screen readers that are able to use ajax may still not be able to properly read the dynamically generated content.[ ] the same-origin policy prevents some ajax techniques from being used across domains,[ ] although the w c has a draft of the xmlhttprequest object that would enable this functionality.[ ] methods exist to sidestep this security feature by using a special cross domain communications channel embedded as an iframe within a page,[ ] or by the use of jsonp. ajax is designed for one-way communications with the server. if two way communications are needed (i.e. for the client to listen for events/changes on the server), then websockets may be a better option.[ ] in pre-html browsers, pages dynamically created using successive ajax requests did not automatically register themselves with the browser's history engine, so clicking the browser's "back" button may not have returned the browser to an earlier state of the ajax-enabled page, but may have instead returned to the last full page visited before it. such behavior — navigating between pages instead of navigating between page states — may be desirable, but if fine-grained tracking of page state is required, then a pre-html workaround was to use invisible iframes to trigger changes in the browser's history. a workaround implemented by ajax techniques is to change the url fragment identifier (the part of a url after the "#") when an ajax-enabled page is accessed and monitor it for changes.[ ][ ] html provides an extensive api standard for working with the browser's history engine.[ ] dynamic web page updates also make it difficult to bookmark and return to a particular state of the application. solutions to this problem exist, many of which again use the url fragment identifier.[ ][ ] on the other hand, as ajax-intensive pages tend to function as applications rather than content, bookmarking interim states rarely makes sense. nevertheless, the solution provided by html for the above problem also applies for this.[ ] depending on the nature of the ajax application, dynamic page updates may disrupt user interactions, particularly if the internet connection is slow or unreliable. for example, editing a search field may trigger a query to the server for search completions, but the user may not know that a search completion popup is forthcoming, and if the internet connection is slow, the popup list may show up at an inconvenient time, when the user has already proceeded to do something else. excluding google,[ ] most major web crawlers do not execute javascript code,[ ] so in order to be indexed by web search engines, a web application must provide an alternative means of accessing the content that would normally be retrieved with ajax. it has been suggested that a headless browser may be used to index content provided by ajax-enabled websites, although google is no longer recommending the ajax crawling proposal they made in .[ ] examples[edit] javascript example[edit] an example of a simple ajax request using the get method, written in javascript. get-ajax-data.js: // this is the client-side script. // initialize the http request. var xhr = new xmlhttprequest(); xhr.open('get', 'send-ajax-data.php'); // track the state changes of the request. xhr.onreadystatechange = function () { var done = ; // readystate means the request is done. var ok = ; // status is a successful return. if (xhr.readystate === done) { if (xhr.status === ok) { console.log(xhr.responsetext); // 'this is the output.' } else { console.log('error: ' + xhr.status); // an error occurred during the request. } } }; // send the request to send-ajax-data.php xhr.send(null); send-ajax-data.php: many developers dislike the syntax used in the xmlhttprequest object, so some of the following workarounds have been created. jquery example[edit] the popular javascript library jquery has implemented abstractions which enable developers to use ajax more conveniently. although it still uses xmlhttprequest behind the scenes, the following is a client-side implementation of the same example as above using the 'ajax' method. $.ajax({ type: 'get', url: 'send-ajax-data.php', datatype: "json", // data type expected from server success: function (data) { console.log(data); }, error: function(error) { console.log('error: ' + error); } }); jquery also implements a 'get' method which allows the same code to be written more concisely. $.get('send-ajax-data.php').done(function(data) { console.log(data); }).fail(function(data) { console.log('error: ' + data); }); fetch example[edit] fetch is a new native javascript api.[ ] according to google developers documentation, "fetch makes it easier to make web requests and handle responses than with the older xmlhttprequest." fetch('send-ajax-data.php') .then(data => console.log(data)) .catch (error => console.log('error:' + error)); es async/await example: async function doajax() { try { const res = await fetch('send-ajax-data.php'); const data = await res.text(); console.log(data); } catch (error) { console.log('error:' + error); } } doajax(); as seen above, fetch relies on javascript promises. see also[edit] actionscript comet (programming) (also known as reverse ajax) google instant http/ list of ajax frameworks node.js remote scripting rich internet application websocket html javascript references[edit] ^ a b c jesse james garrett ( february ). "ajax: a new approach to web applications". adaptivepath.com. archived from the original on september . retrieved june . ^ "ajax - web developer guides". mdn web docs. archived from the original on february . retrieved february . ^ a b ullman, chris (march ). beginning ajax. wrox. isbn - - - - . archived from the original on july . retrieved june . ^ a b "article on the history of xmlhttp by an original developer". alexhopmann.com. january . archived from the original on june . retrieved july . ^ "specification of the ixmlhttprequest interface from the microsoft developer network". msdn.microsoft.com. archived from the original on may . retrieved july . ^ dutta, sunava ( january ). "native xmlhttprequest object". ieblog. microsoft. archived from the original on march . retrieved november . ^ "dynamic html and xml: the xmlhttprequest object". apple inc. archived from the original on may . retrieved june . ^ hopmann, alex. "story of xmlhttp". alex hopmann’s blog. archived from the original on march . retrieved may . ^ "a brief history of ajax". aaron swartz. december . archived from the original on june . retrieved august . ^ english, paul ( april ). "kayak user interface". official kayak.com technoblog. archived from the original on may . retrieved may . ^ a b van kesteren, anne; jackson, dean ( april ). "the xmlhttprequest object". w .org. world wide web consortium. archived from the original on may . retrieved june . ^ kesteren, anne; aubourg, julian; song, jungkee; steen, hallvord r. m. "xmlhttprequest level ". w .org. w c. archived from the original on july . retrieved february . ^ "xmlhttprequest standard". xhr.spec.whatwg.org. retrieved april . ^ "javascript object notation". apache.org. archived from the original on june . retrieved july . ^ "speed up your ajax-based apps with json". devx.com. archived from the original on july . retrieved july . ^ quinsey, peter. "user-proofing ajax". archived from the original on february . retrieved january . ^ "wai-aria overview". w c. archived from the original on october . retrieved october . ^ edwards, james ( may ). "ajax and screenreaders: when can it work?". sitepoint.com. archived from the original on march . retrieved june . ^ "access control for cross-site requests". world wide web consortium. archived from the original on may . retrieved june . ^ "secure cross-domain communication in the browser". the architecture journal (msdn). archived from the original on march . retrieved april . ^ pimentel, victoria; nickerson, bradford g. ( may ). "communicating and displaying real-time data with websocket". ieee internet computing. ( ): – . doi: . /mic. . . ^ a b "why use ajax?". interakt. november . archived from the original on may . retrieved june . ^ a b "deep linking for ajax". archived from the original on july . retrieved april . ^ a b "html specification". archived from the original on october . retrieved october . ^ hendriks, erik ( may ). "official news on crawling and indexing sites for the google index". google. archived from the original on may . retrieved may . ^ prokoph, andreas ( may ). "help web crawlers efficiently crawl your portal sites and web sites". ibm. archived from the original on february . retrieved april . ^ "deprecating our ajax crawling scheme". google webmaster central blog. october . archived from the original on october . retrieved october . ^ "fetch api - web apis". mdn. archived from the original on may . retrieved may . external links[edit] wikimedia commons has media related to ajax (programming). wikibooks has a book on the topic of: ajax ajax: a new approach to web applications — article that coined the ajax term and q&a ajax (programming) at curlie ajax tutorial with get, post, text and xml examples. v t e web interfaces server-side protocols http cgi scgi fcgi ajp wsrp websocket server apis c nsapi c asapi c isapi com asp jakarta servlet container cli owin asp.net handler python wsgi ruby rack javascript jsgi perl psgi lua wsapi portlet container apache modules mod_include mod_jk mod_lisp mod_mono mod_parrot mod_perl mod_php mod_proxy mod_python mod_wsgi mod_ruby phusion passenger topics web resource vs. web service open api webhook application server comparison scripting client-side browser apis c npapi liveconnect xpconnect c npruntime c ppapi nacl activex bho xbap web apis w c audio canvas cors dom dom events eme file geolocation indexeddb mse sse svg video webauthn webgpu webrtc websocket web messaging web storage web worker xmlhttprequest webassembly khronos webcl webgl others gears web sql database (formerly w c) webusb topics ajax and remote scripting vs. dhtml mashup web idl scripting topics dynamic web page web standards rich web application web api security web application web framework v t e javascript code analysis eslint jshint jslint supersets js++ objective-j typescript transpilers babel.js coffeescript livescript dart emscripten google closure compiler google web toolkit morfik atscript opa nim haxe clojurescript websharper purescript reason elm concepts client-side javascript library javascript syntax unobtrusive javascript debuggers firebug komodo ide microsoft script debugger microsoft script editor opera dragonfly web inspector doc generators jsdoc editors (comparison) ace cloud ide atom codemirror light table brackets phpstorm orion visual studio visual studio express visual studio code visual studio team services vim engines comparison of engines dom support list of ecmascript engines frameworks comparison of javascript frameworks list of javascript libraries related technologies cascading style sheets document object model html dynamic html ajax json webassembly asm.js package managers npm application bundlers webpack server-side active server pages commonjs deno jsgi node.js wakanda unit testing jasmine mocha qunit list of javascript unit testing frameworks jest people douglas crockford brendan eich john resig authority control bne: xx bnf: cb c (data) gnd: - lccn: sh sudoc: retrieved from "https://en.wikipedia.org/w/index.php?title=ajax_(programming)&oldid= " categories: ajax (programming) cloud standards inter-process communication web . neologisms web development hidden categories: articles with short description short description matches wikidata use dmy dates from july commons category link is on wikidata articles with curlie links wikipedia articles with bne identifiers wikipedia articles with bnf identifiers wikipedia articles with gnd identifiers wikipedia articles with lccn identifiers wikipedia articles with sudoc identifiers articles with example javascript code articles with example php code navigation menu personal tools not logged in talk contributions create account log in namespaces article talk variants views read edit view history more search navigation main page contents current events random article about wikipedia contact us donate contribute help learn to edit community portal recent changes upload file tools what links here related changes upload file special pages permanent link page information cite this page wikidata item print/export download as pdf printable version in other projects wikimedia commons wikibooks languages afrikaans العربية azərbaycanca বাংলা Български català Čeština dansk deutsch eesti Ελληνικά español euskara فارسی français gaeilge galego 한국어 Հայերեն हिन्दी bahasa indonesia italiano עברית ქართული Қазақша lietuvių magyar Македонски മലയാളം bahasa melayu Монгол nederlands 日本語 norsk bokmål norsk nynorsk piemontèis polski português română Русский shqip simple english slovenčina slovenščina Српски / srpski suomi svenska தமிழ் ไทย türkçe türkmençe Українська ئۇيغۇرچە / uyghurche tiếng việt 吴语中文 edit links this page was last edited on january , at : (utc). text is available under the creative commons attribution-sharealike license; additional terms may apply. by using this site, you agree to the terms of use and privacy policy. wikipedia® is a registered trademark of the wikimedia foundation, inc., a non-profit organization. privacy policy about wikipedia disclaimers contact wikipedia mobile view developers statistics cookie statement the digital disease in academic libraries | canadian journal of academic librarianship skip to main content skip to main navigation menu skip to site footer current archives announcements about about the journal submissions tips for authors guidelines for peer review book reviews editorial team privacy statement contact search search register login home / archives / vol ( ): including special focus on academic libraries and the irrational / special focus on academic libraries and the irrational the digital disease in academic libraries kris joseph york university doi: https://doi.org/ . /cjal-rcbu.v . keywords: digital scholarship, library management, library organization abstract this article uses organizational design and management literature to shed critical light on a peculiar quirk of academic library organizational structures: the existence of job titles and departments that exist to highlight “digital” functions and workflows. an exploration of the literature along four interrelated themes provides insight into the origin of the problem: organizational design theory and the arrangement of work in academic libraries; the reliance on strategic alignment through buzzwords as a means of coping with uncertainty; the tendency of academic library structures to resemble one another; and challenges associated with knowledge sharing and professional development in hierarchical organizations. these contexts frame the symptoms of the digital disease, all of which are derived from the convergence of the article’s four thematic preconditions. though the disease is the lens through which contemporary academic library organization is analyzed, its existence serves to highlight pre-existing patterns in academic library management that warrant further scrutiny. downloads download data is not yet available. author biography kris joseph, york university kris joseph is the digital scholarship librarian at york university. pdf published - - how to cite joseph, kris. . “the digital disease in academic libraries”. canadian journal of academic librarianship (december), - . https://doi.org/ . /cjal-rcbu.v . . more citation formats apa chicago harvard mla download citation endnote/zotero/mendeley (ris) bibtex issue vol ( ): including special focus on academic libraries and the irrational section special focus on academic libraries and the irrational copyright (c) kris joseph this work is licensed under a creative commons attribution-noncommercial . international license. authors who publish with this journal agree to the following terms: authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a creative commons attribution non-commercial license that allows others to use and share the work with an acknowledgement of the work's authorship and initial publication in this journal, as long as it is not used for commercial purposes. this license does not waive the author’s moral rights. authors may choose a different creative commons license by indicating their preference to the editors. language english français (canada) make a submission browse categories special focus on research and scholarship special focus on diversity information for readers for authors for librarians the canadian journal of academic librarianship is published by the canadian association of professional academic librarians. follow us: @cjalrcbu issn - x domain-validated certificate - wikipedia domain-validated certificate from wikipedia, the free encyclopedia jump to navigation jump to search "domain validation" redirects here. for the technique employed in parallel scsi, see parallel scsi § ultra- . a domain validated certificate (dv) is an x. digital certificate typically used for transport layer security (tls) where the domain name of the applicant is validated by proving some control over a dns domain. [ ] domain validated certificates were first distributed by geotrust in before becoming a widely accepted method. [ ] contents issuing criteria user interface characteristics references issuing criteria[edit] the sole criterion for a domain validated certificate is proof of control over whois records, dns records file, email or web hosting account of a domain. typically control over a domain is determined using one of the following: response to email sent to the email contact in the domain's whois details response to email sent to a well-known administrative contact in the domain, e.g. (admin@, postmaster@, etc.) publishing a dns txt record publishing a nonce provided by an automated certificate issuing system a domain validated certificate is distinct from an extended validation certificate in that this is the only requirement for issuing the certificate. in particular, domain validated certificates do not assure that any particular legal entity is connected to the certificate, even if the domain name may imply a particular legal entity controls the domain. user interface[edit] most web browsers may show a lock (often in grey, rather than the green lock typically used for an extended validation certificate) and a dns domain name. a legal entity is never displayed, as domain validated certificates do not include a legal entity in their subject.[ ] mozilla firefox historically showed domain validated certificates with a grey lock,[ ] but this was modified to show a green lock for domain-validated connections after mozilla launched let's encrypt (which only provides domain validated certificates). safari shows domain validated certificates with a grey lock. microsoft edge displays domain validated certificates with a hollow grey lock. chrome and chromium display a green lock.[ ] characteristics[edit] as the low assurance requirements allow domain validated certificates to be issued quickly without requiring human intervention, domain validated certificates have a number of unique characteristics: domain validated certificates are used in automated x. certificate issuing systems, such as let's encrypt. domain validated certificates are often cheap or free. domain validated certificates can be generated and validated without any documentation. most domain validated certificates can be issued instantly. references[edit] ^ coclin, dean ( - - ). "what are the different types of ssl certificates?". certificate authority security council. retrieved - - . ^ "there's certs and certs – verisign badmouths rivals". www.theregister.com. ^ "ssl explained simply - what's the best free option?". hostingcanada.org. ^ vyas, tanvi. "updated firefox security indicators". mozilla security blog. ^ "check if a site's connection is secure". support.google.com. v t e tls and ssl protocols and technologies transport layer security / secure sockets layer (tls/ssl) datagram transport layer security (dtls) server name indication (sni) application-layer protocol negotiation (alpn) dns-based authentication of named entities (dane) dns certification authority authorization (caa) https http strict transport security (hsts) http public key pinning (hpkp) ocsp stapling opportunistic tls perfect forward secrecy public-key infrastructure automated certificate management environment (acme) certificate authority (ca) ca/browser forum certificate policy certificate revocation list (crl) domain-validated certificate (dv) extended validation certificate (ev) online certificate status protocol (ocsp) public key certificate public-key cryptography public key infrastructure (pki) root certificate self-signed certificate see also domain name system security extensions (dnssec) internet protocol security (ipsec) secure shell (ssh) history export of cryptography from the united states server-gated cryptography implementations bouncy castle boringssl botan bsafe cryptlib gnutls jsse libressl matrixssl mbed tls nss openssl s n schannel ssleay stunnel wolfssl notaries certificate transparency convergence https everywhere perspectives project vulnerabilities theory man-in-the-middle attack padding oracle attack cipher bar mitzvah attack protocol beast breach crime drown logjam poodle (in regards to ssl . ) implementation certificate authority compromise random number generator attacks freak goto fail heartbleed lucky thirteen attack poodle (in regards to tls . ) kazakhstan mitm attack retrieved from "https://en.wikipedia.org/w/index.php?title=domain-validated_certificate&oldid= " categories: key management public key infrastructure transport layer security navigation menu personal tools not logged in talk contributions create account log in namespaces article talk variants views read edit view history more search navigation main page contents current events random article about wikipedia contact us donate contribute help learn to edit community portal recent changes upload file tools what links here related changes upload file special pages permanent link page information cite this page wikidata item print/export download as pdf printable version languages 中文 edit links this page was last edited on january , at : (utc). text is available under the creative commons attribution-sharealike license; additional terms may apply. by using this site, you agree to the terms of use and privacy policy. wikipedia® is a registered trademark of the wikimedia foundation, inc., a non-profit organization. privacy policy about wikipedia disclaimers contact wikipedia mobile view developers statistics cookie statement gene (@dzshuniper@ausglam.space) - aus glam space profile directory about mobile apps log in sign up gene @dzshuniper@ausglam.space toots following followers follow pronouns he/him toot retention days username:eng juniper username:yid דזשׁוּניפער data curator. speculative fiction fan. reads stories by women/non-binary people and writers of colour. crochets own kippot. avatar by h-p lehkonen: https://hplehkonen.com/ joined mar following followers toots toots and replies media feb , , : gene @dzshuniper@ausglam.space sportsball 🏉 show more watching the aflw's pride round. loving the lesbian and bi women's representation. 🏳️‍🌈🧡 and the western bulldogs' pride guernsey prominently featured the trans flag, continuing the club's advocacy for trans women's inclusion in the league. 🏳️‍⚧️ feb , , : gene @dzshuniper@ausglam.space white libraries show more just had a meeting with library colleagues and found myself looking at six white faces wearing glasses on zoom, including mine. sheesh, we all look the same. 🧑🏻‍🏫 feb , , : gene @dzshuniper@ausglam.space colleague a: mentions a birds-of-a-feather meeting that recently took place. colleague b: counters by suggesting we hold a snakes-of-a-scale meeting. colleague a: suggests perhaps a hair-of-the-dog meeting? colleague b: is outraged and insists academic rigour must be maintained - it is a dogs-of-a-hair meeting! feb , , : gene @dzshuniper@ausglam.space looking into data management systems and have veered from research systems into enterprise systems. all the websites have sections for both products and solutions. what is the difference between the two? who knows? i mean, one of the solutions on offer is financial crime and compliance? is financial crime a solution now? to what, i wonder? i am very confused and a bit dubious about whatever it is these companies are trying to sell me... gene boosted feb , , : eric @ericireland@aus.social eel basket https://museumsandcollections.unimelb.edu.au/news/items/eel-basket-for-melbourne-connect feb , , : gene @dzshuniper@ausglam.space show more my partner on his escapades from last night: "i was hanging out on black twitter and then somehow accidentally found myself on archivist twitter. i had no idea what was going on. i think i offended a bunch of people. now i have archivist twitter hangover and am following michelle caswell." 🤣 feb , , : gene @dzshuniper@ausglam.space yay! glad to hear that the asa have decided not to renew their contract with taylor and francis, in favour of moving the archives and manuscripts journal to an open access publishing model. feb , , : gene @dzshuniper@ausglam.space more from sen: "academic standards of rigour are constructed to exclude and disallow certain sectors of society to participate with full voice in their own stories or transmission of their narratives. we are not passive recipients of our own history. our sense-making of our stories doesn't belong to scholars, whose privileged access to academia push us out of our own picture. i wonder how many academics have been under the objectification they rate so highly?" show thread feb , , : gene @dzshuniper@ausglam.space colonial theft show more attending a panel discussion that my lecturer put together to fill in the gaps she saw in the content in my class on collections. one of the speakers is journalist marc fennell, discussing his show stuff the british stole: https://www.abc.net.au/radionational/programs/stuff-the-british-stole/ feb , , : gene @dzshuniper@ausglam.space mh🔥 show more reading 'more voice, less ventriloquism- exploring the relational dynamics in a participatory archive of mental health recovery' by anna sexton & dolly sen for critical archives reading group. sen says "[m]ost of the archives that depict mental health lived experience are filtered by mental health professionals, but that is like lions representing bird song in roars. why should the hunters give the hunted's history? [...] how can we tell our true stories when our words are seen as sickness?" jan , , : gene @dzshuniper@ausglam.space writing a libguide on open research, including open code. mistyped "softer" instead of "software". not really sure if i want to correct myself because, really, don't we need more softness and openness in research? jan , , : gene @dzshuniper@ausglam.space auspol, neo-nazis show more well, this is scary, despite being predictable: https://www.smh.com.au/politics/federal/we-do-not-need-to-wait-for-a-christchurch-grampians-cross-burning-spurs-call-for-action- -p xer.html and of course: "this group and others are creating an echo chamber and incubator on the net, taking full advantage of their virtual audience to feed and amplify their vitriolic fantasies" the abc's take also emphasises how this affects indigenous australians: https://www.abc.net.au/news/ - - /anger-over-neo-nazis-chanting-camping-in-the-grampians/ jan , , : gene @dzshuniper@ausglam.space so much love to the feminist library. their statement on transphobia and accountability is how you do allyship and feminism. 💙💗🏳️‍⚧️ http://feministlibrary.co.uk/statement-on-transphobia-and-accountability/ jan , , : gene @dzshuniper@ausglam.space happy new year for trees 🌳🌴🌲🍎🍊🍐🍑🍒🥝 gene boosted jan , , : ksclarke @ksclarke@code lib.social #uspol show more i've been wondering what sort of library a president who doesn't read would have... i think this answers the question: https://djtrumplibrary.com jan , , : gene @dzshuniper@ausglam.space the artist is stephen millership: http://www.stephenmillership.com/ show thread jan , , : gene @dzshuniper@ausglam.space i also bought a print of the cornerhouse art-house cinema, which was my home from home during my teenage years. this was the cinema responsible for introducing me to the princess bride, fred astaire and ginger rogers, muriel's wedding, ken loach, alain delon, thelma and louise, and countless others. don't think i would have made it through school without it. d b b .jpg show thread jan , , : gene @dzshuniper@ausglam.space the library art i ordered has arrived. it's a print of manchester central library, drawn in travel poster style. this is the library that was responsible for introducing me to octavia butler. ac df ed e .jpg jan , , : gene @dzshuniper@ausglam.space a video of the talk is here: https://doi.org/ . / show thread jan , , : gene @dzshuniper@ausglam.space notes from a talk about exploring the state library of nsw social media archive to see what insights could be gleaned from the creative and cultural sectors for teaching in higher education, and discovering an interesting sentiment analysis of the #auslibchat hashtag from the early months of the covid lockdown: https://doi.org/ . / .v show older pronouns he/him toot retention days username:eng juniper username:yid דזשׁוּניפער data curator. speculative fiction fan. reads stories by women/non-binary people and writers of colour. crochets own kippot. avatar by h-p lehkonen: https://hplehkonen.com/ joined mar this is a mastodon instance primarily for australasian galleries, libraries, archives, museums and records people, and anyone else who wants to hang out with them. resources terms of service privacy policy developers documentation api what is mastodon? ausglam.space about v . . more… source code mobile apps deploying a rails application to elastic beanstalk - aws elastic beanstalk awsdocumentationaws elastic beanstalkdeveloper guide prerequisiteslaunch an elastic beanstalk environmentinstall rails and generate a websiteconfigure rails settingsdeploy your applicationcleanupnext steps deploying a rails application to elastic beanstalk rails is an open source, model-view-controller (mvc) framework for ruby. this tutorial walks you through the process of generating a rails application and deploying it to an aws elastic beanstalk environment. sections prerequisites launch an elastic beanstalk environment install rails and generate a website configure rails settings deploy your application cleanup next steps prerequisites this tutorial assumes you have knowledge of the basic elastic beanstalk operations and the elastic beanstalk console. if you haven't already, follow the instructions in getting started using elastic beanstalk to launch your first elastic beanstalk environment. to follow the procedures in this guide, you will need a command line terminal or shell to run commands. commands are shown in listings preceded by a prompt symbol ($) and the name of the current directory, when appropriate. ~/eb-project$ this is a command this is outputon linux and macos, use your preferred shell and package manager. on windows , you can install the windows subsystem for linux to get a windows-integrated version of ubuntu and bash. the rails framework has the following dependencies. be sure you have all of them installed. ruby . . or newer – for installation instructions, see setting up your ruby development environment. in this tutorial we use ruby . . and the corresponding elastic beanstalk platform version. node.js – for installation instructions, see installing node.js via package manager. yarn – for installation instructions, see installation on the yarn website. launch an elastic beanstalk environment use the elastic beanstalk console to create an elastic beanstalk environment. choose the ruby platform and accept the default settings and sample code. to launch an environment (console) open the elastic beanstalk console using this preconfigured link: console.aws.amazon.com/elasticbeanstalk/home#/newapplication?applicationname=tutorials&environmenttype=loadbalanced for platform, select the platform and platform branch that match the language used by your application. for application code, choose sample application. choose review and launch. review the available options. choose the available option you want to use, and when you're ready, choose create app. environment creation takes about minutes and creates the following resources: ec instance – an amazon elastic compute cloud (amazon ec ) virtual machine configured to run web apps on the platform that you choose. each platform runs a specific set of software, configuration files, and scripts to support a specific language version, framework, web container, or combination of these. most platforms use either apache or nginx as a reverse proxy that sits in front of your web app, forwards requests to it, serves static assets, and generates access and error logs. instance security group – an amazon ec security group configured to allow inbound traffic on port . this resource lets http traffic from the load balancer reach the ec instance running your web app. by default, traffic isn't allowed on other ports. load balancer – an elastic load balancing load balancer configured to distribute requests to the instances running your application. a load balancer also eliminates the need to expose your instances directly to the internet. load balancer security group – an amazon ec security group configured to allow inbound traffic on port . this resource lets http traffic from the internet reach the load balancer. by default, traffic isn't allowed on other ports. auto scaling group – an auto scaling group configured to replace an instance if it is terminated or becomes unavailable. amazon s bucket – a storage location for your source code, logs, and other artifacts that are created when you use elastic beanstalk. amazon cloudwatch alarms – two cloudwatch alarms that monitor the load on the instances in your environment and that are triggered if the load is too high or too low. when an alarm is triggered, your auto scaling group scales up or down in response. aws cloudformation stack – elastic beanstalk uses aws cloudformation to launch the resources in your environment and propagate configuration changes. the resources are defined in a template that you can view in the aws cloudformation console. domain name – a domain name that routes to your web app in the form subdomain.region.elasticbeanstalk.com. all of these resources are managed by elastic beanstalk. when you terminate your environment, elastic beanstalk terminates all the resources that it contains. note the amazon s bucket that elastic beanstalk creates is shared between environments and is not deleted during environment termination. for more information, see using elastic beanstalk with amazon s . install rails and generate a website install rails and its dependencies with the gem command. ~$ gem install rails fetching: concurrent-ruby- . . .gem ( %) successfully installed concurrent-ruby- . . fetching: i n- . . .gem ( %) successfully installed i n- . . ... test your rails installation. ~$ rails --version rails . . use rails new with the name of the application to create a new rails project. ~$ rails new ~/eb-rails rails creates a directory with the name specified, generates all of the files needed to run a sample project locally, and then runs bundler to install all of the dependencies (gems) defined in the project's gemfile. test your rails installation by running the default project locally. ~$ cd eb-rails eb-rails $ rails server => booting puma => rails . . application starting in development => run `rails server -h` for more startup options puma starting in single mode... * version . . (ruby . . -p ), codename: love song * min threads: , max threads: * environment: development * listening on tcp:// . . . : use ctrl+c to stop ... open http://localhost: in a web browser to see the default project in action. this page is only visible in development mode. add some content to the front page of the application to support production deployment to elastic beanstalk. use rails generate to create a controller, route, and view for your welcome page. ~/eb-rails$ rails generate controller welcomepage welcome create app/controllers/welcome_page_controller.rb route get 'welcome_page/welcome' invoke erb create app/views/welcome_page create app/views/welcome_page/welcome.html.erb invoke test_unit create test/controllers/welcome_page_controller_test.rb invoke helper create app/helpers/welcome_page_helper.rb invoke test_unit invoke assets invoke coffee create app/assets/javascripts/welcome_page.coffee invoke scss create app/assets/stylesheets/welcome_page.scss. this gives you all you need to access the page at /welcome_page/welcome. before you publish the changes, however, change the content in the view and add a route to make this page appear at the top level of the site. use a text editor to edit the content in app/views/welcome_page/welcome.html.erb. for this example, you'll use cat to simply overwrite the content of the existing file. example app/views/welcome_page/welcome.html.erb welcome!

this is the front page of my first rails application on elastic beanstalk.

finally, add the following route to config/routes.rb: example config/routes.rb rails.application.routes.draw do get 'welcome_page/welcome' root 'welcome_page#welcome' this tells rails to route requests to the root of the website to the welcome page controller's welcome method, which renders the content in the welcome view (welcome.html.erb). configure rails settings use the elastic beanstalk console to configure rails with environment properties. set the secret_key_base environment property to a string of up to alphanumeric characters. rails uses this property to create keys. therefore you should keep it a secret and not store it in source control in plain text. instead, you provide it to rails code on your environment through an environment property. to configure environment properties in the elastic beanstalk console open the elastic beanstalk console, and in the regions list, select your aws region. in the navigation pane, choose environments, and then choose the name of your environment from the list. note if you have many environments, use the search bar to filter the environment list. in the navigation pane, choose configuration. in the software configuration category, choose edit. under environment properties, enter key-value pairs. choose apply. now you're ready to deploy the site to your environment. deploy your application create a source bundle containing the files created by rails. the following command creates a source bundle named rails-default.zip. ~/eb-rails$ zip ../rails-default.zip -r * .[^.]* upload the source bundle to elastic beanstalk to deploy rails to your environment. to deploy a source bundle open the elastic beanstalk console, and in the regions list, select your aws region. in the navigation pane, choose environments, and then choose the name of your environment from the list. note if you have many environments, use the search bar to filter the environment list. on the environment overview page, choose upload and deploy. use the on-screen dialog box to upload the source bundle. choose deploy. when the deployment completes, you can choose the site url to open your website in a new tab. cleanup when you finish working with elastic beanstalk, you can terminate your environment. elastic beanstalk terminates all aws resources associated with your environment, such as amazon ec instances, database instances, load balancers, security groups, and alarms. to terminate your elastic beanstalk environment open the elastic beanstalk console, and in the regions list, select your aws region. in the navigation pane, choose environments, and then choose the name of your environment from the list. note if you have many environments, use the search bar to filter the environment list. choose environment actions, and then choose terminate environment. use the on-screen dialog box to confirm environment termination. with elastic beanstalk, you can easily create a new environment for your application at any time. next steps for more information about rails, visit rubyonrails.org. as you continue to develop your application, you'll probably want a way to manage environments and deploy your application without manually creating a .zip file and uploading it to the elastic beanstalk console. the elastic beanstalk command line interface (eb cli) provides easy-to-use commands for creating, configuring, and deploying applications to elastic beanstalk environments from the command line. finally, if you plan on using your application in a production environment, you will want to configure a custom domain name for your environment and enable https for secure connections. javascript is disabled or is unavailable in your browser. to use the aws documentation, javascript must be enabled. please refer to your browser's help pages for instructions. document conventions procfile tutorial - sinatra did this page help you? - yes thanks for letting us know we're doing a good job! if you've got a moment, please tell us what we did right so we can do more of it. did this page help you? - no thanks for letting us know this page needs work. we're sorry we let you down. if you've got a moment, please tell us how we can make the documentation better. securing your webhooks - github docs github docs all products developers overview about github's apis managing deploy keys viewing deployment history using ssh agent forwarding secret scanning replacing github services github developer program webhooks and events webhooks about webhooks creating webhooks configuring your server to receive payloads testing webhooks securing your webhooks webhook events and payloads events github event types issue event types apps getting started with apps about apps activating beta features for apps differences between github apps and oauth apps migrating oauth apps to github apps setting up your development environment to create a github app building github apps creating a github app setting permissions for github apps authenticating with github apps identifying and authorizing users for github apps rate limits for github apps refreshing user-to-server access tokens creating a github app from a manifest creating a github app using url parameters creating a custom badge for your github app managing github apps installing github apps modifying a github app editing a github app's permissions making a github app public or private suspending a github app installation transferring ownership of a github app deleting a github app building oauth apps creating an oauth app authorizing oauth apps scopes for oauth apps creating a custom badge for your oauth app managing oauth apps modifying an oauth app transferring ownership of an oauth app troubleshooting authorization request errors troubleshooting oauth app access token request errors deleting an oauth app guides using the github api in your app using content attachments creating ci tests with the checks api github marketplace github docs explore by product developers github.com enterprise administrators github discussions github actions github packages developers rest api graphql api github insights education github desktop github cli atom electron english english 简体中文 (simplified chinese) 日本語 (japanese) español (spanish) português do brasil (portuguese) article version: github.com github.com enterprise server . enterprise server . enterprise server . enterprise server . github ae see all enterprise releases developers webhooks and events webhooks securing your webhooks article version: github.com github.com enterprise server . enterprise server . enterprise server . enterprise server . github ae see all enterprise releases securing your webhooks ensure your server is only receiving the expected github requests for security reasons. in this article setting your secret token validating payloads from github once your server is configured to receive payloads, it'll listen for any payload sent to the endpoint you configured. for security reasons, you probably want to limit requests to those coming from github. there are a few ways to go about this--for example, you could opt to allow requests from github's ip address--but a far easier method is to set up a secret token and validate the information. you can use the repository, organization, and app webhook rest apis to create, update, delete, and ping webhooks. you can also use the rest api to change the configuration of the webhook. for example, you can modify the payload url, content type, ssl verification, and secret. for more information, see: repository webhooks rest api organization webhooks rest api github app webhooks rest api setting your secret token you'll need to set up your secret token in two places: github and your server. to set your token on github: navigate to the repository where you're setting up your webhook. fill out the secret textbox. use a random string with high entropy (e.g., by taking the output of ruby -rsecurerandom -e 'puts securerandom.hex( )' at the terminal). click update webhook. next, set up an environment variable on your server that stores this token. typically, this is as simple as running: $ export secret_token=your_token never hardcode the token into your app! validating payloads from github when your secret token is set, github uses it to create a hash signature with each payload. this hash signature is included with the headers of each request as x-hub-signature- . note: for backward-compatibility, we also include the x-hub-signature header that is generated using the sha- hash function. if possible, we recommend that you use the x-hub-signature- header for improved security. the example below demonstrate using the x-hub-signature- header. for example, if you have a basic server that listens for webhooks, it might be configured similar to this: require 'sinatra' require 'json' post '/payload' do push = json.parse(params[:payload]) "i got some json: #{push.inspect}" end the intention is to calculate a hash using your secret_token, and ensure that the result matches the hash from github. github uses an hmac hex digest to compute the hash, so you could reconfigure your server to look a little like this: post '/payload' do request.body.rewind payload_body = request.body.read verify_signature(payload_body) push = json.parse(params[:payload]) "i got some json: #{push.inspect}" end def verify_signature(payload_body) signature = 'sha =' + openssl::hmac.hexdigest(openssl::digest.new('sha '), env['secret_token'], payload_body) return halt , "signatures didn't match!" unless rack::utils.secure_compare(signature, request.env['http_x_hub_signature_ ']) end your language and server implementations may differ from this example code. however, there are a number of very important things to point out: no matter which implementation you use, the hash signature starts with sha =, using the key of your secret token and your payload body. using a plain == operator is not advised. a method like secure_compare performs a "constant time" string comparison, which helps mitigate certain timing attacks against regular equality operators. did this doc help you? privacy policy want to learn about new docs features and updates? sign up for updates! we're continually improving our docs. we'd love to hear how we can do better. what problem did you have? required choose an option information was unclear the content was confusing the article didn't answer my question other let us know what we can do better optional can we contact you if we have more questions? optional send thank you! your feedback has been submitted. help us make these docs great! all github docs are open source. see something that's wrong or unclear? submit a pull request. make a contribution or, learn how to contribute. still need help? ask the github community contact support © github, inc. terms privacy security status help contact github pricing developer api training about parade of quartets collection collection items - digital library of georgia the digital library of georgia site may be unavailable on friday, february th and saturday february th due to network maintenance. toggle navigation home search explore items collections counties institutions map teach educator resources national history day galileo new georgia encyclopedia using dlg materials reusable materials participate archivesspace hosting contribute your collections nominate collections subgranting program services for partners georgia homeplace documentation and reports presentations and workshops promotional materials donate connect contact us blog listserv newsletter facebook twitter about mission, guiding principles, and goals collection development policy dlg partners and sponsors dlg api search box search records and full text records full text records and full text search box × searching help select "records and full text" to search both the metadata and available full text. select "records" to search only metadata for items. (all items have metadata.) select "full text" to search only the scanned or transcribed text for items. (not all items have full text) close parade of quartets collection toggle refine refine creator howard, karlton howard, henry, - subject african american gospel singers--georgia--augusta african american gospel singers--united states african americans--politics and government african americans--religion augusta (ga.)--religion augusta (ga.)--religious life and customs gospel music gospel music--georgia--augusta gospel musicians--georgia--augusta gospel musicians--united states see all values for subject » people king, martin luther, jr., - mckinney, cynthia, - washington, grover, jr., - location united states, georgia, richmond county, augusta united states, georgia, richmond county, augusta united states, mississippi, madison county, ridgeland united states, mississippi, pike county, summit united states, georgia, fulton county, atlanta united states, south carolina, greenville county, greenville georgia county richmond fulton year year range begin – year range end current results range from to view distribution medium moving images television programs type moving image full text available no rights in copyright collection name parade of quartets collection holding institution walter j. brown media archives and peabody awards collection refine info the collection consists of hundreds of hours of footage from parade of quartets, a gospel program aired on wjbf-tv in augusta, georgia. this collection documents decades of regional gospel music performances, religious practices, and political activities, and represents possibly the largest collection of gospel performance footage at any north american library. the collection is a rare example of a sustained african american media presence on a southern television affiliate. from to the present, wjbf-tv in augusta, georgia has broadcast the gospel music program parade of quartets, forming the backbone of the region's african american gospel music tradition for nearly sixty years. hundreds of well-known black gospel musicians such as shirley caesar, the mighty clouds of joy, the dixie hummingbirds, james brown, and the swanee quintet have appeared on the program over the years. in the last few decades, the program's content has expanded to include appearances by both local and national african american political leaders. more about this collection medium television programs date of original / subject gospel music--georgia--augusta gospel music gospel musicians--georgia--augusta gospel musicians--united states television broadcasting of music--georgia--augusta african american gospel singers--georgia--augusta african american gospel singers--united states gospel singers--georgia--augusta gospel singers--united states caesar, shirley, - mighty clouds of joy (musical group) dixie hummingbirds swanee quintet african american politicians--georgia--augusta african american politicians--united states augusta (ga.)--religious life and customs augusta (ga.)--religion african americans--religion african americans--politics and government augusta (ga.)--politics and government-- th century augusta (ga.)--politics and government-- st century parade of quartets (television program) religious works location united states, georgia, richmond county, augusta, . , - . type movingimage description the collection consists of hundreds of hours of footage from parade of quartets, a gospel program aired on wjbf-tv in augusta, georgia. this collection documents decades of regional gospel music performances, religious practices, and political activities, and represents possibly the largest collection of gospel performance footage at any north american library. the collection is a rare example of a sustained african american media presence on a southern television affiliate. from to the present, wjbf-tv in augusta, georgia has broadcast the gospel music program parade of quartets, forming the backbone of the region's african american gospel music tradition for nearly sixty years. hundreds of well-known black gospel musicians such as shirley caesar, the mighty clouds of joy, the dixie hummingbirds, james brown, and the swanee quintet have appeared on the program over the years. in the last few decades, the program's content has expanded to include appearances by both local and national african american political leaders. rights search constraints « previous | - of | next » sort by relevance relevance title date (ascending) date (descending) collections first number of results to display per page per page per page per page per page per page view results as: gallery list map map search results . parade of quartets. [unknown episode] creator: howard, karlton date of original: / holding institution: walter j. brown media archives and peabody awards collection . parade of quartets. [unknown episode] creator: howard, karlton date of original: / holding institution: walter j. brown media archives and peabody awards collection . parade of quartets. [unknown episode] creator: howard, karlton date of original: / holding institution: walter j. brown media archives and peabody awards collection . parade of quartets. [unknown episode] creator: howard, karlton date of original: / holding institution: walter j. brown media archives and peabody awards collection . parade of quartets. [unknown episode] creator: howard, karlton date of original: / holding institution: walter j. brown media archives and peabody awards collection . parade of quartets. [unknown episode] creator: howard, karlton date of original: / holding institution: walter j. brown media archives and peabody awards collection . parade of quartets. [unknown episode] creator: howard, karlton date of original: / holding institution: walter j. brown media archives and peabody awards collection . parade of quartets. [unknown episode] creator: howard, karlton date of original: / holding institution: walter j. brown media archives and peabody awards collection . parade of quartets. [unknown episode] creator: howard, karlton date of original: / holding institution: walter j. brown media archives and peabody awards collection . parade of quartets. [unknown episode] creator: howard, karlton date of original: / holding institution: walter j. brown media archives and peabody awards collection « previous next » … home about digital public library of america georgia historic newspapers civil rights digital library civil war in the american south the digital library of georgia is part of the galileo initiative © digital library of georgia number go up! new bitcoin peak, exactly three years after the last — what’s happening here – attack of the foot blockchain skip to content attack of the foot blockchain blockchain and cryptocurrency news and analysis by david gerard about the author attack of the foot blockchain: the book book extras business bafflegab, but on the blockchain buterin’s quantum quest dogecoin ethereum smart contracts in practice icos: magic beans and bubble machines imogen heap: “tiny human”. total sales: $ . index libra shrugged: how facebook tried to take over the money my cryptocurrency and blockchain commentary and writing for others press coverage: attack of the foot blockchain press coverage: libra shrugged table of contents the conspiracy theory economics of bitcoin the dao: the steadfast iron will of unstoppable code search for: main menu number go up! new bitcoin peak, exactly three years after the last — what’s happening here th december th december - by david gerard - comment so firstly, i called it two years ago: i'd expect the bubbly headlines to start again - , it feels to me like that's about how long it will take to grow a fresh crop of greater fools. in general – there's always going to be people who are desperate to buy into this year's version of ostrich farming. — david gerard 🐍👑 (@davidgerard) november , not that this was any feat of prognostication. bitcoin is a speculative commodity without a use case — there is nothing it can do, other than bubble, or fail to be bubbling. it was always destined to stagger along, and then jump at some point, almost certainly for stupid reasons. remember that the overarching goal of the entire crypto industry is to get those rare actual-dollars flowing in again. that’s what this is about. we saw about million tethers being lined up on binance and huobi in the week previously. these were then deployed en masse. pump it! you can see the pump starting at : utc on december. btc was $ , . on coinbase at : utc. notice the very long candles, as bots set to sell at $ , sell directly into the pump. tether hit billion, i hit , twitter followers, btc hit $ , . it is clearly as jfk jr. foretold. a series of peaks followed, as the pumpers competed with bagholders finally taking their chance to cash out — including $ , , at : utc december, $ , . precisely at : utc december, and the peak as i write this, $ , . precisely at : utc december. this was exactly three years after the previous high of $ , . on december . [fortune] coinbase btc/usd chart at the moment of the peak. dull, isn’t it? can you cash out? the coinbase chart showed quite a lot of profit-taking, as bagholders could finally dump. when you see a pump immediately followed by a drop, that’s what’s happening. approximately zero of the people saying “best investment of the last decade” got in a decade ago — they bought at a peak, and are deliriously happy that they can finally cash out. seriously: if you have bitcoin bags, this is the time to at least make up your cost basis — sell enough btc to make up the actualmoney you put in. then the rest is free money. binance and coinbase showed the quality we’ve come to expect of cryptocurrency exchanges — where you can make them fall over by using them. but i’m sure coinbase will get right onto letting you have your actual dollars. [cointelegraph] so does everyone love bitcoin now? not on the evidence. the google chart is still dead — see above. that peak in is what genuine organic retail interest looks like. this isn’t that, at all. retail still isn’t diving into bitcoin — even with michael saylor at microstrategy and jack dorsey at square spending corporate funds on marketing bitcoin as an investment for actual institutions, and getting holders at hedge funds to do the same. but the marketing will continue. remember that there’s a lot of stories happening in crypto right now. bitmex is still up, but arthur hayes is a fugitive. the department of justice accuses bitmex — and specifically hayes — of trading directly against their own customers. ifinex (bitfinex/tether) are woefully short of the documents that the new york attorney general subpoenaed and that they’ve been fighting against producing for two years now, and that they must produce by january . remember: if ifinex had the documents, they’d have submitted them by now. so there’s a pile of people in trouble — who have coins they need to dump. (and john mcafee did promise “i will eat my dick on national television” if btc didn’t hit $ , within three years of . so maybe this is a last ditch mcafee penis pump.) [the dickening] a pumped bitcoin peak is just one story among many going on right now — and a completely expected one. update: here’s the smoking gun that this was a coordinated pump fueled by stablecoins — different addresses trying to deposit stablecoins to exchanges in one block of transactions on ethereum, just a few minutes before the first price peak. [twitter] lots of people deposited stablecoins to exchanges mins before breaking $ k. price is all about consensus. i guess the sentiment turned around to buy $btc at that time. this indicator is helpful to see buying power. set alert 👉 https://t.co/xjy mvaa pic.twitter.com/gv j n r g — ki young ju 주기영 (@ki_young_ju) december , start the new year by finding a way to create a little joy, no matter how small or fleeting pic.twitter.com/fiicb bw c — foone (@foone) january , your subscriptions keep this site going. sign up today! share this: click to share on twitter (opens in new window) click to share on facebook (opens in new window) click to share on linkedin (opens in new window) click to share on reddit (opens in new window) click to share on telegram (opens in new window) click to share on hacker news (opens in new window) click to email this to a friend (opens in new window) taggedbinancebitcoincoinbasenumber go uptrading post navigation previous article me in bitcoinblog.de: “bitcoin piles up so many layers of ideas that are bad or wrong or just don’t work.” next article sec sues ripple labs, claiming xrp is a security one comment on “number go up! new bitcoin peak, exactly three years after the last — what’s happening here” sherman mccoy says: st december at : am i am curious what pair are the arbitrage bots arbitraging? they end up holding tether i presume after selling into the pump, what then? reply leave a reply cancel reply your email address will not be published. required fields are marked * comment name * email * website notify me of follow-up comments by email. notify me of new posts by email. this site uses akismet to reduce spam. learn how your comment data is processed. search for: click here to get signed copies of the books! get blog posts by email! email address subscribe support this site on patreon! hack through the blockchain bafflegab: $ /month for early access to works in progress! $ /month for early access and even greater support! buy the books! libra shrugged us paperback uk/europe paperback isbn- : kindle: uk, us, australia, canada (and all other kindle stores) — no drm google play books (pdf) apple books kobo smashwords other e-book stores attack of the foot blockchain us paperback uk/europe paperback isbn- : kindle: uk, us, australia, canada (and all other kindle stores) — no drm google play books (pdf) apple books kobo smashwords other e-book stores available worldwide rss - posts rss - comments recent blog posts tether printer go brrrrr — cryptocurrency’s substitute dollar problem news: craig wright copyright claims, grayscale, mt. gox, reddit on ethereum, the french bitcoin neo-nazi blockchain debate podcast: diem is a glorified paypal (david gerard vs. bryce weiner) stablecoins through history — michigan bank commissioners report, news: twitter on the blockchain, the us coup on the blockchain, gensler in at the sec, ripple ex-cto loses bitcoin keys excerpts from the book table of contents the conspiracy theory economics of bitcoin dogecoin buterin’s quantum quest icos: magic beans and bubble machines ethereum smart contracts in practice the dao: the steadfast iron will of unstoppable code business bafflegab, but on the blockchain imogen heap: “tiny human”. total sales: $ . index about press coverage for attack of the foot blockchain press coverage for libra shrugged my cryptocurrency and blockchain press commentary and writing facebook author page about the author contact the content of this site is journalism and personal opinion. nothing contained on this site is, or should be construed as providing or offering, investment, legal, accounting, tax or other advice. do not act on any opinion expressed here without consulting a qualified professional. i do not hold a position in any crypto asset or cryptocurrency or blockchain company. amazon product links on this site are affiliate links — as an amazon associate i earn from qualifying purchases. (this doesn’t cost you any extra.) copyright © – david gerard powered by wordpress and hitmag. send to email address your name your email address cancel post was not sent - check your email addresses! email check failed, please try again sorry, your blog cannot share posts by email. our mission | adho skip to main content main menu about statements committees sigs publications conference awards news resources adho membership options our mission the alliance of digital humanities organizations (adho) promotes and supports digital research and teaching across all arts and humanities disciplines, acting as a community-based advisory force, and supporting excellence in research, publication, collaboration and training. the following organizations are current members of adho: association for computers and the humanities (ach) australasian association for digital humanities (aadh) canadian society for digital humanities / société canadienne des humanités numériques (csdh/schn) centernet digital humanities association of southern africa (dhasa) european association for digital humanities (eadh) humanistica, l'association francophone des humanités numériques/digitales (humanistica) japanese association for digital humanities (jadh) red de humanidades digitales (redhd) taiwanese association for digital humanities (tadh) news minutes of the constituent organization board & executive board meetings the minutes from adho's constituent organization board & executive board meetings have been published. the meetings were held virtually in conjunction with the virtual digital humanities conference. a pdf version is available here. read more roy wisbey - the following was originally posted on the eadh website. adho also wishes to extend condolences to professor wisbey's family, friends, and colleagues. it is with great sadness that we inform our community of the death of roy wisbey, who has passed away aged ( . . - . . ). prof. wisbey has been a founder member and first chair of the association of literary and linguistic computing (now eadh), as much as founder of our journal "literary and linguistic computing" (now "dsh"). read more subscribe to adho news follow us on facebook | twitter | youtube | event calendar search form search advance issue of digital scholarship in the humanities (formerly llc) data visualization technique to study the conceptual metaphors in divan of hafiz and bustan of sa'adi machine translation and global research: towards improved machine translation literacy in the scholarly community. lynne bowker and jairo buitrago ciro open scholarship in australia: a review of needs, barriers, and opportunities advance issue of digital humanities quarterly (dhq) playing with unicorns: ai dungeon and citizen nlp preview author biographies introduction: digital humanities & colonial latin american studies modeling amerindian sea travel in the early colonial caribbean contact email us | facebook | twitter this work is licensed under a creative commons attribution . unported license. adho constituent organizations association for computers and the humanities (ach), australasian association for digital humanities (aadh), canadian society for digital humanities / société canadienne des humanités numériques (csdh/schn), centernet, digital humanities association of southern africa (dhasa), european association for digital humanities (eadh), humanistica, l'association francophone des humanités numériques/digitales (humanistica), japanese association for digital humanities (jadh), red de humanidades digitales (redhd), taiwanese association for digital humanities (tadh) vol ( ): including special focus on academic libraries and the irrational | canadian journal of academic librarianship skip to main content skip to main navigation menu skip to site footer current archives announcements about about the journal submissions tips for authors guidelines for peer review book reviews editorial team privacy statement contact search search register login home / archives / vol ( ): including special focus on academic libraries and the irrational special focus on academic libraries and the irrational: karen nicholson, western university jane schmidt, ryerson university lisa sloniowski, york university cover image: francisco de goya y lucientes, the literate ass, - , pencil and iron gall ink, x mm, museo nacional del prado, madrid (permalink: https://www.museodelprado.es/en/the-collection/art-work/the-literate-ass/ de -f - f b- f- c fc ). copyright: museo nacional del prado. published: - - special focus on academic libraries and the irrational editorial karen p. nicholson, jane schmidt, lisa sloniowski - pdf pdf (français (canada)) making space for the ‘irrational’ practice of anthropology in libraries donna lanclos - pdf now you see it, now you don't the disappearing collection of western university's d.b weldon library alec mullender, marnie james - pdf the quarter floors alan harnum - pdf the digital disease in academic libraries kris joseph - pdf material entanglement and technology fetishism in academic libraries lisa levesque - pdf “nice white meetings” unpacking absurd library bureaucracy through a critical race theory lens lalitha nataraj, holly hampton, talitha r. matlin, yvonne nalani meulemans - pdf the antinomies of academic freedom reason, trans rights, and constituent power sam popowich - pdf the coin of love and virtue: academic libraries and value in a global pandemic maura seale, rafia mirza - pdf the efficiency toolkit: a subversive manual andrea kohashi - html pdf critical library performativity: toward progressive change in academic library management and organizations danya leebaw - pdf the labour of austerity absurdity, performative resistance, and cultural transformation nora almeida - pdf book reviews toward a critical-inclusive assessment practice for library instruction, by lyda fontes mccartin and rachel dineen emily hector - pdf a matter of facts: the value of evidence in an information age, by laura a. millar cathy troupos - pdf libraries promoting reflective dialogue in a time of political polarization, edited by andrea baer, ellysa stern cahoy, and robert schroeder jeremy mcginniss - pdf open and equitable scholarly communications: creating a more inclusive future, by acrl susan bond - pdf foundations of information ethics, edited by john t.f. burgess and emily j.m. knox justin fuhr - pdf language english français (canada) make a submission browse categories special focus on research and scholarship special focus on diversity information for readers for authors for librarians the canadian journal of academic librarianship is published by the canadian association of professional academic librarians. follow us: @cjalrcbu issn - x rails autoscale | heroku dev center skip navigationshow navheroku dev center get started documentation changelog search get started node.js ruby on rails ruby python java php go scala clojure documentation changelog moreadditional resources home elements products pricing careers help status events podcasts compliance center heroku blogheroku blog find out what's new with heroku on our blog. visit blog log inorsign up view categories categories heroku architecture dynos (app containers) stacks (operating system images) networking & dns platform policies platform principles command line deployment deploying with git deploying with docker deployment integrations continuous delivery continuous integration language support node.js ruby working with bundler rails support python working with django background jobs in python java working with maven java database operations working with the play framework java advanced topics working with spring boot php go go dependency management scala clojure databases & data management heroku postgres postgres basics postgres performance postgres data transfer & preservation postgres availability postgres special topics heroku redis apache kafka on heroku other data stores monitoring & metrics logging app performance add-ons all add-ons collaboration security app security identities & authentication compliance heroku enterprise private spaces infrastructure networking enterprise accounts enterprise teams heroku connect (salesforce sync) single sign-on (sso) patterns & best practices extending heroku platform api app webhooks heroku labs building add-ons add-on development tasks add-on apis add-on guidelines & requirements building cli plugins developing buildpacks dev center accounts & billing troubleshooting & support add-ons all add-ons rails autoscale this add-on is operated by rails autoscale simple, reliable queue-based autoscaling for your rails app. rails autoscale last updated june table of contents full documentation how is this different from heroku’s own autoscaling? installation adjust your settings in the rails autoscale dashboard worker autoscaling troubleshooting migrating between plans removing the add-on support rails autoscale is an add-on that automatically scales web and worker dynos for rails applications. autoscaling ensures that your app can gracefully handle increased traffic while only paying for the dynos you need. full documentation this documentation page is a high-level summary. see the official documentation site for more. how is this different from heroku’s own autoscaling? heroku offers a native autoscaling solution that’s worth a try if you run performance dynos and you only need to autoscale web dynos. rails autoscale goes beyond this by autoscaling based on request queue times, autoscaling worker dynos, and working with all dyno types. for more info, check out the common questions doc. installation i’ll walk you through the full installation and setup process in the -minute video below. additional instructions are in the getting started guide. rails autoscale can be installed from railsautoscale.com, the heroku elements marketplace, or the via cli: check out rails autoscale on heroku elements for pricing and plans. $ heroku addons:create rails-autoscale -----> adding rails_autoscale_url to sharp-mountain- ... done, v (free) after rails autoscale is provisioned, a rails_autoscale_url config var is available in the app configuration. you don’t have to do anything with it. this setting is used by the rails_autoscale_agent gem when communicating with the rails autoscale service. rails autoscale supports rails . + and ruby . . +. if you run into any trouble with the initial setup, check out the official getting started docs or the troubleshooting guide. adjust your settings in the rails autoscale dashboard after you install the gem and deploy your application, you’ll begin to see activity in your rails autoscale dashboard. access the dashboard via the cli: $ heroku addons:open rails-autoscale opening rails_autoscale_url for sharp-mountain- you can also access the dashboard from your app’s heroku dashboard: the rails autoscale dashboard documentation is here. request queue time threshold rails autoscale triggers scale events by monitoring the request queue time for every request. this is the time between heroku accepting a request and your application beginning processing. queue time will increase when your application servers are too busy to handle all incoming requests. for a thorough explanation of queue time, read nate berkopec’s excellent article “scaling ruby apps to requests per minute - a beginner’s guide”. rails autoscale tracks the th percentile queue time, which for most applications will hover well below the default threshold of ms. a breach of this threshold will trigger an upscale for your application, adding one web dyno. after minutes of measurements below the downscale threshold ( ms by default), a downscale event is triggered. rails autoscale’s queue time metrics will not match apm tools like scout and new relic. those tools typically show an average, while rails autoscale tracks the th percentile. settings page rails autoscale provides default settings that work for most apps, but some apps may need a higher or lower threshold. these settings are described in detail in the official docs. worker autoscaling rails autoscale supports autoscaling worker dynos that are using sidekiq, resque, delayed job, or que. it’s as easy as autoscaling your web dynos, and the settings page will walk you through the setup. see the launch announcment for a video walkthrough. troubleshooting check out the official troubleshooting guide if you run into any problems. if you’re still stuck, please email help@railsautoscale.com. migrating between plans use the heroku addons:upgrade command to migrate to a new plan. $ heroku addons:upgrade rails-autoscale:newplan -----> upgrading rails-autoscale:newplan to sharp-mountain- ... done, v ($ /mo) your plan has been updated to: rails-autoscale:newplan removing the add-on rails autoscale can be removed via the cli. this will destroy all associated data and cannot be undone! $ heroku addons:destroy rails-autoscale -----> removing rails-autoscale from sharp-mountain- ... done, v (free) support all rails autoscale support and runtime issues should be submitted via heroku’s support channels or emailed to help@railsautoscale.com. keep reading all add-ons feedback log in to submit feedback. ziggeo raygun crash reporting information & support getting started documentation changelog compliance center training & education blog podcasts support channels status language reference node.js ruby java php python go scala clojure other resources careers elements products pricing subscribe to our monthly newsletter your email address: rss dev center articles dev center changelog heroku blog heroku news blog heroku engineering blog heroku podcasts twitter dev center articles dev center changelog heroku heroku status facebook instagram github linkedin youtube heroku is acompany © salesforce.com heroku.com terms of service privacy cookies dan cohen – vice provost, dean, and professor at northeastern university skip to the content search dan cohen vice provost, dean, and professor at northeastern university menu about blog newsletter podcast publications social media cv rss search search for: close search close menu about blog newsletter podcast publications social media cv rss what’s new podcast humane ingenuity newsletter blog publications © dan cohen powered by wordpress to the top ↑ up ↑ media – safer communities in a 'smart tech' world skip to the content search safer communities in a 'smart tech' world resisting ring in windsor, ontario menu home media menu search search for: close search close menu home media blog contact home media email media cbc news – february , what canada can learn from this michigan city’s use of doorbell cameras to catch criminals (bonnie stewart) cbc tv the national – february , (bonnie stewart) cbc radio – january , windsor partnership with amazon ring doorbell could do more harm than good, experts say (kristen thomasen) am – january , the dan macdonald show (natalie deckard) ctv news – january , should windsor be the first amazon ring city in canada? (bonnie stewart) the conversation – january , one ring to rule them all: surveillance ‘smart’ tech won’t make canadian cities safer (bonnie stewart) — reprinted, the vancouver sun — reprinted, the province rose city politics – january , mayor dilkens and the ring (bonnie stewart, natalie deckard, mita williams) related media am – dan mac donald show – february , friday roundtable with rino bortolin, yvonne pilon, & jon liedtke email about this site this event is a project of: natalie delia deckard, department. of sociology, anthropology & criminology bonnie stewart, faculty of education kristen thomasen, law and mita williams, leddy library. thank you to our sponsors the transnational law & justice network (tljn), windsor law faculty of education, university of windsor department. of sociology, anthropology & criminology, university of windsor © safer communities in a 'smart tech' world powered by wordpress to the top ↑ up ↑ how can i get a job in d&i(part )? | by aubrey blanche, the mathpath | noteworthy - the journal blog sign in your stories try journal how can i get a job in d&i(part )? this is perhaps the second most common question i get when people reach out to me for advice. you’ll find that mine is pretty unorthodox, and that i veer towards the practical (rather than philosophically perfect). this post is the second in my series seeking to balance helping more people with protecting my personal and professional time. aubrey blanche, the mathpathfollow apr , · min read you’d basically be hiding under a rock if you weren’t aware of the growth in diversity & inclusion functions and roles across almost every industry. businesses are (far too slowly) recognizing the value of building a properly supported function, and that it’s not just an ethical question, but a financial viability one. on a personal note, i have been getting more requests than ever to share how i got into the d&i field, and what advice i could offer to other job seekers. i’ll be honest that i don’t know that most of my experience is replicable, and also strongly believe that most of the folks who are asking shouldn’t be going into d&i as a job or career. so here, you’re going to get my frankest advice on pursuing a d&i career in tech, and i’ll caveat that here i’m focused on practical, usable advice based on what happens in the world. while i very much dislike the terms “diversity & inclusion”, and prefer balance & belonging, i’ll be using d&i here for legibility for folks newer to the space. read my rant on “diversity” here. please don’t get a job in d&i my first piece of advice is if you care about diversity & inclusion, please do not go have a career in it. below are my strongly considered reasons why. this is going to sound really negative, and i want to emphasize that i absolutely love my job and career. but just like a romantic partner, you need to opt into the downsides with eyes wide open if you’re going to be successful, and the job probably isn’t what you think it is. d&i gives up almost all formal authority. the fact is, even in a highly supportive organization, the d&i has very little formal power over the processes and systems that they are charged with modifying. you usually have basically no sticks, and a couple of small carrots. while you’ll be responsible for changing hiring, development, retention, engagement, etc. you likely won’t have formal authority over the programs and processes that directly drive those numbers. you’re also a cost center, since you’re not directly driving revenue for the business in most cases, and quantifying the impact of your work is much more complicated than if you were responsible for an operational function like recruiting. working on a cause you care about isn’t necessarily fulfilling. many people who are unfulfilled in their current roles and passionate about positive culture and equity (or maybe they’re a marginalized or underrepresented person) think they should move into d&i and they’ll feel more fulfilled. mostly, they’re wrong. research from imperative and the taproot foundation show that fulfillment and meaning are driven less by focusing on a cause (no matter how meaningful) than how we do our work. if you’re emotionally burned out from being an underrepresented person in tech and think that the shift will solve that, it won’t. it will probably make it worse, because you won’t have the content of your job to distract you from the kyriarchy. it’s more org strategy and process design than activism, most days. there are days when i have huge activism wins, and really move the needle for making an organizational process or program more equitable (or equity creating). but most of the time, my work is about people strategy, process design, program management, acting as a therapist and corporate conscience, etc. if you’re not excited about doing the nitty gritty there, you might be happier in a different role. it’s a *lot* of answering very basic questions with a smile. in my personal time, i don’t have to answer basic questions about meritocracy. in my job, i have opted in to answering these questions. repeatedly. with compassion. times. i’m definitely ok with that, but you should know you need to be to be effective in this job. there are a lot of people at the beginning of their journey and you’re their teacher. most companies want to shine a turd, not create structural change. most companies are willing to commit to some things, like updating their careers page with more black people and maybe conducting pay equity audits. think of a major tech company that touts their yearly pay equity corrections and hired the majority of the d&i team from marketing (there are plenty). there’s a reason: most leadership teams aren’t ready to cosign things like major changes to how employees are incentivized or assessed, or even terminate folks who are abusive. make sure you’re ok with that reality, and can live with yourself working in that system. every job is a d&i job. every. job. say it with me: every job is a d&i job. to build an equitable, inclusive company, every single person needs to do their part. if you stay exactly where you are, and turn your job into a d&i job, you can often create more organizational change. let’s say that you’re a director of marketing. how might that show up: hiring: tell your recruiting counterparts that you require a balanced slate of candidates before you make offers. pay, promotion: audit your rates to ensure fairness in your org. culture: have a zero-tolerance policy for exclusive behavior, model that for your org, and be willing to immediately remove team members who aren’t aligned. job content: drive inclusion and representation in your company’s brand and customer segmentation. influence: openly talk about building equity with your peers. be part of the social pressure to invest in these topics across the organization. budget: fund d&i initiatives for under-resourced d&i teams. employees in the business are who make me look successful and actually create organizational change. unless you can’t live without a % d&i job, please go be that employee: you will be doing d&i work. if you must have a job in d&i alright, you’ve read my thoughts on why you shouldn’t take after me, and still are undeterred. here’s my best advice on how to get a job in d&i. what expertise you need most folks think about the subject matter expertise required to be a successful d&i practitioner (e.g., gender and race theory, queer studies, disability and universal access theory, etc.), but i believe that that’s only about % of the job, if you want to be effective. here’s the rest of what i think you’ll need to understand: influence & persuasion: this career is all about soft power. you’ll need to be able to speak to a variety of stakeholders with a dizzying array of priorities, motivations, and levels of giving a shit about d&i. research design & methods: as the field becomes more data-informed, having an understanding of how to identify the most important problems to solve and how to measure impact separate passionate advocates from effective professionals. strategic planning: the work is by definition cross-functional, and the more effective you are at sequencing work and managing dependencies, the less hair you’ll want to rip out and the more effective you’ll be at landing change. social organizing and change management: despite a lot of branding, no company (at least at scale) has nailed this yet. that means there is a lot of change you’re going to need to drive or support, and you’re going to need to herd a lot of cats to get there. personal insight and a hardcore self care routine: i don’t know any serious practitioner in the field that hasn’t re-oriented a lot of their life to support the emotional challenge of this work. compassion fatigue is real, pervasive, and debilitating. building a routine and regular practice around caring for yourself first is the only way you’ll healthfully have this career. i personally have a therapist; coach; food, exercise, and sleep routine; gravity blanket; emotional support animal; and deep community of friend and family support. in part , i’ll detail the most common paths to get a job in the d&i field (just in case you’re really, really serious). stay tuned! 📝 read this story later in journal. 👩‍💻 wake up every sunday morning to the week’s most noteworthy stories in tech waiting in your inbox. read the noteworthy in tech newsletter. noteworthy - the journal blog the official journal blog follow diversity inclusion careers tech activism claps claps responses written by aubrey blanche, the mathpath follow equitable design & impact @cultureamp. advisor, investor. mathpath = (math nerd + empath). queer dog mom, latina. your contribution matters. she/her. follow noteworthy - the journal blog follow the official journal blog follow written by aubrey blanche, the mathpath follow equitable design & impact @cultureamp. advisor, investor. mathpath = (math nerd + empath). queer dog mom, latina. your contribution matters. she/her. noteworthy - the journal blog follow the official journal blog more from medium quick start guide to product strategy blaine holt in noteworthy - the journal blog the end of white american christianity sara zarr in noteworthy - the journal blog rxjs better practice kelly woo in noteworthy - the journal blog a year of writing and profit on medium logan daley in noteworthy - the journal blog you are not equal. i’m sorry. dina ley in noteworthy - the journal blog dad songs (say so much) joe shetina in noteworthy - the journal blog impeachment and the th amendment won’t work, but there is something that will mitchell plitnick in noteworthy - the journal blog how to use fetch in javascript daniel yankiver in noteworthy - the journal blog learn more. medium is an open platform where million readers come to find insightful and dynamic thinking. here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. learn more make medium yours. follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. explore share your thinking. if you have a story to tell, knowledge to share, or a perspective to offer — welcome home. it’s easy and free to post your thinking on any topic. write on medium about help legal get the medium app secure the internet | nature skip to main content thank you for visiting nature.com. you are using a browser version with limited support for css. to obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in internet explorer). in the meantime, to ensure continued support, we are displaying the site without styles and javascript. advertisement view all nature research journals search my account login explore content journal information publish with us sign up for alerts rss feed nature comment article secure the internet download pdf published: november computing secure the internet ben laurie & cory doctorow Â nature volumeÂ ,Â pages â€“ ( )cite this article accesses citations altmetric metrics details subjects business information technology mathematics and computing software engineers must close the loophole used to intercept online communications, say ben laurie and cory doctorow. download pdf in , a fake adobe flash updater was discovered on the internet. to any user it looked authentic. the software's cryptographic certificates, which securely verify the authenticity and integrity of internet connections, bore an authorized signature. internet users who thought they were applying a legitimate patch unwittingly turned their computers into spies. an unknown master had access to all of their data. the keys used to sign the certificates had been stolen from a 'certificate authority' (ca), a trusted body (in this case, the malaysian agricultural research and development institute) whose encrypted signature on a website or piece of software tells a browser program that the destination is bona fide. until the breach was found and the certificate revoked, the keys could be used to impersonate virtually any site on the internet. credit: illustration by andrew rae fake certificates are used by hackers and governments to harvest online communications. in , for example, a hacker based in iran stole the signing keys from diginotar, a dutch ca that declared bankruptcy soon afterwards. the keys were used to impersonate sites such as facebook and gmail in iranian dissidents' browsers, allowing all of their messages to be read. certificates allow the web to work. they secure transactions and allow users to enter credit-card numbers, share data across networks or chat in private forums. without certificates, hackers could easily stop, corrupt or eavesdrop on these exchanges. but certificates are in trouble. as more authorizing bodies are added to browsers' lists of trusted cas, and as governments, hackers and unscrupulous insiders weaken the internet's security system, it is becoming virtually impossible to know whether a connection is legitimate. it is becoming virtually impossible to know whether a connection is legitimate. many internet technicians, ourselves included, agree that it is time to fix the problem. what remains is the substantial hurdle of reaching consensus about how. the international internet engineering task force is adding registration information to the (already overburdened) domain name system (dns). but websites can still be taken over. the electronic frontier foundation (where c.d. is a fellow), based in san francisco, california, has proposed a cryptographic protocol called sovereign keys (sk) that would make it impossible for a third party to impersonate any website. a third effort is under way, led by a team (including b.l.) at google, based in mountain view, california. this protocol â€” called certificate transparency (ct) â€” is similar to sk, but it includes an independent cross-checking system. release dates have not been set for either protocol, but ct has the potential to be rolled out sooner, through regular software updates for google's web browser, chrome. we see it as a stepping stone to a more ambitious system, such as sk. we call on browser vendors to support a shift to a more secure system. there are economic barriers â€” no one is likely to make money from shoring up the internet. but the risks of ignoring this security loophole are too great. cross-talk before your browser connects to a website, it asks your local network's dns server for the numeric address corresponding to the website's domain name. (for example, one of the addresses for www.facebook.com is . . . .) but dns is not secure â€” its communications with browsers are unscrambled, and they are easy to intercept. anyone sharing your network can steal your credit-card information or passwords. to scramble messages and keep them private, you need encryption. if a browser has received a cryptographic certificate signed by a ca, its address bar shows a key or padlock icon. all browsers have pre-installed lists of trusted cas against which to check certificates. cas do a lot of due diligence before issuing a certificate, but they are fallible. cryptographic methods can be used to spot forgery and tampering, but not to distinguish real certificates issued by diligent cas from those issued by mistake or by a ca that has been conned or taken over. the proliferation of cas is putting the entire internet at risk. governments are not responding to the problem â€” indeed, some policy-makers have shown a remarkable willingness to undermine online security for law-enforcement reasons. india's government, for example, is seeking weaker security for skype and blackberry mobile devices . in , us lawmakers proposed the stop online piracy act, which would require dns providers to return false results when users try to connect to sites accused of facilitating copyright infringement . luckily, software engineers are in a position to fix the certificate loophole. certificate transparency ct and sk rely on a type of record that uses cryptographic methods to prove that none of its past entries has been erased â€” an 'untrusted, verifiable, append-only log'. this log is based on a mathematical principle called a merkle tree â€” a hierarchy of linked items, or 'leaves'. in ct, each leaf is a certificate. for a particular node in the tree, a value can be generated in a few steps on the basis of the values of other nodes. for a tree with, say, a million leaves, a verifier would have to track only nodes to confirm any particular leaf. even a tiny alteration throws off the calculations entirely. with merkle trees it is possible to prove efficiently that a particular leaf is in the tree without revealing the contents of the other leaves. it is also impossible to fake such a proof for a leaf that is not in the tree. merkle-tree logs are stored on a small number of computers, or log servers. every time a ca generates a new certificate, it sends a copy to all the log servers, which then return a cryptographically signed proof that the certificate has been added to the log. browsers could be pre-configured with a list of verified log servers (in addition to the list of cas now installed). periodically â€” perhaps hourly â€” a number of 'monitor' servers contact the log servers and ask for a list of all the new certificates for which they have issued proofs. these monitors â€” operated by companies, banks, individuals and service providers â€” would discover any unauthorized certificates, just as credit reports alert people to cards or loans issued falsely in their names. this process works only if the log servers are honest; here, auditor servers come in. every so often, a browser sends all the proofs it has received to a number of auditors â€” anyone may act as one, because the logs are public. if a proof has been signed by a log server but does not appear in its log, the auditor knows that something is wrong. within an hour of committing their first transgressions, rogue cas and log servers could be detected and removed from browsers' lists. even with modest uptake, ct will begin cleaning up the internet immediately on roll-out. blocking will improve as more organizations, browser vendors and users participate. initially, browsers that adopt ct will not be able to block connections for which no proofs are offered. after a year, chrome will be updated to warn users before establishing a secure connection without a proof. later, it will not connect to any site without one. we hope that other browsers will follow a similar path. also based on merkle-tree logs, sk is a more theoretical approach. instead of requiring cas and domain registrars, sk issues a private key for each website to only one holder. no one else may use that key, so misuse by a third party is not a problem. anything unverified is blocked, so sk would foil attempts by governments to use domain seizures to censor content that they find to be objectionable. long-term gain through systems such as ct and sk, software engineers can and should solve the certificate problem. after all, the internet is international and independent; governments cannot mandate a solution. there are some economic barriers to improving security, but it is a worthwhile investment. cas have no short-term incentive to support these measures, but the ca network offers the best avenue for rolling out steps such as ct. similarly, someone must run the log servers, even though doing so will not lead to direct economic gain. but the long-term stability and security of the internet is good for business. in line with this view, google will run some log servers â€” but others should as well, so that we can avoid putting all of our eggs in one basket. browser vendors should also commit to supporting ct. history tells us that those who seek new avenues of attack will eventually find them. but this troubling breach must be closed down now. references anonymous. india to seek interpol help to intercept encrypted data from blackberry, gmail, skype. the economic times ( december ). us house of representatives, th congress, st sess. bill no. hr ( ). download references author information affiliations ben laurie is a visiting industrial fellow at the university of cambridge, cambridge cb tn, uk. ben laurie cory doctorow is a visiting senior lecturer in the computing department at the open university, milton keynes mk bj, uk. cory doctorow authors ben laurieview author publications you can also search for this author in pubmedÂ google scholar cory doctorowview author publications you can also search for this author in pubmedÂ google scholar corresponding authors correspondence to ben laurie or cory doctorow. ethics declarations competing interests competing interests statement: b.l. is an employee of and holds shares in google, which originated the certificate transparency proposal. c.d. is a donor to, former employee of and fellow at the electronic frontier foundation, which originated the sovereign keys proposal. c.d. has been paid by google to speak on unrelated topics. related links related links related links in nature research digital archives: don't let copyright block data mining stitching science together sociology of science: big data deserve a bigger audience related external links ct specifications sovereign keys rights and permissions reprints and permissions about this article cite this article laurie, b., doctorow, c. secure the internet. nature , â€“ ( ). https://doi.org/ . / a download citation published: november issue date: november doi: https://doi.org/ . / a further reading tls/pki challenges and certificate pinning techniques for iot and m m secure communications daniel diaz-sanchez , andres marin-lopez , florina almenarez mendoza , patricia arias cabarcos & r. simon sherratt ieee communications surveys & tutorials ( ) dns/dane collision-based distributed and dynamic authentication for microservices in iot â€ daniel dÃaz-sÃ¡nchez , andrÃ©s marÃn-lopez , florina almenÃ¡rez mendoza & patricia arias cabarcos sensors ( ) comments by submitting a comment you agree to abide by our terms and community guidelines. if you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate. download pdf explore content research articles news opinion research analysis careers books & culture podcasts videos current issue browse issues collections subjects follow us on facebook follow us on twitter sign up for alerts rss feed journal information about the journal awards publish with us for authors for referees submit manuscript search article search search advanced search quick links explore articles by subject find a job guide to authors editorial policies nature issn - (online) nature.com sitemap about us press releases press office contact us discover content journals a-z articles by subject nano protocol exchange nature index publishing policies nature portfolio policies open access author & researcher services reprints & permissions research data language editing scientific editing nature masterclasses nature research academies libraries & institutions librarian service & tools librarian portal open research recommend to library advertising & partnerships advertising partnerships & services media kits branded content career development nature careers nature conferences nature events regional websites nature china nature india nature japan nature korea nature middle east legal & privacy privacy policy use of cookies manage cookies/do not sell my data legal notice accessibility statement terms & conditions california privacy statement © springer nature limited close sign up for the nature briefing newsletter â€” what matters in science, free to your inbox daily. email address sign up i agree my information will be processed in accordance with the nature and springer nature limited privacy policy. close get the most important science stories of the day, free in your inbox. sign up for nature briefing finding articles for your social sciences research assignment finding articles for your social sciences research assignment this interactive tutorial worksheet will help you find quality sources for your assignment using library databases and google scholar. you will learn how to build a search strategy and then will get experience searching using three separate databases (library article search, jstor, and google scholar). what is your name? your answer . describe what you're researching. what are you trying to find information about? your answer scholarly vs. popular articles when doing research in the social sciences, you will likely find a mixture of popular and scholarly (also known as peer-reviewed) sources. depending on your assignment requirements, both might be useful. it's helpful to understand the characteristics of each: popular articles scholarly articles . often written by non-expert journalists . written by experts on that subject . reviewed by editors who are often . reviewed by other experts on not subject-matter experts that specific subject (peer-review) . written for a general audience . written for other scholars . published in newspapers and . published in scholarly journals magazines . sometimes the language and subject . easy for a non-expert to read matter can make it difficult to read . often includes reporting from a mix . reports the results of a research of interviews with regular people, study. interviews with experts, and summaries of reports and other scholarly articles it's also important to know what each type of source is good for in your research: popular articles... . are published more quickly so they provide timely information on a topic . often provide useful background information to help you understand the big picture . can provide a sense of popular opinion about a certain issue . often contain first-hand accounts of people who experienced an event or are impacted by an issue so they give you a sense of how everyday people are impacted . can help the reader see an issue from many different points of view. news stories often include perspectives from a variety of people involved in an issue from government officials, experts in that particular area, people working in that area, and/or the people who are impacted. scholarly articles... . provide strong expert evidence based on quality research data . can often be more convincing because most people have been taught to respect traditional educational expertise . have more validity than popular articles because they are usually studying a larger population in a systematic way. brainstorming keywords on your topic before you start searching on your topic, it's helpful to consider the various possible keywords you could use in searching. here's why: . the more words you put into a database search, the fewer results you'll get, so breaking down your research question or topic down into the most important keywords is critical. . computers are clueless and don't know that certain words are used interchangeably. for example, they don't know that a word like salary means about the same thing as earnings, wages, or pay, so it's important to think of terms that mean the same thing. . the terms that you naturally think of when you think about your topic may not be the same words that experts use to write about the topic. this requires you to think not only of the terms that naturally come to your mind, but to brainstorm terms that mean the same thing or are related. you can identify other keywords by looking for a wikipedia entry on your topic or simply doing a google search on your topic and seeing what related or synonymous words are used in the titles of results. try to think of as many synonyms or related terms as you can. here's an example research question: does the presence of a smartphone impact people's ability to pay attention in face-to-face conversations? the most important keywords are smartphone, attention, and face-to-face conversations. now, i will brainstorm other keywords that are similar to those original terms: smartphone attention face-to-face conversation cell phone focus face-to-face interaction mobile phone concentration face-to-face communication mobile device distraction face-to-face dialogue listening . now brainstorm keywords for your topic and list them here. your answer once you have your keywords, you can start searching. using connecting words (and, or, and not) will help you develop a targeted search and save time. this four-minute video from portland state university library will introduce ways to develop your search query using connecting words and other search tools. using the library's article search the libraryâ€™s article search on the front page of the library website searches our two largest multidisciplinary library databases, academic search premier and masterfile premier. they contain not just scholarly articles, but also magazine and newspaper articles written for a non-expert audience. before you search, be sure to click on the articles tab, or you'll find yourself searching for books and videos. if you want to limit your search there to just scholarly articles, you can check the box for peer reviewed on the front page (shown below). watch this four-minute video on finding articles through the library to see how our library article search works and how to search it. now you're ready to start searching on your own in the library's article search. if your assignment requires you to use scholarly sources, be sure to check the box that limits your search to scholarly peer-reviewed articles. if not, you can just enter your keywords into the library's article search and click search. try different combinations of keywords and see how they change your results list. when you find articles that look useful, email them to yourself so you'll have the article as well as the formatted citation without having to go back into the database again. . keywords you used that were successful: your answer . find at least one article that looks useful for your research project. click on its title and then click on the cite icon on the right-hand side of the page. copy the citation in mla or apa format (whatever your instructor requires) and paste it in here. your answer next page of never submit passwords through google forms. this form was created inside of portland community college. report abuse forms github - fcrepo-exts/fcrepo-migration-validator: a command-line tool for validating migrations of fedora datasets to fedora . skip to content sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories → security → team enterprise explore explore github → learn & contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → nonprofit → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this organization all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up {{ message }} fcrepo-exts / fcrepo-migration-validator watch star fork a command-line tool for validating migrations of fedora datasets to fedora . apache- . license stars forks star watch code issues pull requests actions projects security insights more code issues pull requests actions projects security insights main branch tags go to file code clone https github cli use git or checkout with svn using the web url. work fast with our official cli. learn more. open with github desktop download zip launching github desktop if nothing happens, download github desktop and try again. go back launching github desktop if nothing happens, download github desktop and try again. go back launching xcode if nothing happens, download xcode and try again. go back launching visual studio if nothing happens, download the github extension for visual studio and try again. go back latest commit git stats commits files permalink failed to load latest commit information. type name latest commit message commit time .github/workflows src .gitignore license readme.md pom.xml view code readme.md fcrepo-migration-validator a command-line tool for validating migrations of fedora datasets to fedora . about a command-line tool for validating migrations of fedora datasets to fedora . resources readme license apache- . license releases no releases published packages no packages published contributors languages java . % freemarker . % © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. university of texas libraries collections university of texas libraries collections about help contact all fields title creator/contributor date created/date issued owning repository type search browse all previous next limit your search owning repository alexander architectural archives , architecture and planning library benson latin american collection , perry-castañeda library maps , university of texas libraries , date created/date issued more date created/date issued » time period covered - bce - - bce - - - - ?- - - more time period covered » type administrative regulations advertising photographs annals and chronicles architectural photography architectural drawings (visual works) architectural photography atlases biographies biography bird's-eye perspectives more type » language arabic english , french german hebrew hindi italian latin malayalam mixtec more language » topic mission san josé y san miguel de aguayo (san antonio, tex.) mission san juan capistrano (san antonio, tex.) "a letter from ulster," movie premiere "choose for victory" poster "coal sunday" -- yorkshire, great britain more topic » place name san antonio, bexar county, texas dallas, dallas county, texas great britain houston, harris county, texas fort worth, tarrant county, texas galveston, galveston county, texas france germany beaumont, jefferson county, texas italy more place name » public domain status public domain mark , no copyright - us , recently added items great britain ministry of information: daily press notices and bulletins: - - great britain ministry of information: daily press notices and bulletins: - - great britain ministry of information: daily press notices and bulletins: - - great britain ministry of information: daily press notices and bulletins: - - great britain ministry of information: daily press notices and bulletins: - - great britain ministry of information: daily press notices and bulletins: - - great britain ministry of information: daily press notices and bulletins: - - great britain ministry of information: daily press notices and bulletins: - - great britain ministry of information: daily press notices and bulletins: - - great britain ministry of information: daily press notices and bulletins: - - great britain ministry of information: daily press notices and bulletins: - - great britain ministry of information: daily press notices and bulletins: - - great britain ministry of information: daily press notices and bulletins: - - great britain ministry of information: daily press notices and bulletins: - - great britain ministry of information: daily press notices and bulletins: - - utl chat none github - actions/runner: the runner for github actions skip to content sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories → security → team enterprise explore explore github → learn & contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → nonprofit → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this organization all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up {{ message }} actions / runner watch star . k fork the runner for github actions 🚀 github.com/features/actions mit license . k stars forks star watch code issues pull requests discussions actions projects security insights more code issues pull requests discussions actions projects security insights main branches tags go to file code clone https github cli use git or checkout with svn using the web url. work fast with our official cli. learn more. open with github desktop download zip launching github desktop if nothing happens, download github desktop and try again. go back launching github desktop if nothing happens, download github desktop and try again. go back launching xcode if nothing happens, download xcode and try again. go back launching visual studio if nothing happens, download the github extension for visual studio and try again. go back latest commit hollowman fix typos (# ) … a cb feb , fix typos (# ) accidentially -> accidentally neglible -> negligible a cb git stats commits files permalink failed to load latest commit information. type name latest commit message commit time .github generate sha's for released packages and include them in package notes ( jan , docs fix typos (# ) feb , scripts feat(scripts): add labels in the script that register runner (# ) dec , src display github_token permissions (# ) feb , .gitattributes github actions runner oct , .gitignore change hashfiles() expression to use @actions/glob. (# ) jan , codeowners add runtime team as codeowners (# ) dec , license update license (# ) dec , readme.md add e e workflow badge dec , releasenote.md generate sha's for released packages and include them in package notes ( jan , releaseversion prepare . . runner release. may , view code readme.md github actions runner the runner is the application that runs a job from a github actions workflow. it is used by github actions in the hosted virtual environments, or you can self-host the runner in your own environment. get started for more information about installing and using self-hosted runners, see adding self-hosted runners and using self-hosted runners in a workflow runner releases: pre-reqs | download pre-reqs | download pre-reqs | download contribute we accept contributions in the form of issues and pull requests. read more here before contributing. about the runner for github actions 🚀 github.com/features/actions resources readme license mit license releases v . . latest jan , + releases packages no packages published contributors + contributors languages c# . % javascript . % shell . % other . % © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. public_library_map/minify-map-json.yml at main · hughrun/public_library_map · github skip to content sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories → security → team enterprise explore explore github → learn & contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → nonprofit → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this user all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up {{ message }} hughrun / public_library_map watch star fork code issues pull requests actions security insights more code issues pull requests actions security insights permalink main public_library_map/.github/workflows/minify-map-json.yml go to file go to file t go to line l copy path cannot retrieve contributors at this time lines ( sloc) . kb raw blame name: minify map json on: push: branches: - master - main paths: - 'boundaries.geo.json' workflow_dispatch: jobs: processjson: if: "!contains(github.event.head_commit.message, 'from hughrun/geo-to-topo')" runs-on: ubuntu-latest steps: - uses: actions/checkout@v with: fetch-depth: - name: update geojson boundaries run: | sudo npm install -g mapshaper mapshaper boundaries.geo.json snap -clean -o force precision= . format=geojson boundaries.geo.json sudo npm install -g topojson-server geo topo boundaries.geo.json -q > website/data/boundaries.topo.json python ./.github/scripts/merge_csv_to_topojson.py - name: create pull request uses: peter-evans/create-pull-request@v with: author: library map bot commit-message: update topojson boundaries title: update topojson boundaries body: 'clean & minify geojson; process to topojson; merge library info csv to topo.' branch: geo-to-topo labels: auto update,data copy lines copy permalink view git blame reference in new issue go © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. meta interchange meta interchange libraries, computing, metadata, and more trading for images let&# ;s search a koha catalog for something that isn&# ;t at all controversial: what you search for in a library catalog ought to be only between... entering a very brief post to start the new year. i&# ;m not inclined to make elaborate resolutions for the new year other than being very firm... data cleanup as a force for evil a quotidian concern of anybody responsible for a database is the messy data it contains. see a record about a pedro gonzÃ¡lez? bah, the assumption of... on being wrong, wrong, wrong yesterday i gave a lightning talk at the evergreen conference on being wrong. appropriately, i started out the talk on the wrong foot. i intended... fostering a habit of nondisclosure it almost doesn&# ;t need to be said that old-fashioned library checkout cards were terrible for patron privacy. want to know who had checked out a... scaling the annual code lib conference one of the beautiful things about code lib qua banner is that it can be easily taken up by anyway without asking permission. if i wanted... amelia, - last year, i wrote about the blossoming of the mellie-cat, and closed with this line: &# ;sixteen years is not long enough to get to know... mashcat at ala annual + shared notes i&# ;m leaving for chicago tomorrow to attend ala annual (and to eat some real pizza), and while going over the schedule i found some... what makes an anti-librarian? assuming the order gets made and shipped in time (update - - : it did), i&# ;ll be arriving in chicago for ala annual carrying a few tens... imls support for free and open source software the institute of museum and library services is the u.s. government&# ;s primary vehicle for direct federal support of libraries, museums, and archives across the entire... github - mbloch/mapshaper: tools for editing shapefile, geojson, topojson and csv files skip to content sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories → security → team enterprise explore explore github → learn & contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → nonprofit → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this user all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up {{ message }} mbloch / mapshaper watch star . k fork tools for editing shapefile, geojson, topojson and csv files mapshaper.org view license . k stars forks star watch code issues pull requests actions projects wiki security insights more code issues pull requests actions projects wiki security insights master branches tags go to file code clone https github cli use git or checkout with svn using the web url. work fast with our official cli. learn more. open with github desktop download zip launching github desktop if nothing happens, download github desktop and try again. go back launching github desktop if nothing happens, download github desktop and try again. go back launching xcode if nothing happens, download xcode and try again. go back launching visual studio if nothing happens, download the github extension for visual studio and try again. go back latest commit mbloch v . . … a jan , v . . a git stats , commits files permalink failed to load latest commit information. type name latest commit message commit time bin fix saving to subdirectories with mapshaper-gui jul , src v . . jan , test v . . jan , www add -classify and -colors commands jan , .gitignore refactored codebase to use es modules apr , .travis.yml fix travis config by building project before running tests apr , changelog.md v . . jan , license update readme, add mpl . license jul , readme.md add user-contributed resourced to readme dec , package-lock.json v . . jan , package.json v . . jan , rollup.config.js really fix rollup config apr , view code readme.md mapshaper introduction mapshaper is software for editing shapefile, geojson, topojson, csv and several other data formats, written in javascript. mapshaper supports essential map making tasks like simplifying shapes, editing attribute data, clipping, erasing, dissolving, filtering and more. see the project wiki for documentation on how to use mapshaper. to suggest improvements, add an issue. command line tools mapshaper includes several command line programs, which can be run under mac os x, linux and windows. mapshaper runs mapshaper commands. mapshaper-xl works the same as mapshaper, but runs with more ram to support larger files. mapshaper-gui runs the mapshaper web interface locally. the project wiki has an introduction to using the command line tool that includes many simple examples. for a detailed reference, see the command reference. interactive web interface visit the public website at www.mapshaper.org or use the web ui locally via the mapshaper-gui script. all processing is done in the browser, so your data stays private, even when using the public website. the web ui works in recent desktop versions of chrome, firefox, safari and internet explorer. safari before v . and ie before v are not supported. user-contributed resources rmapshaper is an r package written by andy teucher that gives r users access to many of mapshaper's editing commands. here are resources for using mapshaper with docker, provided by christian weiss. you can find a number of mapshaper tutorials online, including a two part guide to command line cartography by dylan moriarty and this introduction by jack dougherty. large file support web interface firefox is able to load shapefiles and geojson files larger than gb. chrome has improved in recent versions, but is still prone to out-of-memory errors when importing files larger than several hundred megabytes. command line interface there are hard limits for reading and writing most file types. the maximum output size of a single file of any type is gb. some file types (geojson, csv, .shp) are read incrementally, so much larger files can be imported. when working with very large files, mapshaper may become unresponsive or crash with the message "javascript heap out of memory." one option is to run mapshaper-xl, which allocates more memory than the standard mapshaper program ( gb by default). starting with version . . , you can specify the amount of memory to allocate like this: mapshaper-xl gb [commands]. another solution is to run node directly with the --max-old-space-size option. the following example (mac or linux) allocates gb of memory: $ node --max-old-space-size= `which mapshaper` installation mapshaper requires node.js. with node installed, you can install the latest release version of mapshaper using npm. install with the "-g" flag to make the executable scripts available systemwide. npm install -g mapshaper to install and run the latest development code from github: git clone git@github.com:mbloch/mapshaper.git cd mapshaper npm install # install dependencies npm run build # bundle source code files npm link # (optional) add global symlinks so scripts are available systemwide building and testing from the project directory, run npm run build to build both the cli and web ui modules. run npm test to run mapshaper's tests. license this software is licensed under mpl . . according to mozilla's faq, "the mpl's ‘file-level’ copyleft is designed to encourage contributors to share modifications they make to your code, while still allowing them to combine your code with code under other licenses (open or proprietary) with minimal restrictions." acknowledgements my colleagues at the new york times, for countless suggestions, bug reports and general helpfulness. mark harrower, for collaborating on the original "mapshaper" program at the university of wisconsin–madison. about tools for editing shapefile, geojson, topojson and csv files mapshaper.org topics svg topojson csv geojson gis shapefile resources readme license view license releases v . . latest jan , + releases packages no packages published used by + contributors languages javascript . % other . % © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. trump urls in wayback but missing from trump twitter archive - check playback in wayback if you can. - google sheets javascript isn't enabled in your browser, so this file can't be opened. enable and reload. some excel features can't be displayed in google sheets and will be dropped if you make changes view details trump urls in wayback but missing from trump twitter archive - check playback in wayback if you can. share sign in the version of the browser you are using is no longer supported. please upgrade to a supported browser.dismiss file edit view insert format data tools form add-ons help accessibility unsaved changes to drive see new changes $ % conditional formatting conditional formatting sum average count max min learn more accessibility still loadingâ€¦ a b c d e original tweet url trump tweet archive wayback machine urls (click on these) does it playback in wayback? is retweet? https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ no no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes yes https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ no no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ no no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ no no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ no no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ no no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ no no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ no no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes yes https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ no no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ no no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ no no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ no no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ no no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ no no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ no no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ http://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /http://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ no (captured twitter error message) https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ no (captured twitter error message) https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ https://twitter.com/realdonaldtrump/status/ https://www.thetrumparchive.com/tweets/ https://web.archive.org/web/ /https://twitter.com/realdonaldtrump/status/ yes no loading... quotes are not sourced from all markets and may be delayed up to minutes. information is provided 'as is' and solely for informational purposes, not for trading purposes or advice.disclaimer trump-tweets-missing-from-archive t a browser error has occurred. please press ctrl-f to refresh the page and try again. a browser error has occurred. please hold the shift key and click the refresh button to try again. data unbound : helping organizations access and share data effectively. special focus on web apis for data integration. data unbound helping organizations access and share data effectively. special focus on web apis for data integration. skip to content about some of what i missed from the cmd-d automation conference the cmd-d|masters of automation one-day conference in early august would have been right up my alley: it’ll be a full day of exploring the current state of automation technology on both apple platforms, sharing ideas and concepts, and showing what’s possible—all with the goal of inspiring and furthering development of your own automation projects. fortunately, those of us who missed it can still get a meaty summary of the meeting by listening to the podcast segment upgrade # : masters of automation – relay fm. i've been keen on automation for a long time now and was delighted to hear the panelists express their own enthusiasm for customizing their macs, iphones, or ipads to make repetitive tasks much easier and less time-consuming. noteworthy take-aways from the podcast include: something that i hear and believe but have yet to experience in person: non-programmers can make use of automation through applications such as automator — for macos — and workflow for ios. also mentioned often as tools that are accessible to non-geeks: hazel and alfred – productivity app for mac os x. automation can make the lives of computer users easier but it's not immediately obvious to many people exactly how. to make a lot of headway in automating your workflow, you need a problem that you are motivated to solve. many people use applescript by borrowing from others, just like how many learn html and css from copying, pasting, and adapting source on the web. once you get a taste for automation, you will seek out applications that are scriptable and avoid those that are not. my question is how to make it easier for developers to make their applications scriptable without incurring onerous development or maintenance costs? e-book production is an interesting use case for automation. people have built businesses around scripting photoshop [is there really a large enough market?] omnigroup's automation model is well worth studying and using. i hope there will be a conference next year to continue fostering this community of automation enthusists and professionals. raymond yee automation macos comments ( ) permalink fine-tuning a python wrapper for the hypothes.is web api and other #ianno followup in anticipation of #ianno hack day, i wrote about my plans for the event, one of which was to revisit my own python wrapper for the nascent hypothes.is web api. instead of spending much time on my own wrapper, i spent most of the day working with jon udell's wrapper for the api. i've been working on my own revisions of the library but haven't yet incorporated jon's latest changes. one nice little piece of the puzzle is that i learned how to introduce retries and exponential backoff into the library, thanks to a hint from nick stenning and a nice answer on stackoverflow . other matters in addition to the python wrapper, there are other pieces of follow-up for me. i hope to write more extensively on those matters down the road but simply note those topics for the moment. videos from the conference i might start by watching videos from #ianno conference: i annotate – youtube. because i didn't attend the conference per se, i might glean insight into two particular topics of interest to me (the role of page owner in annotations and the intermingling of annotations in ebooks.) an extension for embedding selectors in the url i will study and try treora/precise-links: browser extension to support web annotation selectors in uris. i've noticed that the same annotation is shown in two related forms: https://hyp.is/zj dyi teeetmxvupjlhsw/blog.dataunbound.com/ / / /revisiting-hypothes-is-at-i-annotate- / https://blog.dataunbound.com/ / / /revisiting-hypothes-is-at-i-annotate- /#annotations:zj dyi teeetmxvupjlhsw does the precise-links extension let me write the selectors into the url? raymond yee annotation comments ( ) permalink revisiting hypothes.is at i annotate i'm looking forward to hacking on web and epub annotation at the #ianno hack day. i won't be at the i annotate conference per se but will be curious to see what comes out of the annual conference. i continue to have high hopes for digital annotations, both on the web and in non-web digital contexts. i have used hypothesis on and off since oct . my experiences so far: i like the ability to highlight and comment on very granular sections of articles for comment, something the hypothes.is annotation tool makes easy to do. i appreciate being able to share annotation/highlight with others (on twitter or facebook), though i'm pretty sure most people who bother to click on the links might wonder "what's this" when they click on the link. a small user request: hypothes.is should allow a user to better customize the facebook preview image for the annotation. i've enjoyed using hypothes.is for code review on top of github. (exactly how hypothes.is complements the extensive code-commenting functionality in github might be worth a future blog post.) my plans for hack day python wrapper for hypothes.is this week, i plan to revisit rdhyee/hypothesisapi: a python wrapper for the nascent hypothes.is web api to update or abandon it in favor of new developments. (for example, i should look at kshaffer/pypothesis: python scripts for interacting with the hypothes.is api.) epubs + annotations i want to figure out the state of art for epubs and annotations. i'm happy to see the announcement of a partnership to bring open annotation to ebooks from march . i'd definitely like to figure out how to annotate epubs (e.g., oral literature in africa (at unglue.it) or moby dick). the best approach is probably for me to wait until summer at which time we'll see the fruits of the partnership: together, our goal is to complete a working integration of hypothesis with both epub frameworks by summer . nyu plans to deploy the readiumjs implementation in the nyu press enhanced networked monographs site as a first use case. based on lessons learned in the nyu deployment, we expect to see wider integration of annotation capabilities in ebooks as epub uptake continues to grow. in the meantime, i can catch up on the current state of futurepress/epub.js: enhanced ebooks in the browser., grok epub cfi updates, and relearn how to parse epubs using python (e.g., rdhyee/epub_avant_garde: an experiment to apply ideas from https://github.com/sandersk/ebook_avant_garde to arbitrary epubs). role of page owners i plan to check in on what's going on with efforts at hypothes.is to involve owners in page annotations: in the past months we launched a small research initiative to gather different points of view about website publishers and authors consent to annotation. our goal was to identify different paths forward taking into account the perspectives of publishers, engineers, developers and people working on abuse and harassment issues. we have published a first summary of our discussion on our blog post about involving page owners in annotation. i was reminded of these efforts after reading that audrey watters had blocked annotation services like hypothes.is and genius from her domains: un-annotated episode : marginalia in the spirit of communal conversation, i threw in my two cents: have there been any serious exploration of easy opt-out mechanisms for domain owners? something like robots.txt for annotation tools? raymond yee annotation comments ( ) permalink my thoughts about fargo.io using fargo.io raymond yee uncategorized comments ( ) permalink organizing your life with python: a submission for pycon ? i have penciled into my calendar a trip to montreal to attend pycon . in my moments of suboptimal planning, i wrote an overly ambitious abstract for a talk or poster session i was planning to submit. as i sat down this morning to meet the deadline for submitting a proposal for a poster session (nov ), i once again encountered the ominous (but for me, definitive) admonition: avoid presenting a proposal for code that is far from completion. the program committee is very skeptical of "conference-driven development". it's true: my efforts to organize my life with python are in the early stages. i hope that i'll be able to write something like the following for pycon . organizing your life with python david allen's getting things done (gtd) system is a popular system for personal productivity. although gtd can be implemented without any computer technology, i have pursued two different digital implementations, including my current implementation using evernote, the popular note-taking program. this talk explores using python in conjunction with the evernote api to implement gtd on top of evernote. i have found that a major practical hinderance for using gtd is that it way too easy to commit to too many projects. i will discuss how to combine evernote, python, gtd with concepts from personal kanban to solve this problem. addendum: whoops…i find it embarrassing that i already quoted my abstract in a previous blog post in september that i had forgotten about. oh well. where's my fully functioning organization system when i need it! tagged pycon, python raymond yee evernote gtd comments ( ) permalink current status of data unbound llc in pennsylvania i'm currently in the process of closing down data unbound llc in pennsylvania. i submitted the paperwork to dissolve the legal entity in april and have been amazed to learn that it may take up to a year to get the final approval done. in the meantime, as i establishing a similar california legal entity, i will certainly continue to write on this blog about apis, mashups, and open data. raymond yee data unbound llc comments ( ) permalink must get cracking on organizing your life with python talk and tutorial proposals for pycon are due tomorrow ( / ) . i was considering submitting a proposal until i took the heart the appropriate admonition against "conference-driven" development of the program committee. i will nonetheless use the oct and nov deadlines for lightning talks and proposals respectively to judge whether to submit a refinement of the following proposal idea: organizing your life with python david allen's getting things done (gtd) system is a popular system for personal productivity. although gtd can be implemented without any computer technology, i have pursued two different digital implementations, including my current implementation using evernote, the popular note-taking program. this talk explores using python in conjunction with the evernote api to implement gtd on top of evernote. i have found that a major practical hinderance for using gtd is that it way too easy to commit to too many projects. i will discuss how to combine evernote, python, gtd with concepts from personal kanban to solve this problem. raymond yee getting things done python comments ( ) permalink embedding github gists in wordpress as i gear up i to write more about programming, i have installed the embed github gist plugin. so by writing [gist id= ] in the text of this post, i can embed https://gist.github.com/rdhyee/ into the post to get: from itertools import islice def triangular(): n = i = while true: yield n i += n += i # for i, n in enumerate(islice(triangular(), )): print i+ , n tagged gist, github raymond yee wordpress comments ( ) permalink working with open data i'm very excited to be teaching a new course working with open data at the uc berkeley school of information in the spring semester: open data — data that is free for use, reuse, and redistribution — is an intellectual treasure-trove that has given rise to many unexpected and often fruitful applications. in this course, students will ) learn how to access, visualize, clean, interpret, and share data, especially open data, using python, python-based libraries, and supplementary computational frameworks and ) understand the theoretical underpinnings of open data and their connections to implementations in the physical and life sciences, government, social sciences, and journalism. raymond yee uncategorized comments ( ) permalink a mundane task: updating a config file to retain old settings i want to have a hand in creating an excellent personal information manager (pim) that can be a worthy successor to ecco pro. so far, running eccoext (a clever and expansive hack of ecco pro) has been a eminently practical solution. you can download the most recent version of this actively developed extension from the files section of the ecco_pro yahoo! group. i would do so regularly but one of the painful problems with unpacking (using unrar) the new files is that there wasn't an updater that would retain the configuration options of the existing setup. so a mundane but happy-making programming task of this afternoon was to write a python script to do exact that function, making use of the builtin configparser library. """ compare eccoext.ini files my goal is to edit the new file so that any overlapping values take on the current value """ current_file_path = "/private/tmp/ /c/program files/ecco/eccoext.ini" new_file_path = "/private/tmp/ /c/utils/eccoext.ini" updated_file = "/private/tmp/ /c/utils/updated_eccoext.ini" # extract the key value pairs in both files to compare the two # http://docs.python.org/library/configparser.html import configparser def extract_values(fname): # generate a parsed configuration object, set of (section, options) config = configparser.safeconfigparser() options_set = set() config.read(fname) sections = config.sections() for section in sections: options = config.options(section) for option in options: #value = config.get(section,option) options_set.add((section,option)) return (config, options_set) # process current file and new file (current_config, current_options) = extract_values(current_file_path) (new_config, new_options) = extract_values(new_file_path) # what are the overlapping options overlapping_options = current_options & new_options # figure out which of the overlapping options are the values different for (section,option) in overlapping_options: current_value = current_config.get(section,option) new_value = new_config.get(section,option) if current_value != new_value: print section, option, current_value, new_value new_config.set(section,option,current_value) # write the updated config file with open(updated_file, 'wb') as configfile: new_config.write(configfile) raymond yee ecco pro python comments ( ) permalink « older posts pages about categories amazon annotation announcments apis architecture art history automation bibliographics bioinformatics bplan chickenfoot citizendium collaboration consulting copyright creative commons data mining data unbound llc digital scholarship ecco pro education evernote firefox flickr freebase getting things done google government gtd hardware hci higher education humanities imaging ischool journalism libraries macos mashups meta mith api workshop mixing and remixing information notelets oclc open access open data openid personal information management personal news politics processing programming tip prototype publishing python recovery.gov tracking repositories rest screen scraping screencast services soap training tutorial uc berkeley uncategorized web hosting web services web weblogging wikipedia wordpress writing zotero tags api art history books chickenfoot codepad coins creative commons data hosting data portability educause exif firefox flickr freebase jcdl jcdl kses library of congress mashups mashup symfony django metadata news nytimes amazonec amazons omb openid openlibrary openoffice.org photos politics project bamboo python pywin recovery.gov tracking screencast stimulus sychronization video webcast wikipedia windows xp wmi wordpress workshops xml in libraries zotero blogroll information services and technology, uc berkeley uc berkeley rss feeds all posts all comments meta log in blog search © | thanks, wordpress | barthelme theme by scott allan wallick | standards compliant xhtml & css | rss posts & comments websolr - add-ons - heroku elements skip navigation show nav heroku products heroku platform heroku dx heroku flow continuous delivery continuous integration heroku opex heroku runtime heroku dynos heroku data services heroku postgres heroku redis apache kafka on heroku heroku enterprise heroku private spaces heroku connect heroku shield heroku teams marketplace add-ons buttons buildpacks about pricing documentation support more resources what is heroku? help customers careers events podcasts compliance center heroku is for developers ctos team collaboration startups enterprises agencies students see more languages node ruby java php python go scala clojure see more latest news from the heroku blog heroku blog find out what's new with heroku on our blog. more news view all blog posts search: log in or sign up add-ons buttons buildpacks websolr index and search with apache solr, the most popular open source search engine. starting at $ /mo. lightning-fast responses studies show that sites with fast response times are perceived to be more interesting and attractive. so don’t make your customers wait for that sql like to run! solr is specially optimized to provide lightning-fast responses to your queries. smart matching your users have high expectations for the quality of their search results. introduce a bit of intelligence into your full-text search with built-in text analysis tools. perform simple analysis to match case-insensitive or substring queries, substitute terms and expand abbreviations for known synonyms, or perform sophisticated linguistic analysis to match different pluralizations and parts of speech. relevant results solr comes with smart result ranking out of the box, which uses statistical analysis of the relative frequency and popularity of the terms within your content to determine the order of your results. and if that’s not enough, you can provide custom scoring functions to boost term matches however you see fit. that way the best results always show first on the list. experts on call websolr has been serving and supporting production full-text search since , as one of the very first heroku add-ons. we’re indexing billions of documents and serving thousands of requests per second for thousands of applications. our customers’ sites include everything from side projects and small blogs, to multi-million-dollar ecommerce applications and well-known at-scale social media sites. we’re here to make sure your search engine just works™, so you can get back to spending your valuable time on building your business. high availability for your peace of mind to meet our own high standards for availability, our indexes are built with a replicated hot spare in another availability zone. our indexes can tolerate an instance or availability zone outage without losing a single search request, and we can provision indexes in multiple regions for even greater redundancy. platform agnostic get started integrating search into your site with one of the excellent open source solr clients available for your application’s language and platform. we recommend sunspot for ruby on rails, haystack for django, solrj for play! and other java applications, among many others. do you have a question about which solr client you should use? consult our list of solr clients or send us a tweet at @websolr and let us know what you’re using to build your application. support sla basic support - free - / monitoring, -hour response sla ( am- pm pst, monday-friday, non-us holidays) through email premium support - / -hour sla, private slack channel, pager, senior level devops engineer, optional -hour monthly check-in enterprise support - / -minute sla, private slack channel, pager, senior level devops engineer, optional -hour weekly support calls. region availability the available application locations for this add-on are shown below, and depend on whether the application is deployed to a common runtime region or private space. learn more common runtime private spaces region available united states available europe available region available installable in space virginia oregon frankfurt tokyo sydney dublin plans & pricing staging $ /mo standard small $ /mo standard medium $ /mo standard large $ /mo business small $ /mo business medium $ /mo business large $ /mo need a larger plan? let our customer success team help! learn more. documents , replication daily requests , basic ticket support index limit / priority support / enterprise support storage capacity mb concurrent requests dedicated load balancer production grade documents , , replication daily requests , basic ticket support index limit / priority support / enterprise support storage capacity mb concurrent requests dedicated load balancer production grade documents , , replication daily requests , basic ticket support index limit / priority support / enterprise support storage capacity . gb concurrent requests dedicated load balancer production grade documents , , replication daily requests , basic ticket support index limit / priority support / enterprise support storage capacity . gb concurrent requests dedicated load balancer production grade documents unlimited replication daily requests , basic ticket support index limit / priority support / enterprise support storage capacity . gb concurrent requests unlimited dedicated load balancer production grade documents unlimited replication daily requests , , basic ticket support index limit / priority support / enterprise support storage capacity . gb concurrent requests unlimited dedicated load balancer production grade documents unlimited replication daily requests , , basic ticket support index limit / priority support / enterprise support storage capacity . gb concurrent requests unlimited dedicated load balancer production grade install websolr heroku addons:create websolr to provision, copy the snippet into your cli or use the install button above. websolr documentation install the add-on choosing a solr client sunspot for ruby on rails haystack for django using a different solr client configuring your index questions? install websolr quick links add-on details region availability plans & pricing documentation heroku elements terms of use (default) shareable details not shareable across apps multiple installs/app add-on category search language support clojure go java node php python ruby scala add-on provider admin become an add-on provider products heroku platform heroku data services heroku postgres heroku redis apache kafka on heroku heroku enterprise heroku private spaces heroku connect heroku shield heroku teams elements marketplace pricing resources documentation compliance center training & education blog podcasts get started about about us what is heroku heroku & salesforce our customers careers partners elements marketplace support help center status premium support contact us subscribe to our monthly newsletter your email address: rss heroku blog heroku news blog heroku engineering blog dev center articles dev center changelog heroku status podcasts twitter heroku dev center articles dev center changelog heroku status facebook instagram github linkedin youtube heroku is a company heroku.com terms of service privacy cookies © salesforce.com rails autoscale - add-ons - heroku elements skip navigation show nav heroku products heroku platform heroku dx heroku flow continuous delivery continuous integration heroku opex heroku runtime heroku dynos heroku data services heroku postgres heroku redis apache kafka on heroku heroku enterprise heroku private spaces heroku connect heroku shield heroku teams marketplace add-ons buttons buildpacks about pricing documentation support more resources what is heroku? help customers careers events podcasts compliance center heroku is for developers ctos team collaboration startups enterprises agencies students see more languages node ruby java php python go scala clojure see more latest news from the heroku blog heroku blog find out what's new with heroku on our blog. more news view all blog posts search: log in or sign up add-ons buttons buildpacks rails autoscale simple, reliable queue-based autoscaling for your rails app. starting at $ /mo. a smarter way to autoscale rails autoscale is the only autoscaler based on request queue time, not total response time. some endpoints naturally take longer than others, and that shouldn’t trigger scaling. rails autoscale watches for real capacity issues to scale your app reliably and consistently. “if i was the king of the world, i would make it illegal to horizontally scale based on execution time. scale based on queue depths, people!” – nate berkopec, the complete guide to rails performance sleep well, without stress never worry about slow-downs or traffic spikes again. stop guessing about the “right” number of dynos for your app. let rails autoscale do the dirty work so you can focus on building, not firefighting. cut your heroku bill in half most apps are overscaled so they can handle peak traffic. rails autoscale easily pays for itself by scaling down your app during quieter periods, some apps earn back half of what they were previously paying for heroku. try it and be amazed at how many dynos you don’t need. “rails autoscale has allowed us to reduce our dyno usage for our primary application down to less than half of what we were using before, which is amazing (and allows us to not have to upgrade our heroku plan). thanks for making this, it really is a great little service.” – mark urich impossibly simple rails autoscale embraces the / rule and provides sensible defaults that work for % of apps. no sweat if you’re in the remaining %. just a few tweaks is all it takes to get your app scaling smoothly. “rails autoscale saved me big time yesterday!! i’m new to this and i’m learning on the fly, but you made it very easy.” – sam wood worker dynos included sidekiq, resque, delayed job, and que are all supported. even if you split your worker queues across multiple processes, it’s no problem. configure and autoscale each of them independently. worker autoscaling is part of the package, no extra charge. pricing that makes sense the more dynos you use, the more money you save by autoscaling. rails autoscale is priced accordingly. each plan has access to every rails autoscale feature. the only difference between plans is the number and type of dynos supported. no api key required giving out your heroku api key is no different than giving out your heroku password. rails autoscale will never ask for it. tight integration as a heroku add-on means rails autoscale can only access the necessary endpoints of the heroku api on behalf of your app, nothing more. extensive documentation rails autoscale was built for heroku, so the entire documentation site is designed to help you and your heroku application. if you have a question, there’s a good chance i’ve answered it for you on the common questions page. a partner to support you i’m adam, the founder of rails autoscale and a rails developer like you. i personally answer every support request, and it makes my day to help developers run their heroku apps with confidence. got a question? send me an email! “i’ve used most of them. adam’s is the best. i highly recommend it.” – andrew culver, bullet train region availability the available application locations for this add-on are shown below, and depend on whether the application is deployed to a common runtime region or private space. learn more common runtime private spaces region available united states available europe available region available installable in space virginia available oregon available frankfurt available tokyo available sydney available dublin available plans & pricing trial free bronze - standard $ /mo silver - standard $ /mo bronze - performance $ /mo silver - performance $ /mo gold - standard $ /mo platinum - standard $ /mo gold - performance $ /mo platinum - performance $ /mo need a larger plan? let our customer success team help! learn more. trial length days autoscale limit (standard dynos) unlimited autoscale limit (performance/private/shield dynos) unlimited web autoscaling worker autoscaling (sidekiq, resque, dj, que) chat notifications configurable upscale threshold configurable downscale threshold one-on-one customer support trial length n/a autoscale limit (standard dynos) dynos autoscale limit (performance/private/shield dynos) n/a web autoscaling worker autoscaling (sidekiq, resque, dj, que) chat notifications configurable upscale threshold configurable downscale threshold one-on-one customer support trial length n/a autoscale limit (standard dynos) dynos autoscale limit (performance/private/shield dynos) n/a web autoscaling worker autoscaling (sidekiq, resque, dj, que) chat notifications configurable upscale threshold configurable downscale threshold one-on-one customer support trial length n/a autoscale limit (standard dynos) dynos autoscale limit (performance/private/shield dynos) dynos web autoscaling worker autoscaling (sidekiq, resque, dj, que) chat notifications configurable upscale threshold configurable downscale threshold one-on-one customer support trial length n/a autoscale limit (standard dynos) dynos autoscale limit (performance/private/shield dynos) dynos web autoscaling worker autoscaling (sidekiq, resque, dj, que) chat notifications configurable upscale threshold configurable downscale threshold one-on-one customer support trial length n/a autoscale limit (standard dynos) dynos autoscale limit (performance/private/shield dynos) n/a web autoscaling worker autoscaling (sidekiq, resque, dj, que) chat notifications configurable upscale threshold configurable downscale threshold one-on-one customer support trial length n/a autoscale limit (standard dynos) unlimited autoscale limit (performance/private/shield dynos) n/a web autoscaling worker autoscaling (sidekiq, resque, dj, que) chat notifications configurable upscale threshold configurable downscale threshold one-on-one customer support trial length n/a autoscale limit (standard dynos) dynos autoscale limit (performance/private/shield dynos) dynos web autoscaling worker autoscaling (sidekiq, resque, dj, que) chat notifications configurable upscale threshold configurable downscale threshold one-on-one customer support trial length n/a autoscale limit (standard dynos) unlimited autoscale limit (performance/private/shield dynos) unlimited web autoscaling worker autoscaling (sidekiq, resque, dj, que) chat notifications configurable upscale threshold configurable downscale threshold one-on-one customer support install rails autoscale heroku addons:create rails-autoscale to provision, copy the snippet into your cli or use the install button above. rails autoscale documentation full documentation how is this different from heroku’s own autoscaling? installation adjust your settings in the rails autoscale dashboard worker autoscaling troubleshooting migrating between plans removing the add-on support install rails autoscale quick links add-on details region availability plans & pricing documentation heroku elements terms of use (default) shareable details not shareable across apps single install/app add-on category dynos language support ruby add-on provider admin become an add-on provider products heroku platform heroku data services heroku postgres heroku redis apache kafka on heroku heroku enterprise heroku private spaces heroku connect heroku shield heroku teams elements marketplace pricing resources documentation compliance center training & education blog podcasts get started about about us what is heroku heroku & salesforce our customers careers partners elements marketplace support help center status premium support contact us subscribe to our monthly newsletter your email address: rss heroku blog heroku news blog heroku engineering blog dev center articles dev center changelog heroku status podcasts twitter heroku dev center articles dev center changelog heroku status facebook instagram github linkedin youtube heroku is a company heroku.com terms of service privacy cookies © salesforce.com utah state archives | fromthepage find a projectsign up to transcribesign in utah state archives the state of utah vs. joe hill born in in gavle, sweden, joseph hillstrom (also known as joe hill) immigrated to the united states in , and worked in a variety of jobs including laborer, miner, lumberman, and longshoreman. the often brutal working conditions he witnessed led him to join the industrial workers of the... governor spry ( - ) joseph hillstrom case records as chief executive officer of the state, governor spry had the authority to commute the sentences of those convicted of crimes. the murder conviction and execution of joseph hillstrom (joe hill) became one of the most controversial criminal cases in utah history; it generated national and... governor spry ( - ) joseph hillstrom correspondence as chief executive officer of the state, the governor has the authority to commute the sentences of those convicted of crimes. the conviction and execution of joseph hillstrom (joe hill) became one of the most controversial criminal cases in utah history; it generated national and international... brigham young probate case file the first governor of the utah territory and president of the church of jesus christ of latter-day saints (mormon) died august , . at the time, the church practiced an arrangement wherein the president held property and real estate as "trustee-in-trust." this allowed young to direct and... project by utah state archives john d. lee case file all transcription of the john d. lee case files are done. please don't join expecting any more works to be added.explore the subjects tab to see all the names and places getting linked from within documents. case files document criminal cases as they proceed through the court system and... project by utah state archives the state of utah vs. joe hill born in in gavle, sweden, joseph hillstrom (also known as joe hill) immigrated to the united states in , and worked in a variety of jobs including laborer, miner, lumberman, and longshoreman. the often brutal working conditions he witnessed led him to join the industrial workers of the... start transcribingproject by utah state archives governor spry ( - ) joseph hillstrom case records governor spry ( - ) joseph hillstrom correspondence governor spry ( - ) joseph hillstrom petitions utah women suffrage various documents and records from the utah state archives and records service related to the suffrage of women in the utah territory and state of utah. see also an online exhibit for "suffrage and stories of utah women" and blog posts featuring women's stories throughout . project by utah state archives bill file constitutional convention bill file bill files recent activity about hours agodknight edited page page in the state of utah vs. joe hill collection about hours agodknight marked page page as needing review in the state of utah vs. joe hill collection about hours agodknight edited page page in the state of utah vs. joe hill collection about hours agodknight marked page page as needing review in the state of utah vs. joe hill collection about hours agodknight marked page page as needing review in the state of utah vs. joe hill collection about hours agodknight marked page page as needing review in the state of utah vs. joe hill collection about hours agocwynkoop edited page page in the state of utah vs. joe hill collection about hours agocwynkoop marked page page as needing review in the state of utah vs. joe hill collection about hours agocwynkoop edited page page in the state of utah vs. joe hill collection about hours agocwynkoop marked page page as needing review in the state of utah vs. joe hill collection show more documentationblogaboutterms & conditionsprivacy policycontact us © fromthepage. all rights reserved. gitter — where developers come to talk. integrations apps open source sign in to start talkinglogin where communities thrive latest news we've joined element gitter is a chat and networking platform that helps to manage, grow and connect communities through messaging, content and discovery. create your own community explore other communities learn more about creating communities and creating rooms. by signing up you agree to our terms and conditions. free without limits enjoy free public and private communities with unlimited people, message history and integrations. simple to create simply create your community and start talking - no need to set up any invitation services. markdown and katex format your messages just like with your other favorite developer tools. out in the open complete history archives(indexed by search engines) and sharable permalinks or use sidecar to embed gitter directly in your own site. trusted by some of our communities create your own explore rooms explore gitterhq let’s talk about gitter! gitterhq let’s talk about gitter! gitterhq let’s talk about gitter! gitterhq let’s talk about gitter! designed for community collaboration gitter is designed to make community messaging, collaboration and discovery as smooth and simple as possible. you can easily create, organise and grow your communities, inviting others to join just in one click. we also provide integrations with github, trello, jenkins, travis ci, heroku, sentry, bitbucket, huboard, logentries, pagerduty & sprintly. we accept custom webhooks, have an open source repository for integrations as well as a flexible api. chris nicholson, deeplearning j: we have an engineering team and open source community distributed over several continents, but the conversation lives in one place - and that's gitter. valentin anastase, freezing moon: other communication solutions for the projects i’ve been involved in were far from ideal. switching to gitter was one of the best decisions i've made and there's no going back now. troy mcconaghy, bigchaindb developer: gitter is the main way people contact us to ask questions about bigchaindb. i guess it just feels more open, immediate and personal. krishna srinivas, minio: gitter is the irc of the post-irc era. it has helped us at minio build and connect with our community really well. sam saccone, marionette.js lead: gitter has helped to bring the marionette community together, turning it into a thriving collaborative environment that has been exceedingly beneficial to both our endusers and maintainers. david dias, lxjs conference organiser: gitter was an essential tool, not only for organising lxjs but to create a thriving community amongst our attendees before, during and even after the conference. doge, chief woofing officer: wow chat. much awesome. loved by our users gitter is nothing without the people using it day in and day out. we asked a few people what gitter means to them, this is what they had to say. if you love gitter, tweet us @gitchat and we'll update these quotes every month. get gitter now! gitter is available in all modern browsers as well as apps for desktops. mac windows linux linux the above mac installer is a signed version of . (backed by safari). for an unsigned . . installer (backed by nw.js), click here to download. mac users on . or lower, download here. we also have a sweet little irc bridge. the dedicated gitter android/ios apps are no longer recommended and are officially deprecated. for android/ios, we bridge all public rooms over to matrix, so you can use any matrix client like element (android) or element (ios) to access those rooms. our efforts are focused on the webapp which is the backbone of the mobile/desktop apps but mainly focused on the web experience. there are a number of bugs in these desktop/mobile clients and they spread the gitter team too thin for us to give them proper attention. the apps are open-source if you want to tackle something particularly annoying to you. (desktop, ios, android) copyright © new vector ltd support api blog twitter want this in ? wocintech stock - | women of color in tech stock images … | flickr explore recent photos trending events the commons flickr galleries world map camera finder flickr blog prints prints & wall art photo books get pro upload log in sign up log in explore trending events the commons flickr galleries flickr blog prints & wall art photo books get pro about jobs blog developers guidelines help help forum privacy terms cookies english ← → back to photostream wocintech chat by: wocintech chat wocintech stock - women of color in tech stock images take done , views faves comments taken on march , some rights reserved about jobs blog developers guidelines privacy terms help report abuse help forum english smugmug+flickr. privacy terms cookies smugmug+flickr. connecting people through photography. about jobs blog developers guidelines report abuse privacy terms help forum english privacy terms cookies help smugmug+flickr. connecting people through photography. bug # “encrypted home-directory is not unmounted on logou...” : bugs : gnome-session package : ubuntu log in / register ubuntu gnome-session package overview code bugs blueprints translations answers encrypted home-directory is not unmounted on logout bug # reported by jojo on - - this bug affects people affects status importance assigned to milestone gnome-session edit new undecided unassigned edit you need to log in to change this bug's status. affecting: gnome-session filed here by: pawel when: - - target distribution baltix boss juju charms collection elbuntu guadalinex guadalinex edu kiwi linux nubuntu pld linux tilix tuxlab ubuntu ubuntu linaro evaluation build ubuntu rtm package (find…) project (find…) status importance new undecided assigned to nobody me remote watch none, the status of the bug is updated manually. none, the status of the bug is updated manually. debian bug tracker # url: the information about this bug in launchpad is automatically pulled daily from the remote bug. comment on this change (optional) email me about changes to this bug report ecryptfs-utils (debian) edit new unknown debbugs # you need to log in to change this bug's status. affecting: ecryptfs-utils (debian) filed here by: klaus bielke when: - - target distribution baltix boss juju charms collection debian elbuntu guadalinex guadalinex edu kiwi linux nubuntu pld linux tilix tuxlab ubuntu ubuntu linaro evaluation build ubuntu rtm package (find…) project (find…) status importance new unknown assigned to unknown remote watch none, the status of the bug is updated manually. none, the status of the bug is updated manually. debian bug tracker # url: the information about this bug in launchpad is automatically pulled daily from the remote bug. this information was last pulled hours ago. comment on this change (optional) email me about changes to this bug report ecryptfs-utils (ubuntu) edit confirmed undecided unassigned edit you need to log in to change this bug's status. affecting: ecryptfs-utils (ubuntu) filed here by: wes when: - - confirmed: - - target distribution baltix boss juju charms collection elbuntu guadalinex guadalinex edu kiwi linux nubuntu pld linux tilix tuxlab ubuntu ubuntu linaro evaluation build ubuntu rtm package (find…) project (find…) status importance confirmed undecided assigned to nobody me comment on this change (optional) email me about changes to this bug report gdm (ubuntu) edit confirmed medium unassigned edit you need to log in to change this bug's status. affecting: gdm (ubuntu) filed here by: jojo when: - - confirmed: - - target distribution baltix boss juju charms collection elbuntu guadalinex guadalinex edu kiwi linux nubuntu pld linux tilix tuxlab ubuntu ubuntu linaro evaluation build ubuntu rtm package (find…) project (find…) status importance confirmed medium assigned to nobody me comment on this change (optional) email me about changes to this bug report gnome-session (ubuntu) edit confirmed medium unassigned edit you need to log in to change this bug's status. affecting: gnome-session (ubuntu) filed here by: daniel van vugt when: - - confirmed: - - target distribution baltix boss juju charms collection elbuntu guadalinex guadalinex edu kiwi linux nubuntu pld linux tilix tuxlab ubuntu ubuntu linaro evaluation build ubuntu rtm package (find…) project (find…) status importance confirmed medium assigned to nobody me comment on this change (optional) email me about changes to this bug report also affects project (?) also affects distribution/package nominate for series bug description current situation: if you log out from an user account with an encrypted home directory, it is not automatically unmounted and encrypted again. expected behaviour: if i log out from an user account with an encrypted home directory, id expect the homedir to be unmounted and encrypted again. stepts to reproduce: log into an account with encrypted home directory. (lets call it: user) log out again log into another account (which has sudo rights, lets call it: user ) and now enter the following into a terminal: user @ubuntu: sudo su user @ubuntu: ls -la /home/user you can see the files of the user reasons: this is a security issue, because as a user you can reasonable expect your data to be safe, if you log out. if you would simply log in as another user but keep your data accessable you would simply switch user, instead of loggin out. many users only suspend their laptop while carrying it with them. logging out and suspending the user expects to have at least the home directory encrypted. problemtype: bug distrorelease: ubuntu . package: gdm . . - ubuntu procversionsignature: ubuntu . . - . -generic . . uname: linux . . - -generic x _ apportversion: . . - ubuntu architecture: amd currentdesktop: ubuntu:gnome date: sun nov : : ecryptfsinuse: yes procenviron: term=xterm- color path=(custom, no user) xdg_runtime_dir= lang=de_de.utf- shell=/bin/bash sourcepackage: gdm upgradestatus: no upgrade log present (probably fresh install) tags: bionic amd apport-bug wayland-session edit tag help jojo (derdiedasjojo) wrote on - - : # dependencies.txt edit ( . kib, text/plain; charset="utf- ") journalerrors.txt edit ( . kib, text/plain; charset="utf- ") proccpuinfominimal.txt edit ( bytes, text/plain; charset="utf- ") daniel van vugt (vanvugt) on - - information type: public → public security daniel van vugt (vanvugt) on - - changed in gdm (ubuntu): importance: undecided → medium changed in gnome-session (ubuntu): importance: undecided → medium launchpad janitor (janitor) wrote on - - : # status changed to 'confirmed' because the bug affects multiple users. changed in gdm (ubuntu): status: new → confirmed changed in gnome-session (ubuntu): status: new → confirmed star man (starman) wrote on - - : # i confirm this issue is affecting me too. star man (starman) wrote on - - : # sorry, this are my system specs: ubuntu . lts amd gnome shell desktop ecryptfs in use kernel linux . . - -generic x _ ubuntu_neuer (pcspezialist-deactivatedaccount) wrote on - - : # ich bin ebenfalls davon betroffen. i confirm this issue is affecting me too. das ist mein system: this are my system: linux mint (beta) bit - based on ubuntu . cinnamon . . ecryptfs in use (home - directory) kernel . . - -generic x _ wes (wesinator) on - - tags: added: bionic changed in ecryptfs-utils (ubuntu): status: new → confirmed mikko rantalainen (mira) wrote on - - : # still happens with ubuntu lts . . i can provide additional info if needed. jarno suni (jarnos) wrote on - - : # the bug seems to be present in . . , too. daniel van vugt (vanvugt) on - - tags: added: xenial removed: artful dronus (paul-geisler) wrote on - - : # still an issue as of today - - , for ubuntu . . please set this critical immediately, this is a strong security issue for the use case of multiuser device! using a "guest" user to share your device with others is a quite common use case. everyone would expect that logging out and pass the device to another person would benefit from a ecrypt fs user homdedir setup. i myself usually log out and set my device to standby on a daily routine. that has me left unprotected by a while now it seems. i usually log out when taking the laptop to unsafe spaces like traveling, conferences and events in public spaces etc. i had expected that logging out would keep my data safe to some level. as this is a regression too, many already adapted to logout-is-safe behaviour. dronus (paul-geisler) wrote on - - : # also don't rely on the "affects me" counter above, as for non-power-users this bug is subtle and undetectable in most cases, leaving them unsafe without knowledge. klaus bielke (k-bielke) wrote on - - : # see for workaround: - https://wiki.ubuntuusers.de/ecryptfs/einrichten/#problembehebung - https://bugs.debian.org/cgi-bin/bugreport.cgi?bug= # bug watch updater (bug-watch-updater) on - - changed in ecryptfs-utils (debian): status: unknown → new kai kasurinen (kai-kasurinen) on - - affects: gnome-session-shutdown → gnome-session dave j (bigcus) wrote on - - : # note this was also reported in bug back in daniel van vugt (vanvugt) on - - tags: removed: xenial see full activity log to post a comment you must log in. report a bug this report contains public security information edit everyone can see this security related information. mark as duplicate convert to a question link a related branch link to cve you are not directly subscribed to this bug's notifications. subscribing... edit bug mail other bug subscribers subscribe someone else bug attachments dependencies.txt (edit) journalerrors.txt (edit) proccpuinfominimal.txt (edit) add attachment remote bug watches debbugs # [open critical security] edit bug watches keep track of this bug in other bug trackers. • take the tour • read the guide © - canonical ltd. • terms of use • data privacy • contact launchpad support • blog • careers • system status • r d abcb (get the code!) github - fcrepo-exts/fcrepo-aws-deployer: a terraform script for deploying fedora repository to aws. skip to content sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories → security → team enterprise explore explore github → learn & contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → nonprofit → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this organization all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up {{ message }} fcrepo-exts / fcrepo-aws-deployer watch star fork a terraform script for deploying fedora repository to aws. apache- . license stars forks star watch code issues pull requests actions projects security insights more code issues pull requests actions projects security insights main branch tags go to file code clone https github cli use git or checkout with svn using the web url. work fast with our official cli. learn more. open with github desktop download zip launching github desktop if nothing happens, download github desktop and try again. go back launching github desktop if nothing happens, download github desktop and try again. go back launching xcode if nothing happens, download xcode and try again. go back launching visual studio if nothing happens, download the github extension for visual studio and try again. go back latest commit git stats commits files permalink failed to load latest commit information. type name latest commit message commit time elasticbeanstalk .gitignore license readme.md main.tf variables.tf view code readme.md fcrepo-aws-deployer a terraform script for automatically deploying a fedora repository to aws. by default, fedora is deployed on a t .small instance and is backed by postgresql . hosted in rds on a db.t .micro instance. requirements terraform (https://www.terraform.io/downloads.html) installation after installing terraform git clone https://github.com/lyrasis/fcrepo-terraform terraform init then set up an aws profile in ~/.aws/config (c.f. https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html) deploy fedora terraform apply -var 'aws_profile=' -var 'ec _keypair=' -var 'aws_artifact_bucket_name=' nb: make sure that the aws bucket you designate does not already exist and do not anything in that bucket that you do not want deleted on teardown. tear it down terraform destroy -var 'aws_profile=' -var 'ec _keypair=' other variables see ./variables.tf for a complete list of optional parameters. about a terraform script for deploying fedora repository to aws. resources readme license apache- . license releases no releases published packages no packages published contributors languages hcl . % © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. unt digital collections | university of north texas libraries information for... graduate students faculty/staff community & visitors persons with disabilities unt links myunt eagleconnect blackboard people & departments maps calendars giving to unt other library sites unt dallas library unt health science library unt dallas law library other libraries library catalog interlibrary loan & document delivery (illiad) subject & course guides unt digital libraries the portal to texas history finding aids exhibits student study rooms reservations faculty/staff room reservations course reserves digital audio reserves video on demand unt press open access @ unt university libraries' scholarly blogs university libraries unt libraries search all online articles books & more databases e-journals unt digital collections subject guides this website menu home about research services news calendar ask us accounts login search all digital collections what resource does this search? search fulltext metadata title subject creator about the unt libraries' digital collections go to: unt digital library | the portal to texas history | unt scholarly works locations & hours select location willis library discovery park eagle commons media library annex mail university libraries union circle # denton , tx - partnerships the university libraries are a designated federal depository library. the university libraries are a funding information network partner of the foundation center. contact . . research questions staff directory computer & technology problems facilities problems disclaimer aa/eoe/ada privacy electronic accessibility state of texas online emergency preparedness student consumer information unt compliance hotline unt home find unt digital collections searches online content hosted in the unt digital library, the portal to texas history, and the the gateway to oklahoma history. the unt digital library is home to materials from the university's research, creative and scholarly activities, and also showcases content from the unt libraries collections. the portal is a gateway to texas history materials. it includes unique collections of texas libraries, museums, archives, historical societies, genealogical societies, and private family collections. the gateway is an online repository of oklahoma history. it provides free online access to the unique items held by the oklahoma historical society. for more information see the unt libraries digital projects. meta interchange – libraries, computing, metadata, and more skip to content meta interchange libraries, computing, metadata, and more search for submit primary menu about comment policy privacy policy search for submit trading for images posted: february categories: libraries, patron privacy let’s search a koha catalog for something that isn’t at all controversial: what you search for in a library catalog ought to be only between you and the library — and that, only briefly, as the library should quickly forget. of course, between “ought” and “is” lies the devil and his details. let’s poke around with chrome’s devtools: hit control-shift-i (on windows) switch to the network tab. hit control-r to reload the page and get a list of the http requests that the browser makes. we get something like this: there’s a lot to like here: every request was made using https rather than http, and almost all of the requests were made to the koha server. (if you can’t trust the library catalog, who can you trust? well… that doesn’t have an answer as clear as we would like, but i won’t tackle that question here.) however, the two cover images on the result’s page come from amazon: https://images-na.ssl-images-amazon.com/images/p/ . .tzzzzzzz.jpg https://images-na.ssl-images-amazon.com/images/p/ . .tzzzzzzz.jpg what did i trade in exchange for those two cover images? let’s click on the request on and see: :authority: images-na.ssl-images-amazon.com :method: get :path: /images/p/ . .tzzzzzzz.jpg :scheme: https accept: image/webp,image/apng,image/,/*;q= . accept-encoding: gzip, deflate, br accept-language: en-us,en;q= . cache-control: no-cache dnt: pragma: no-cache referer: https://catalog.libraryguardians.com/cgi-bin/koha/opac-search.pl?q=anarchist sec-fetch-dest: image sec-fetch-mode: no-cors sec-fetch-site: cross-site user-agent: mozilla/ . (windows nt . ; win ; x ) applewebkit/ . (khtml, like gecko) chrome/ . . . safari/ . here’s what was sent when i used firefox: host: images-na.ssl-images-amazon.com user-agent: mozilla/ . (windows nt . ; win ; x ; rv: . ) gecko/ firefox/ . accept: image/webp,/ accept-language: en-us,en;q= . accept-encoding: gzip, deflate, br connection: keep-alive referer: https://catalog.libraryguardians.com/cgi-bin/koha/opac-search.pl?q=anarchist dnt: pragma: no-cache amazon also knows what my ip address is. with that, it doesn’t take much to figure out that i am in georgia and am clearly up to no good; after all, one look at the referer header tells all. let’s switch over to using google book’s cover images: https://books.google.com/books/content?id=phzfwaeacaaj&printsec=frontcover&img= &zoom= https://books.google.com/books/content?id=wdgrjqaacaaj&printsec=frontcover&img= &zoom= this time, the request headers are in chrome: :authority: books.google.com :method: get :path: /books/content?id=phzfwaeacaaj&printsec=frontcover&img= &zoom= :scheme: https accept: image/webp,image/apng,image/,/*;q= . accept-encoding: gzip, deflate, br accept-language: en-us,en;q= . cache-control: no-cache dnt: pragma: no-cache referer: https://catalog.libraryguardians.com/ sec-fetch-dest: image sec-fetch-mode: no-cors sec-fetch-site: cross-site user-agent: mozilla/ . (windows nt . ; win ; x ) applewebkit/ . (khtml, like gecko) chrome/ . . . safari/ . x-client-data: cko yqeiilbjaqimtskbcmg yqeiqz kaqi qsobcmuuygeiz /kaqi smobcje ygei bxkaqinusobgkukygeyvrrkaq== and in firefox: host: books.google.com user-agent: mozilla/ . (windows nt . ; win ; x ; rv: . ) gecko/ firefox/ . accept: image/webp,/ accept-language: en-us,en;q= . accept-encoding: gzip, deflate, br connection: keep-alive referer: https://catalog.libraryguardians.com/ dnt: pragma: no-cache cache-control: no-cache on the one hand… the referer now contains only the base url of the catalog. i believe this is due to a difference in how koha figures out the correct image url. when using amazon for cover images, the isbn of the title is normalized and used to construct a url for an tag. koha doesn’t currently set a referrer-policy, so the default of no-referrer-when-downgrade is used and the full referrer is sent. google book’s cover image urls cannot be directly constructed like that, so a bit of javascript queries a web service and gets back the image urls, and for reasons that are unclear to me at the moment, doesn’t send the full url as the referrer. (cover images from openlibrary are fetched in a similar way, but full referer header is sent.) as a side note, the x-client-data header sent by chrome to books.google.com is… concerning. there are some relatively simple things that can be done to limit leaking the full referring url to the likes of google and amazon, including setting the referrer-policy header via web server configuration or meta tag to something like origin or origin-when-cross-origin. setting referrerpolicy for try topojson-server in your browser. api reference # topojson.topology(objects[, quantization]) <> returns a topojson topology for the specified geojson objects. the returned topology makes a shallow copy of the input objects: the identifier, bounding box, properties and coordinates of input objects may be shared with the output topology. if a quantization parameter is specified, the input geometry is quantized prior to computing the topology, the returned topology is quantized, and its arcs are delta-encoded. quantization is recommended to improve the quality of the topology if the input geometry is messy (i.e., small floating point error means that adjacent boundaries do not have identical values); typical values are powers of ten, such as e , e or e . see also topojson.quantize to quantize a topology after it has been constructed, without altering the topological relationships. command-line reference geo topo # geo topo [options…] [name=]file… <> converts one or more geojson objects to an output topology. for example, to convert a geojson featurecollection in the input file us-states.json to a topologyjson topology in the output file us.json: geo topo states=us-states.json > us.json the resulting topology has a “states” object which corresponds to the input geometry. for convenience, you can omit the object name and specify only the output file name; the object name will then be the basename of the file, with the directory and extension removed. for example, to convert the states.json geojson featurecollection to a topologyjson topology with the “states” object in us.json: geo topo states.json > us.json any properties and identifiers of input feature objects are propagated to the output. if you want to transform or filter properties, try ndjson-cli as demonstrated in command-line cartography. see also topo geo. # geo topo -h # geo topo --help output usage information. # geo topo -v # geo topo --version output the version number. # geo topo -n # geo topo --newline-delimited accept newline-delimited json, with one feature per line. # geo topo -o file # geo topo --out file specify the output topojson file name. defaults to “-” for stdout. # geo topo -q count # geo topo --quantization count specify a pre-quantization paramter. disables quantization. see topojson.topology for a description of quantization. about convert geojson to topojson. resources readme license isc license releases v . . latest sep , + releases packages no packages published used by k + , contributors mbostock mike bostock fil philippe rivière mfogel mike fogel languages javascript . % © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. github - unitedstates/congress-legislators: members of the united states congress, -present, in yaml/json/csv, as well as committees, presidents, and vice presidents. skip to content sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories → security → team enterprise explore explore github → learn & contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → nonprofit → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this organization all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up {{ message }} unitedstates / congress-legislators watch star . k fork members of the united states congress, -present, in yaml/json/csv, as well as committees, presidents, and vice presidents. view license . k stars forks star watch code issues pull requests actions projects security insights more code issues pull requests actions projects security insights master branches tags go to file code clone https github cli use git or checkout with svn using the web url. work fast with our official cli. learn more. open with github desktop download zip launching github desktop if nothing happens, download github desktop and try again. go back launching github desktop if nothing happens, download github desktop and try again. go back launching xcode if nothing happens, download xcode and try again. go back launching visual studio if nothing happens, download the github extension for visual studio and try again. go back latest commit joshdata fix start date for sen. ashurst … dfb b jan , fix start date for sen. ashurst dfb b git stats , commits files permalink failed to load latest commit information. type name latest commit message commit time .circleci update circleci docker image to python . , drop -browsers # may , misc add sen. kelly loeffler, update sen. perdue to senior, add more field… jan , scripts ran house_websites.py jan , test revert "disable bioguide and url checks until this information become… jan , .gitattributes (force) new commit with test data removed, generate json on saving data mar , .gitignore ignore a common script i use jul , contributing.md add a contributors document feb , license adding cc legal text aug , readme.md update readme.md jan , committee-membership-current.yaml ran committee_membership.py jan , committees-current.yaml ran house_contacts.py, senate_contacts.py, committee_membership.py sep , committees-historical.yaml update historical committees and yaml file (# ) dec , congress_lookup.py fix pyflakes-detected errors feb , executive.yaml add executive.yaml terms for biden and harris (# ) jan , legislators-current.yaml ran house_websites.py jan , legislators-district-offices.yaml harris resigns to become vp jan , legislators-historical.yaml fix start date for sen. ashurst jan , legislators-social-media.yaml added more twitter accounts jan , view code readme.md congress-legislators members of the united states congress ( -present), congressional committees ( -present), committee membership (current only), and presidents and vice presidents of the united states in yaml, json, and csv format. overview this project provides the following data files: file download description legislators-current yaml json csv currently serving members of congress. legislators-historical yaml json csv historical members of congress (i.e. all members of congress except those in the current file). legislators-social-media yaml json current social media accounts for members of congress. official accounts only (no campaign or personal accounts). committees-current yaml json current committees of the congress, with subcommittees. committee-membership-current yaml json current committee/subcommittee assignments. committees-historical yaml json current and historical committees of the congress, with subcommittees, from the rd congress ( ) and on. legislators-district-offices yaml json csv district offices for current members of congress. executive yaml json presidents and vice presidents. the data formats are documented below. the files are maintained in yaml format in the master branch of this project. yaml is a serialization format similar in structure to json but typically written with one field per line. like json, it allows for nested structure. each level of nesting is indicated by indentation or a dash. csv and json formatted files are also provided in the gh-pages branch --- they're linked above. this database is maintained through a combination of manual edits by volunteers (from govtrack, propublica, maplight, fivethirtyeight, and others) and automated imports from a variety of sources including: govtrack.us (http://www.govtrack.us). the congressional biographical directory (http://bioguide.congress.gov). congressional committees, historical standing committees data set by garrison nelson and charles stewart (http://web.mit.edu/ . /www/data_page.html). martis’s “the historical atlas of political parties in the united states congress”, via rosenthal, howard l., and keith t. poole. united states congressional roll call voting records, - (http://voteview.com/dwnl.htm). the sunlight labs congress api (http://sunlightlabs.github.com/congress/). the library of congress's thomas website (http://thomas.loc.gov). c-span's congressional chronicle (http://www.c-spanvideo.org/congress) data format documentation legislators file structure overview legislators-current.yaml and legislators-historical.yaml contain biographical information on all members of congress that have ever served in congress, that is, since , as well as cross-walks into other databases. each legislator record is grouped into four guaranteed parts: id's which relate the record to other databases, name information (first, last, etc.), biographical information (birthday, gender), and terms served in congress. a typical record looks something like this: - id: bioguide: r thomas: ' ' govtrack: opensecrets: n votesmart: fec: - h wi cspan: wikipedia: paul ryan ballotpedia: paul ryan maplight: house_history: icpsr: name: first: paul middle: d. last: ryan bio: birthday: ' - - ' gender: m terms: ... - type: rep start: ' - - ' end: ' - - ' ... - type: rep start: ' - - ' end: ' - - ' state: wi party: republican district: url: http://paulryan.house.gov address: longworth hob; washington dc - phone: - - fax: - - contact_form: http://www.house.gov/ryan/email.htm office: longworth house office building terms correspond to elections and are listed in chronological order. if a legislator is currently serving, the current term information will always be the last one. to check if a legislator is currently serving, check that the end date on the last term is in the future. the split between legislators-current.yaml and legislators-historical.yaml is somewhat arbitrary because these files may not be updated immediately when a legislator leaves office. if it matters to you, just load both files. a separate file legislators-social-media.yaml stores social media account information. its structure is similar but includes different fields. data dictionary the following fields are available in legislators-current.yaml and legislators-historical.yaml: id bioguide: the alphanumeric id for this legislator in http://bioguide.congress.gov. note that at one time some legislators (women who had changed their name when they got married) had two entries on the bioguide website. only one bioguide id is included here. this is the best field to use as a primary key. thomas: the numeric id for this legislator on http://thomas.gov and http://beta.congress.gov. the id is stored as a string with leading zeros preserved. lis: the alphanumeric id for this legislator found in senate roll call votes (http://www.senate.gov/pagelayout/legislative/a_three_sections_with_teasers/votes.htm). fec: a list of ids for this legislator in federal election commission data. in the csv format, the fec_ids column is comma-separated. govtrack: the numeric id for this legislator on govtrack.us (stored as an integer). opensecrets: the alphanumeric id for this legislator on opensecrets.org. votesmart: the numeric id for this legislator on votesmart.org (stored as an integer). icpsr: the numeric id for this legislator in keith poole's voteview.com website, originally based on an id system by the interuniversity consortium for political and social research (stored as an integer). cspan: the numeric id for this legislator on c-span's video website, e.g. http://www.c-spanvideo.org/person/ (stored as an integer). wikipedia: the wikipedia page name for the person (spaces are given as spaces, not underscores). ballotpedia: the ballotpedia.org page name for the person (spaces are given as spaces, not underscores). maplight : the numeric id for this legislator on maplight.org (stored as an integer). house_history: the numeric id for this legislator on http://history.house.gov/people/search/. the id is present only for members who have served in the u.s. house. bioguide_previous: when bioguide.congress.gov mistakenly listed a legislator under multiple ids, this field is a list of alternative ids. (this often ocurred for women who changed their name.) the ids in this list probably were removed from bioguide.congress.gov but might still be in use in the wild. name first: the legislator's recognizable first name. many people go by a different name than their legal first name, often their legal middle name, and our approach is to ensure that our first + last name fields combine to a recognizable name of the legislator. normally we'll follow the name as it appears on house.gov or senate.gov (and bioguide.congress.gov), which follows the legislator's own preference for how they want to be named in official places. however, in some cases the legislator goes by a first name that is merely a common short or informal form of their legal first name (e.g. chris vs christopher), and while they may prefer the informal name, we may use their longer legal first name because they would be recognizable by their legal name. if they sign official documents (e.g. letters to agencies, fec filings) using their longer legal first name, we would use their legal first name and put their preferred shorter name in the nickname field. when legislators go by a first initial and middle name, we set the first name field to the initial (one character plus a period). middle: the legislator's middle name or middle initial (with period). it is not recommended to display this field, unless the first name field is an initial (one character plus a period). last: the legislator's last name. some names include non-ascii characters. when building search systems, it is advised to index both the raw value as well as a value with extended characters replaced with their ascii equivalents (in python that's: u"".join(c for c in unicodedata.normalize('nfkd', lastname) if not unicodedata.combining(c))). suffix: a suffix on the legislator's name, such as "jr." or "iii", but only if they use it in official contexts, such as if it appears on house.gov or senate.gov. nickname: the legislator's nick name when used as a common alternative to their first name. usually displayed within quotes after the first name. if they are generally only known by a nickname, we would likely place the name in the first name field instead (see above). official_full: the full name of the legislator according to the house or senate (usually first, middle initial, nickname, last, and suffix). present for those serving on - - and later. other_names, when present, lists other names the legislator has gone by officially. this is helpful in cases where a legislator's legal name has changed. these listings will only include the name attributes which differ from the current name, and a start or end date where applicable. where multiple names exist, other names are listed chronologically by end date. an excerpted example: id: bioguide: b thomas: ' ' govtrack: opensecrets: n name: first: mary middle: whitaker last: bono mack other_names: last: bono end: ' - - ' ... bio birthday: the legislator's birthday, in yyyy-mm-dd format. gender: the legislator's gender, either "m" or "f". (in historical data, we've worked backwards from history.house.gov's women in congress feature.) terms (one entry for each election) type: the type of the term. either "sen" for senators or "rep" for representatives and delegates to the house. start: the date legislative service began: the date the legislator was sworn in, if known, or else the beginning of the legislator's term. since regularly elected terms begin on january at noon on odd-numbered years, but when congress does not first meet on january , term start dates might reflect that swearing-in occurred on a later date. (prior to , terms began on march of odd-numbered years, see here.) formatted as yyyy-mm-dd. end: the date the term ended (because the congress ended or the legislator died or resigned, etc.). end dates follow the constitutional end of a term. since , terms begin and end on january at noon in odd-numbered years, and thus a term end date may also be a term start date. prior to , terms began on march and ended either on march or march . the end date is the last date on which the legislator served this term. unlike the start date, whether congress was in session or not does not affect the value of this field. state: the two-letter, uppercase usps abbreviation for the state that the legislator is serving from. see below. how: how the term came to be. this field is generally not present and is currently only set haphazardly in recent data. the field is set to "appointment" for senators appointed to fill a vacancy. senators and representatives elected by special election are indicated by special-election. for senators currently serving per an appointment, the field end-type may be set to special-election, in which case the end date of the term will reflect the expected special election date to replace the appointed senator. once the special election occurs and the next senator is sworn in, ending the term of the appointed senator, the end date will be updated to reflect the actual end of service (which will follow the election date). district: for representatives, the district number they are serving from. at-large districts are district . in historical data, unknown district numbers are recorded as - . class: for senators, their election class ( , , or ). note that this is unrelated to seniority. state_rank: for senators, whether they are the "junior" or "senior" senator (only valid if the term is current, otherwise the senator's rank at the time the term ended). party: the political party of the legislator. if the legislator changed parties, this is the most recent party held during the term and party_affiliations will be set. values are typically "democrat", "independent", or "republican". the value typically matches the political party of the legislator on the ballot in his or her last election, although for state affiliate parties such as "democratic farmer labor" we will use the national party name ("democrat") instead to keep the values of this field normalized. caucus: for independents, the party that the legislator caucuses with, using the same values as the party field--although not required for independents with no caucus. omitted if the legislator caucuses with the party indicated in the party field. when in doubt about the difference between the party and caucus fields, the party field is what displays after the legislator's name (i.e. "(d)") but the caucus field is what normally determines committee seniority. this field was added starting with terms for the th congress. party_affiliations: this field is present if the legislator changed party or caucus affiliation during the term. the value is a list of time periods, with start and end dates, each of which has a party field and a caucus field if applicable, with the same meanings as the main party and caucus fields. the time periods cover the entire term, so the first start will match the term start, the last end will match the term end, and the last party (and caucus if present) will match the term party (and caucus). url: the official website url of the legislator (only valid if the term is current). address: the mailing address of the legislator's washington, d.c. office (only valid if the term is current, otherwise the last known address). phone: the phone number of the legislator's washington, d.c. office (only valid if the term is current, otherwise the last known number). fax: the fax number of the legislator's washington, d.c. office (only valid if the term is current, otherwise the last known number). contact_form: the website url of the contact page on the legislator's official website (only valid if the term is current, otherwise the last known url). office: similar to the address field, this is just the room and building number, suitable for display (only valid if the term is current, otherwise the last known office). rss_url the url to the official website's rss feed (only valid if the term is current, otherwise the last known url). leadership roles: leadership_roles: - title: minority leader chamber: senate start: ' - - ' end: ' - - ' for members with top formal positions of leadership in each party in each chamber, a leadership_roles field will include an array of start/end dates and titles documenting when they held this role. leadership terms are not identical to legislative terms, and so start and end dates will be different than legislative term dates. however, leaders do need to be re-elected each legislative term, so their leadership terms should all be subsets of their legislative terms. except where noted, fields are omitted when their value is empty or unknown. any field may be unknown. notes: in most cases, a legislator has a single term on any given date. in some cases a legislator resigned from one chamber and was sworn in in the other chamber on the same day. terms for senators list each six-year term, so the terms span three congresses. for representatives and delegates, each two-year term is listed, each corresponding to a single congress. but puerto rico's resident commissioner serves four-year terms, and so the resident commissioner will have a single term covering two congresses (this has not been updated in historical data). historically, some states sending at-large representatives actually sent multiple at-large representatives. thus, state and district may not be a unique key. data on official social media accounts this dataset is designed to include accounts that are paid for with public funds and which represent official communications of their office. we rely on reasonable verification from the legislative office about the status of their accounts. offices are supposed to maintain strict separation of official funds and campaign funds, and official funds are not supposed to be used to further things like re-election efforts. in practice, a campaign account may often look similar to an official account in terms of content, especially when expressing views on issues and legislations. however, there will be differences in what's appropriate for each account, and they will likely be maintained by different staff employed by different organizations. the social media file legislators-social-media.yaml stores current social media account information. each record has two sections: id and social. the id section identifies the legislator using bioguide, thomas, and govtrack ids (where available). the social section has social media account identifiers: twitter: the current official twitter handle of the legislator. youtube: the current official youtube username of the legislator. youtube_id: the current official youtube channel id of the legislator. instagram: the current official instagram handle of the legislator. instagram_id: the numeric id of the current official instagram handle of the legislator. facebook: the username of the current official facebook presence of the legislator. several legislators do not have an assigned youtube username. in these cases, only the youtube_id field is populated. all values can be turned into urls by preceding them with the domain name of the service in question (and in the case of youtube channels, the path /channel): https://twitter.com/[twitter] https://youtube.com/user/[youtube] https://youtube.com/channel/[youtube_id] https://instagram/[instagram] https://facebook.com/[facebook] legislators are only present when they have one or more social media accounts known. fields are omitted when the account is unknown. updating social media accounts available tasks with scripts/social_media.py: --sweep: given a --service, looks through current members for those missing an account on that service, and checks that member's official website's source code for mentions of that service. uses a csv at data/social_media_blacklist.csv to exclude known non-individual account names. a csv of "leads" is produced for manual review. --update: given a --service, reads the csv produced by --sweep back in and updates the yaml accordingly. note: with small updates, for people already in the yaml, it's easiest to just update by hand. --clean: given a --service, removes legislators from the social media file who are no longer current. --resolvefb: uses facebook usernames to look up graph ids, and updates the yaml accordingly. --resolveyt uses youtube usernames to look up any channel ids, and updates the yaml accordingly. --resolveig uses instagram user ids to look up any usernames, and updates the yaml accordingly. options used with the above tasks: --service: can be "twitter", "youtube", or "facebook". --bioguide: limit activity to a single member, by bioguide id. --email: in conjunction with --sweep, send an email if there are any new leads, using settings in scripts/email/config.yml (if it was created and filled out). committees data dictionary the committees-current.yaml file lists all current house, senate, and joint committees of the united states congress. it includes metadata and cross-walks into other databases of committee information. it is based on data scraped from house.gov and senate.gov. the committees-historical.yaml file is a possibly partial list of current and historical committees and subcommittees referred to in the unitedstates/congress project bill data, as scraped from thomas.gov. only committees/subcommmittees that have had bills referred to them are included. the basic structure of a committee entry looks like the following: - type: house name: house committee on agriculture url: http://agriculture.house.gov/ thomas_id: hsag house_committee_id: ag jurisdiction: the u.s. house committee on agriculture, or agriculture committee, is a standing committee of the ... jurisdiction_source: http://en.wikipedia.org/wiki/house_committee_on_agriculture subcommittees: (... subcommittee list ...) the two files are structured each as a list of committees, each entry an associative array of key/value pairs of committee metadata. the fields available in both files are as follows: type: 'house', 'senate', or 'joint' indicating the type of commmittee name: the current (or most recent) official name of the committee. thomas_id: the four-letter code used for the committee on the thomas advanced search page. senate_committee_id: for senate and joint committees, the four-letter code used on http://www.senate.gov/pagelayout/committees/b_three_sections_with_teasers/membership.htm. currently the same as the thomas_id. house_committee_id: for house committees, the two-letter code used on http://clerk.house.gov/committee_info/index.aspx. currently always the same as the last two letters of the thomas_id. jurisdiction: the committee's jurisdiction. jurisdiction_source: the source for the jurisdiction text. subcommittees: a list of subcommittees, with the following fields: name: the name of the subcommittee, excluding "subcommittee on" that appears at the start of most subcommittee names. some subcommittee names begin with a lowercase "the" so bear that in mind during display. thomas_id: the two-digit (zero-padded) code for the subcommittee as it appeared on thomas, and likely also the same code used on the house and senate websites. additional fields are present on current committee entries (that is, in committees-current.yaml): url: the current website url of the committee. address: the mailing address for the committee. phone: the phone number of the committee. rss_url: the url for the committee's rss feed. minority_rss_url: the url for the committee's minority party website's rss feed. two additional fields are present on committees and subcommmittees in the committees-historical.yaml file: congresses: a list of congress numbers in which this committee appears on the thomas advanced search page. it is roughly an indication of the time period during which the committee was in use. however, if a committee was not referred any bills it may not appear on thomas's list and therefore would not appear here. names: a list of past names for the committee. this is an associative array from a congress number to the name of the committee. the name is that given on the thomas advanced search page for previous congresses and does not always exactly match the official names of commmittees. committee membership data dictionary the committee-membership-current.yaml file contains current committee assignments, as of the date of the last update of this file. the file is structured as a mapping from committee ids to a list of committee members. the basic structure looks like this: hsag: - name: frank d. lucas party: majority rank: title: chair bioguide: l thomas: ' ' - name: bob goodlatte party: majority rank: (...snip...) hsag : - name: jean schmidt party: majority rank: title: chair the committee ids in this file are the thomas_id's from the committees-current.yaml file, or for subcommittees the concatentation of the thomas_id of the parent committee and the thomas_id of the subcommittee. each committee/subcommittee entry is a list containing the members of the committee. each member has the following fields: name: the name of the member of congress. this field is intended for debugging. instead, use the id fields. some of the id fields used in the legislators yaml files, such as bioguide and thomas. party: either "majority" or "minority." committee work is divided strictly by party. rank: the apparent rank of the member on the committee, within his or her party. this is based on the order of names on the house/senate committee membership pages. rank is always for the committee chair or ranking member (the most senior minority party member). the rank is essentially approximate, because the house/senate pages don't necessarily make a committment that the order on the page precisely indicates actual rank (if such a concept even applies). but if you want to preserve the order as displayed by the house and senate, you can use this attribute. title: the title of the member on the committee, e.g. chair, ranking member, or ex officio. this field is not normalized, however, so be prepared to accept any string. chamber: for joint committees only, the chamber that the representative is serving in, either house or senate. district offices data dictionary the legistlators-district-offices.yaml file lists district offices for all currently serving members of congress. this data is crowdsourced from members' official websites. it does not include congressional offices in washington, d.c.; these are listed in the legislators-current.yaml file. each current member of congress has a listing in the file, comprised of two parts: ids and offices. the id section contains the fields bioguide, thomas, and govtrack, which correspond to fields with the same names in legislators-current.yaml as described above. the bioguide field is required, and used as the primary key for this file. the offices section is a list of the member's district offices. each listing contains the following fields: address: the street address of the office, e.g. " main st". building: the name of the building containing the office, if applicable, e.g. "dane county courthouse". city: the city containing the office. required fax: the fax machine number of the office, e.g. - - . hours: free-text field describing the days and hours the office is open. phone: the main phone number of the office, .e.g. - - state: the two-letter state code of the state containing the office. required suite: the suite or room number of the office, if applicable, e.g. "suite " zip: the -digit usps zip code of the office, e.g. " ". latitude: the decimal latitude of the office's geocoded location, e.g. . . longitude: the decimal longitude of the office's geocoded location, e.g. - . . id: an identifier for the office, consisting of the member's bioguide id and the city name, e.g. "x -seattle". required to qualify for inclusion in this file, an office must have at least an address or a phone number. the executive branch because of their role in the legislative process, we also include a file executive.yaml which contains terms served by u.s. presidents (who signed legislation) and u.s. vice presidents (who are nominally the president of the senate and occassionally cast tie-breaking votes there). this file has a similar structure as the legislator files. the file contains a list, where each entry is a person. each entry is a dict with id, name, bio, and terms fields. the id, bio, and name fields are the same as those listed above. except: icpsr_prez: the numeric icpsr identifier used in voteview.com historical roll call data when indicating the position of the president on a roll call vote. if the person also served in congress, he or she will also have a regular icpsr id with a different value. each term has the following fields: type: either "prez" (a presidential term) or "viceprez" (a vice presidential term). start: the start date of the term. in modern times, typically january following an election year. end: the end date of the term. in modern times, typically january following an election year. party: the political party from which the person was elected. how: how the term came to be, either "election" (the normal case), "succession" (presidential succession), or "appointment" (the appointment by the president of a new vice president). presidents and vice presidents that previously served in congress will also be listed in one of the legislator files, but their congressional terms will only appear in the legislator files and their executive-branch terms will only appear in executive.yaml. state abbreviations although you can find the usps abbreviations for the states anywhere, non-voting delegates from territories --- including historical territories that no longer exist --- are included in this database. here is a complete list of abbreviations: the states: ak alaska al alabama ar arkansas az arizona ca california co colorado ct connecticut de delaware fl florida ga georgia hi hawaii ia iowa id idaho il illinois in indiana ks kansas ky kentucky la louisiana ma massachusetts md maryland me maine mi michigan mn minnesota mo missouri ms mississippi mt montana nc north carolina nd north dakota ne nebraska nh new hampshire nj new jersey nm new mexico nv nevada ny new york oh ohio ok oklahoma or oregon pa pennsylvania ri rhode island sc south carolina sd south dakota tn tennessee tx texas ut utah va virginia vt vermont wa washington wi wisconsin wv west virginia wy wyoming current territories: legislators serving in the house from these territories are called delegates, except for the so-called "resident commissioner" from puerto rico. as american samoa dc district of columbia gu guam mp northern mariana islands pr puerto rico vi virgin islands historical territories: these territories no longer exist. dk dakota territory ol territory of orleans pi philippines territory/commonwealth helping us maintain the data you can just use the data directly without running any scripts. if you want to develop on and help maintain the data, our scripts are tested and developed on python . . (recommended) first, create a virtualenv in the scripts directory: cd scripts virtualenv virt source virt/bin/activate install the requirements: pip install -r requirements.txt try updating the house members contact information (mailing address, etc.): python house_contacts.py check whether and how the data has changed: git diff ../*.yaml we run the following scripts periodically to scrape for new information and keep the data files up to date. the scripts do not take any command-line arguments. house_contacts.py: updates house members' contact information (address, office, and phone fields on their current term, and their official_full name field) house_websites.py: updates house members' current website urls. senate_contacts.py: updates senator information (party, class, state_rank, address, office, phone, and contact_form fields on their current term, and their official_full name, bioguide id, and lis id fields) committee_membership.py: updates committees-current.yaml (name, address, and phone fields for house committees; name and url fields for senate committees; creates new subcommittees when found with name and thomas_id fields) and writes out a whole new committee-membership-current.yaml file by scraping the house and senate websites. historical_committees.py: updates committees-historical.yaml based on the committees listed on thomas.gov, which are committees to which bills have been referred since the rd congress ( ). social_media.py: generates leads for twitter, youtube, and facebook accounts for members of congress by scraping their official websites. uses a blacklist csv and a whitelist csv to manage false positives and negatives. influence_ids.py: grabs updated fec and opensecrets ids from the influence explorer api. will only work for members with a bioguide id. the following script takes one required command line argument icpsr_ids.py: updates icpsr id's for all members of the house and senate in a given congress, based on roll call vote data files stored by voteview.com. the script takes one command line argument: --congress=congress_number where congress_number is the number of the congress to be updated. as of july, , the permanent url for future roll call data is unclear, and as such, the script may need to be modified when it is run for the th congress. the following script is run to create alternately formatted data files for the gh-pages branch. it takes no command-line arguments. alternate_bulk_formats.py: creates json files for all yaml files and csv files for current legislators, historical legislators, and district offices. the csv files do not include all fields from the legislator yaml files, and do include data from the social media yaml. two scripts help maintain and validate district office data: geocode_offices.py : derives latitude, longitude pairs for office addresses. it should be run whenever new offices are added. by default this script geocodes all offices with addresses that have not already been geocoded. it optionally takes bioguide ids as arguments, and in this case will geocode just offices for the specified ids. this script uses the google maps api, and requires that a key be set in scripts/cache/google_maps_api_key.txt . office_validator.py : validates rules for district office data and reports errors and warnings. an optional --skip-warnings argument will suppress display of warnings. this script should be run whenever offices are added or modified. it is used by continuous integration testing, so errors here will cause the build to fail. every script in scripts/ should be safely import-able without executing code, beyond imports themselves. we typically do this with a def run(): declaration after the imports, and putting this at the bottom of the script: if __name__ == '__main__': run() every pull request will pass submitted scripts through an import, to catch exceptions, and through pyflakes, to catch unused imports or local vars. to contribute updates for district offices, edit the legislators-district-offices.yaml file by hand and submit a pull request. updates should pass validation as defined by scripts/office_validator.py. other scripts the ballotpedia field has been created using code from james michael dupont, using the code in git@github.com:h ck rm k /rootstrikers-wikipedia.git in the branch ballotpedia. related libraries karl nicholas made a set of java classes to easily filter the data. thewalkers maintain congress-turk to do bulk collection of district office data using amazon mechanical turk. who's using this data ongoing projects making use of this data: govtrack.us sunlight congress api propublica congress api represent everypolitician.org stories written with this data: http://swampland.time.com/ / / /meet-the- -democrats-who-voted-with-the-house-on-obamacare/ http://swampland.time.com/ / / /liz-cheney-and-the-family-business-a-chart-of-all-congressional-dynasties/ other projects: margie roswell's committee map public domain this project is dedicated to the public domain. as spelled out in contributing: the project is in the public domain within the united states, and copyright and related rights in the work worldwide are waived through the cc . universal public domain dedication. all contributions to this project will be released under the cc dedication. by submitting a pull request, you are agreeing to comply with this waiver of copyright interest. about members of the united states congress, -present, in yaml/json/csv, as well as committees, presidents, and vice presidents. resources readme license view license releases no releases published packages no packages published contributors + contributors languages python . % shell . % © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. github - code lib/planetcode lib: configuration for https://planet.code lib.org/ skip to content sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories → security → team enterprise explore explore github → learn & contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → nonprofit → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this organization all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up {{ message }} code lib / planetcode lib watch star fork configuration for https://planet.code lib.org/ stars forks star watch code issues pull requests actions projects security insights more code issues pull requests actions projects security insights master branch tags go to file code clone https github cli use git or checkout with svn using the web url. work fast with our official cli. learn more. open with github desktop download zip launching github desktop if nothing happens, download github desktop and try again. go back launching github desktop if nothing happens, download github desktop and try again. go back launching xcode if nothing happens, download xcode and try again. go back launching visual studio if nothing happens, download the github extension for visual studio and try again. go back latest commit git stats commits files permalink failed to load latest commit information. type name latest commit message commit time themes venus @ de .gitignore .gitmodules readme.md config.ini test.ini view code readme.md planet.code lib.org planet code lib aggregates feeds and blogs of interest to the code lib community. it uses planet venus. installation generally > git clone git@github.com:code lib/planetcode lib.git > cd planetcode lib > git submodule init > git submodule update > ./venus/planet.py --verbose the generated files will be in output/. to test it with one feed, run > ./venus/planet.py --verbose test.ini installation on the code lib.org server downloading and cloning is done over https so it's as generic as possible. no updates are to be made on the server; they should be made locally, pushed to github, then pulled down. > # become the c l user > cd /var/www/code lib.org/planet_new > git clone https://github.com/code lib/planetcode lib.git > cd planetcode lib > git submodule init > git submdule update > ./venus/planet.py --verbose --expunge to update: > # become the c l user > cd /var/www/code lib.org/planet_new/planetcode lib > git pull the relevant line in c l's crontab is: , * * * * cd /var/www/code lib.org/planet_new/planetcode lib; ./venus/planet.py --expunge >& adding (or removing) a feed additions are welcome! email william denton or submit a pull request modifying config.ini. if you're on the list but don't want to be, please do the same, and you'll be removed, no questions asked. about configuration for https://planet.code lib.org/ resources readme releases no releases published packages no packages published contributors + contributors languages xslt . % css . % html . % © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. distributions/readme.md at master · nodesource/distributions · github skip to content sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories → security → team enterprise explore explore github → learn & contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → nonprofit → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this organization all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up {{ message }} nodesource / distributions watch star k fork . k code issues pull requests actions projects wiki security insights more code issues pull requests actions projects wiki security insights permalink master distributions/readme.md go to file go to file t go to line l copy path igsu update readme.md latest commit b jan , history contributors users who have contributed to this file + lines ( sloc) . kb raw blame nodesource node.js binary distributions this repository contains documentation for using the nodesource node.js binary distributions via .rpm, .deb and snap packages as well as their setup and support scripts. if you are looking for nodesource's low-impact node.js performance monitoring platform, please get started here. please file an issue if you are experiencing a problem or would like to discuss something related to the distributions. pull requests are encouraged if you have changes you believe would improve the setup process or increase compatibility across linux distributions. table of contents debian and ubuntu based distributions (deb) installation instructions manual installation enterprise linux based distributions (rpm) installation instructions snap packages about installation instructions advanced usage tests faq requested distributions license debian and ubuntu based distributions available architectures: nodesource will continue to maintain the following architectures and may add additional ones in the future. amd ( -bit) armhf (arm -bit hard-float, armv and up: arm-linux-gnueabihf) arm (arm -bit, armv and up: aarch -linux-gnu) supported ubuntu versions: nodesource will maintain ubuntu distributions in active support by canonical, including lts and the intermediate releases. ubuntu . lts (trusty tahr) - not available for node.js and later ubuntu . lts (xenial xerus) ubuntu . lts (bionic beaver) ubuntu . (cosmic cuttlefish) ubuntu . (disco dingo) ubuntu . (eoan ermine) ubuntu . lts (focal fossa) ubuntu . (groovy gorilla) supported debian versions: nodesource will maintain support for stable, testing and unstable releases of debian, due to the long release cycle a considerable number of users are running unstable and testing. debian / oldoldstable (jessie) debian / oldstable (stretch) debian / stable (buster) debian unstable (sid) debian testing (bullseye) supported linux mint versions: linux mint "qiana" (via ubuntu . lts) - not available for node.js and later linux mint . "rebecca" (via ubuntu . lts) - not available for node.js and later linux mint . "rafaela" (via ubuntu . lts) - not available for node.js and later linux mint "sarah" (via ubuntu . lts) linux mint . "serena" (via ubuntu . lts) linux mint . "sonya" (via ubuntu . lts) linux mint . "sylvia" (via ubuntu . lts) linux mint debian edition (lmde) "betsy" (via debian ) linux mint "tara" (via ubuntu . lts) linux mint . "tessa" (via ubuntu . lts) linux mint . "tina" (via ubuntu . lts) linux mint . "tricia" (via ubuntu . lts) linux mint "ulyana" (via ubuntu . lts) linux mint . "ulyssa" (via ubuntu . lts) linux mint debian edition (lmde) "cindy" (via debian ) linux mint debian edition (lmde) "debbie" (via debian ) supported devuan versions: jessie / oldstable (via debian ) ascii / stable (via debian ) beowulf / testing (via debian ) ceres / unstable (via debian unstable) supported elementary os versions: elementary os . freya (via ubuntu . lts) - not available for node.js and later elementary os . loki (via ubuntu . lts) elementary os juno (via ubuntu . lts) elementary os . hera (via ubuntu . lts) elementary os odin (via ubuntu . lts) supported trisquel versions: trisquel "belenos" (via ubuntu . lts) - not available for node.js and later trisquel "flidas" (via ubuntu . lts) trisquel "etiona" (via ubuntu . lts) supported boss versions: boss . "anokha" (via debian ) - not available for node.js and later boss . "anoop" (via debian ) boss . "drishti" (via debian ) boss . "unnati" (via debian ) supported bunsenlabs versions: hydrogen (via debian ) helium (via debian ) lithium (via debian ) supported mx linux versions: mx- horizon (via debian ) mx- continuum (via debian ) mx- patito feo (via debian ) supported sparky linux versions: sparky .x "tyche" (via debian ) sparky .x "nibiru" (via debian ) supported pureos linux versions: pureos . "amber" (via debian ) supported astra linux ce versions: astra linux ce . "orel" (via debian ) supported ubilinux versions: ubilinux . "dolcetto" (via debian ) installation instructions node.js v .x: # using ubuntu curl -sl https://deb.nodesource.com/setup_ .x | sudo -e bash - sudo apt-get install -y nodejs # using debian, as root curl -sl https://deb.nodesource.com/setup_ .x | bash - apt-get install -y nodejs node.js v .x: # using ubuntu curl -sl https://deb.nodesource.com/setup_ .x | sudo -e bash - sudo apt-get install -y nodejs # using debian, as root curl -sl https://deb.nodesource.com/setup_ .x | bash - apt-get install -y nodejs node.js v .x: # using ubuntu curl -sl https://deb.nodesource.com/setup_ .x | sudo -e bash - sudo apt-get install -y nodejs # using debian, as root curl -sl https://deb.nodesource.com/setup_ .x | bash - apt-get install -y nodejs node.js v .x: # using ubuntu curl -sl https://deb.nodesource.com/setup_ .x | sudo -e bash - sudo apt-get install -y nodejs # using debian, as root curl -sl https://deb.nodesource.com/setup_ .x | bash - apt-get install -y nodejs node.js lts (v .x): # using ubuntu curl -sl https://deb.nodesource.com/setup_lts.x | sudo -e bash - sudo apt-get install -y nodejs # using debian, as root curl -sl https://deb.nodesource.com/setup_lts.x | bash - apt-get install -y nodejs node.js current (v .x): # using ubuntu curl -sl https://deb.nodesource.com/setup_current.x | sudo -e bash - sudo apt-get install -y nodejs # using debian, as root curl -sl https://deb.nodesource.com/setup_current.x | bash - apt-get install -y nodejs optional: install build tools to compile and install native addons from npm you may also need to install build tools: # use `sudo` on ubuntu or run this as root on debian apt-get install -y build-essential manual installation if you're not a fan of curl | bash -, or are using an unsupported distribution, you can try a manual installation. these instructions assume sudo is present, however some distributions do not include this command by default, particularly those focused on a minimal environment. in this case, you should install sudo or su to root to run the commands directly. . remove the old ppa if it exists this step is only required if you previously used chris lea's node.js ppa. # add-apt-repository may not be present on some ubuntu releases: # sudo apt-get install python-software-properties sudo add-apt-repository -y -r ppa:chris-lea/node.js sudo rm -f /etc/apt/sources.list.d/chris-lea-node_js-*.list sudo rm -f /etc/apt/sources.list.d/chris-lea-node_js-*.list.save . add the nodesource package signing key curl -ssl https://deb.nodesource.com/gpgkey/nodesource.gpg.key | sudo apt-key add - # wget can also be used: # wget --quiet -o - https://deb.nodesource.com/gpgkey/nodesource.gpg.key | sudo apt-key add - the key id is a ab . . add the desired nodesource repository # replace with the branch of node.js or io.js you want to install: node_ .x, node_ .x, etc... version=node_ .x # the below command will set this correctly, but if lsb_release isn't available, you can set it manually: # - for debian distributions: jessie, sid, etc... # - for ubuntu distributions: xenial, bionic, etc... # - for debian or ubuntu derived distributions your best option is to use the codename corresponding to the upstream release your distribution is based off. this is an advanced scenario and unsupported if your distribution is not listed as supported per earlier in this readme. distro="$(lsb_release -s -c)" echo "deb https://deb.nodesource.com/$version $distro main" | sudo tee /etc/apt/sources.list.d/nodesource.list echo "deb-src https://deb.nodesource.com/$version $distro main" | sudo tee -a /etc/apt/sources.list.d/nodesource.list . update package lists and install node.js sudo apt-get update sudo apt-get install nodejs enterprise linux based distributions available architectures: nodesource will continue to maintain the following architectures and may add additional ones in the future. i ( -bit) — not available for all distros and not available for node.js .x and later x _ ( -bit) arm (arm -bit, armv and up: aarch -linux-gnu) supported red hat® enterprise linux® versions: rhel ( -bit) rhel ( -bit) supported centos versions: centos ( -bit) centos ( -bit) centos stream ( -bit) supported cloudlinux versions: cloudlinux ( -bit for node <= .x and -bit) supported fedora versions: fedora ( -bit) - available for node.js . . and later fedora ( -bit) fedora ( -bit) supported amazon linux versions: amazon linux ( -bit) amazon linux ( -bit) installation instructions note: if you are using rhel or centos , you might want to read about running node.js on older distros. the nodesource rpm package signing key is available here: https://rpm.nodesource.com/pub/el/nodesource-gpg-signing-key-el run on rhel, centos, cloudlinux, amazon linux or fedora: node.js v .x # as root curl -sl https://rpm.nodesource.com/setup_ .x | bash - # no root privileges curl -sl https://rpm.nodesource.com/setup_ .x | sudo bash - node.js v .x # as root curl -sl https://rpm.nodesource.com/setup_ .x | bash - # no root privileges curl -sl https://rpm.nodesource.com/setup_ .x | sudo bash - node.js v .x # as root curl -sl https://rpm.nodesource.com/setup_ .x | bash - # no root privileges curl -sl https://rpm.nodesource.com/setup_ .x | sudo bash - node.js v .x # as root curl -sl https://rpm.nodesource.com/setup_ .x | bash - # no root privileges curl -sl https://rpm.nodesource.com/setup_ .x | sudo bash - node.js lts ( .x) # as root curl -sl https://rpm.nodesource.com/setup_lts.x | bash - # no root privileges curl -sl https://rpm.nodesource.com/setup_lts.x | sudo bash - node.js current ( .x) # as root curl -sl https://rpm.nodesource.com/setup_current.x | bash - # no root privileges curl -sl https://rpm.nodesource.com/setup_current.x | sudo bash - optional: install build tools to compile and install native addons from npm you may also need to install build tools: yum install gcc-c++ make # or: yum groupinstall 'development tools' snap packages about snaps are containerized software packages designed to work across cloud, desktop, and iot devices. they work natively on most popular linux distributions and feature automatic transactional updates. the nodesource-managed node.js snap contains the node.js runtime, along with the two most widely-used package managers, npm and yarn. they are delivered from the snapcraft store and are automatically built and pushed for each supported node.js release line. generally you will have a new version of node.js automatically running on your computer the same day it is released on nodejs.org. the node.js snap can currently be installed on arch linux, debian, fedora, linux mint, manjaro, openembedded/yocto, opernwrt, solus, ubuntu and many other distributions built on top these. nodesource has not tested the node.js snap on all of these distributions and feedback is welcome in this repository if you run into problems. installation instructions the snap command ships with ubuntu, from version . and later. if you do not have it installed, follow the instructions on snapcraft to install snapd. snaps are delivered via "channels"; for node.js, the channel names are the major-version number of node.js. so select a supported node.js version and install with: sudo snap install node --classic --channel= substituting for the major version you want to install. both lts and current versions of node.js are available via snapcraft. the --classic argument is required here as node.js needs full access to your system in order to be useful, therefore it needs snap’s "classic confinement". by default, snaps are much more restricted in their ability to access your disk and network and must request special access from you where they need it. note that on some linux distributions, the snap confinement mechanisms are not fully supported so --classic may not be necessary or even supported. once installed, the node, npm and yarn commands are available for use and will remain updated for the channel you selected. switching channels you can use the refresh command to switch to a new channel at any time: sudo snap refresh node --channel= once switched, snapd will update node.js for the new channel you have selected. bleeding-edge node.js users feeling adventurous or interested in testing the latest code from the node.js core developers can install from the "edge" channel. this has an element of risk: it is a direct pipeline from the upstream node.js git repository to the snap store every day and previews the ongoing development work and may include breaking changes slated for the next major version of node.js. this is only recommend for those users who are willing to participate in testing and bug reporting upstream: sudo snap install node --classic --channel=edge not recommended for production deployments due to their auto-updating nature, snaps are not necessarily appropriate for the deployment of your node.js applications to production. nodesource recommends a stable and integration-tested deployment pipeline for production applications such as the .deb or .rpm distributions outlined above. however, snaps are an excellent way to keep developer machines updated and allow for trivial and convenient switching between node.js versions. advanced usage the snap man page, or canonical’s advanced snap usage tutorial contains details of advanced snapd functionality. tests to test an installation is working (and that the setup scripts are working!) use: curl -sl https://deb.nodesource.com/test | bash - faq q: how do i use this repo when behind a proxy? a: please take a look at issue # q: how do i pin to specific versions of node.js? a: please take a look at issue # q: i upgraded to a new major version of node.js using the scripts, but the old version is still being installed, what is going on? a: you probably need to clear out your package manager's cache. take a look at issue # q: i'm trying to install node.js on centos / rhel and it is failing, why? a: due to the limitations of the compiler toolchain on el and its end of general support, we no longer support. see issue # q: i'm seeing "your distribution, identified as ".i " or ".i , is not currently supported, why? a: node.js .x and newer require a bit os for rpms. see issue # q: why have certain versions of platforms/releases stopped receiving updates to node.js? a: unfortunately, newer versions of v require a modern compiler toolchain. on some platforms, such as arm wheezy, that toolchain is not available. see issue # q: why is my node.js version newer than the one of the script i’ve run? a: your package manager is probably installing a newer node.js version from a different source. see issue # q: what is the current status of ipv support? a: see issue # requested distributions we, unfortunately, do not have the resources necessary to support and test the plethora of linux releases in the wild, so we rely on community members such as yourself to get support on your favorite distributions! this is a list of releases that have been requested by the community. if you are interested in contributing to this project, this would be a great place to start! opensuse - issue # scientific linux - issue # linuxmint nadia - issue # tanglu bartholomea - issue # korora - issue # freepbx - issue # deepin - issue # popos - issue # kylin - issue # makululinux - issue # authors and contributors chris lea github/chrislea twitter/@chrislea rod vagg github/rvagg twitter/@rvagg william blankenship github/retrohacker twitter/@retrohack r harry truong github/harrytruong matteo brunati github/mattbrun brian white github/mscdex matt lewandowsky github/lewellyn jan-hendrik peters github/hennr andris reinman github/andris carvilsi github/carvilsi krasimir trenchev github/ava phil helm github/phelma xmohit github/ xmohit jdarling github/jdarling prayag verma github/pra misha brukman github/mbrukman simon lydell github/lydell sebastian blei github/iamsebastian jorge maldonado ventura notabug/jorgesumle mayank metha github/mayankmetha twitter/@mayankmethad adrian estrada github/edsadr twitter/@edsadr iván iguarán github/igsu contributions are welcomed from anyone wanting to improve this project! license this material is copyright (c) nodesource and licensed under the mit license. all rights not explicitly granted in the mit license are reserved. see the included license.md file for more details. supported with love by the nodesource team this project is not affiliated with debian, ubuntu, red hat, centos or fedora. ubuntu is a registered trademark of canonical ltd. debian is a registered trademark owned by software in the public interest, inc. red hat, centos and fedora are trademarks of red hat, inc. cloudlinux is a trademark of cloud linux, inc go © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. half of all phishing sites now have the padlock — krebs on security advertisement subscribe to rss follow me on twitter join me on facebook krebs on security in-depth security news and investigation about the author advertising/speaking nov half of all phishing sites now have the padlock maybe you were once advised to “look for the padlock” as a means of telling legitimate e-commerce sites from phishing or malware traps. unfortunately, this has never been more useless advice. new research indicates that half of all phishing scams are now hosted on web sites whose internet address includes the padlock and begins with “https://”. a live paypal phishing site that uses https:// (has the green padlock). recent data from anti-phishing company phishlabs shows that percent of all phishing sites in the third quarter of bore the padlock security icon next to the phishing site domain name as displayed in a browser address bar. that’s up from percent just one year ago, and from percent in the second quarter of . this alarming shift is notable because a majority of internet users have taken the age-old “look for the lock” advice to heart, and still associate the lock icon with legitimate sites. a phishlabs survey conducted last year found more than % of respondents believed the green lock indicated a website was either legitimate and/or safe. in reality, the https:// part of the address (also called “secure sockets layer” or ssl) merely signifies the data being transmitted back and forth between your browser and the site is encrypted and can’t be read by third parties. the presence of the padlock does not mean the site is legitimate, nor is it any proof the site has been security-hardened against intrusion from hackers. a live facebook phish that uses ssl (has the green padlock). most of the battle to combat cybercrime involves defenders responding to offensive moves made by attackers. but the rapidly increasing adoption of ssl by phishers is a good example in which fraudsters are taking their cue from legitimate sites. “phishlabs believes that this can be attributed to both the continued use of ssl certificates by phishers who register their own domain names and create certificates for them, as well as a general increase in ssl due to the google chrome browser now displaying ‘not secure’ for web sites that do not use ssl,” said john lacour, chief technology officer for the company. “the bottom line is that the presence or lack of ssl doesn’t tell you anything about a site’s legitimacy.” the major web browser makers work with a number of security organizations to index and block new phishing sites, often serving bright red warning pages that flag the page of a phishing scam and seek to discourage people from visiting the sites. but not all phishing scams get flagged so quickly. i spent a few minutes browsing phishtank.com for phishing sites that use ssl, and found this cleverly crafted page that attempts to phish credentials from users of bibox, a cryptocurrency exchange. click the image below and see if you can spot what’s going on with this web address: this live phish targets users of cryptocurrency exchange bibox. look carefully at the url in the address bar, and you’ll notice a squiggly mark over the “i” in bibox. this is an internationalized domain name, and the real address is https://www.xn--bbox-vw a[.]com/login load the live phishing page at https://www.xn--bbox-vw a[.]com/login (that link has been hobbled on purpose) in google chrome and you’ll get a red “deceptive site ahead” warning. load the address above — known as “punycode” — in mozilla firefox and the page renders just fine, at least as of this writing. this phishing site takes advantage of internationalized domain names (idns) to introduce visual confusion. in this case, the “i” in bibox.com is rendered as the vietnamese character “ỉ,” which is extremely difficult to distinguish in a url address bar. as krebsonsecurity noted in march, while chrome, safari and recent versions of microsoft’s internet explorer and edge browsers all render idns in their clunky punycode state, firefox will happily convert the code to the look-alike domain as displayed in the address bar. if you’re a firefox (or tor) user and would like firefox to always render idns as their punycode equivalent when displayed in the browser address bar, type “about:config” without the quotes into a firefox address bar. then in the “search:” box type “punycode,” and you should see one or two options there. the one you want is called “network.idn_show_punycode.” by default, it is set to “false”; double-clicking that entry should change that setting to “true.” tags: bibox, idn, internationalized domain names, john lacour, phishing, phishlabs, punycode, ssl this entry was posted on monday, november th, at : am and is filed under a little sunshine, latest warnings. you can follow any comments to this entry through the rss . feed. both comments and pings are currently closed. comments david c. november , at : pm fwiw, when i tried to visit the “bibox” phish site using firefox . . on november , , i got a “deceptive site ahead” warning advisory “provided by google safe browsing”. vb november , at : pm what’s the point of a certificate authority if they pass out certs to phishers? there should be a big difference between how the lock is displayed depending on if the cert is chained to a trusted certificate authority or not. james beatty november , at : am the cert merely helps you determine if you’re connected securely to the site in question. basic certs aren’t designed to tell you anything about the quality or intent of the site itself. a phisher could buy a cert from *any* trusted certificate authority; they choose le simply because there’s no cost involved. vb november , at : pm “chain of trust” should mean something. le is crapping on that trust. ralphw november , at : pm (responding to vb’s assertion that “chain of trust” should mean something) misplaced trust is the root of the problem, because trust is not transitive. derek november , at : pm letsencrypt is doing their job perfectly correct. they are validating that the site in question is in fact the site you see in the url. what is the alternative? make the certificate authorities check every site every day to make sure that the contents of the website is not deceptive? readership november , at : am vb, in an ideal world, certificates should reflect something about the trustworthiness of a site. a padlock should not only indicate that a browser has an encrypted connection to a website, but that it’s operated by someone good and decent. unfortunately, we don’t live in an ideal world. governments are not good, but it’s occasionally necessary to use their websites securely. banks and investment firms are a necessary evil, but their malevolence shouldn’t prevent a secure connection to view my accounts. facebook, twitter, and google are the very definition of bad corporatist oppressors who engage in pervasive tracking of everything and everyone, yet their idiot users expect secure account access. smaller websites might be owned creeps, bigots, extremists, or jerks. -does that mean they should all be ineligible to run https? -who’s to be the investigative authority checking into the lives of website owners? -where’s the due process for legitimate websites denied encryption certificate, because the owner has unpopular beliefs? and that’s why the padlock is not a symbol of trustworthiness and decency. it only means the connection from the user to the website is secure. nothing more, nothing less. joe november , at : am well said. people seem to place undue responsibility on a simple icon of a padlock. it cannot possibly convey all the information that might be useful for a consumer, even if they were so inclined to try a holistic profile of the website. browsers do have “more info” for those users who don’t understand what a padlock means. for chrome it says, “your information (for example, passwords or credit card numbers) is private when sent to this site. learn more” for firefox it says, “the page you are viewing was encrypted before being transmitted over the internet. encryption makes it difficult for unauthorized people to view information traveling between computers. it is therefore unlikely that anyone read this page as it traveled across the network.” if people don’t bother to even click to find out what the padlock means, they have little say. readership december , at : am + james davenport november , at : pm thanks for the firefox tip – dumb default these days for us ascii-speakers. david hickton november , at : pm hi brian: would like to catch up and talk to you about bringing you to pittsburgh for an event next year let me know when we can talk or call - - dave hickton lezbehonest november , at : am come on now brian…if you aren’t using letsencrypt during your phishing campaigns, which is even automated in even the average deployment, than you’re really a total scrub at the game. think about it; we need to get past an email filtering service, very likely a proxy or l firewall, so even with ssl intercept having a cert is basic, trivial and free. this really isn’t a great article from a content/usefulness standpoint to be honest. ssl is practically irrelevant and not something i’ve seen most of my user bases opt for in click-through rates. lee november , at : pm i think the point he’s trying to make is that so much mainstream media tried to give people a “simple” method for determining if a website was trying to phish them, and now that’s backfired. it was a flawed plan in the first place, obviously. g november , at : am first, let’s fix the article as firefox actually uses g safebrowsing and puts up a big red warning, so commenters like james d. are not mislead. then, @lezbehonest isn’t right, as average joe is not aware what letsencrypt is at all, not to mention the tech mumbo-jumbo mentioned; having a “grey padlock” is a problem and average people shall be made aware. in chrome/chromium you need at least two well placed clicks to see the actual cert and it doesn’t help since you have to _understand_ what you’re seeing. and i do not expect ev certs just magically appear everywhere, which may (or likely may not) help the issue. even worse it’s not that simple as “do not sign non-ascii” since, let’s be honest, the educated part of the world 🙂 [which is everywhere outside the us] is writing in utf- and it’s valid, most of the time. safebrowsing is a half-good solution, at least. should be implemented in every browser. g november , at : am oh by the way chromium [not chrome, which i don’t use and don’t know about] renders the _translated_ url as well instead of punycode by default. wilfred nilsen november , at : am thank you google for deflated the value of https by forcing everyone to use it. john clark november , at : am does anyone know how to find who the host for these two domains are? forusajobs.net forusajobs.com the scammer’s information is not appearing on icann whois. marty november , at : pm john, it looks like the domains you just posted were created by someone that is trying to load up on google ad money. his site does not seem to be configured to steal identity like the other sites you have found in the past. looks like he figured out how to hide his identity. james beatty november , at : am it takes about seconds to find the company that’s hosting those two sites… seymourb november , at : pm it takes evem less time to answer john clark’s question and tell him how to do it himself. james beatty november , at : am that would encourage john to continue promoting his blog by posting similar comments on every new topic here. drew november , at : pm if you look at his site it’s apparent he is not doing for the money. there is no advertising and it’s rather obvious he is using an fake name. “john clark” is the name used in tom clancy books. it looks like he just wants to get the word out so the scammers stop victimizing the people (much like krebs is doing). readership november , at : am john, congratulations on updating your site and catching a few more scammers. this site has a subscription option for delving into domain historical data, but also a whois tool you can use a few times each day for free: https://whois.domaintools.com/ the sites you inquired about were very recently re-registered. the free tool doesn’t indicate who previously registered those domains, but it shows the current registrant for one in panama and the other in usa. good luck. david november , at : pm i think this is a clear effect of google pushing for the encrypted web. with their funding of let’s encrypt, if is free for anyone to obtain a trusted cert for any domain. browsers need to inform users that a green lock icon does not mean they are accessing the site they think they are. janne koschinski november , at : pm regarding punycode display of idn domain names: you can’t just forcefully show all websites as punycode, as there are some legitimate sites which only have idn names, such as flüge.de. the only alternative url that’d be usable would be fluege.de, but that’s already in use by a competitor. as result flüge.de has no alternative. and if you’d force all urls to show as punycode, users would soon get used to xn--bcher-kva.de or xn--flge- ra.de, and would stop paying any attention whatsoever to urls (and honestly, punycode urls always look like phishing domains anyway). sovyett november , at : pm god bless russia and putin. god bless carders fraudsters, hustlers, moneymakers. god bless africa harry stoner november , at : pm for firefox in about:config i set property network.idn_show_punycode to “true”. i think this keeps you from being deceived. bill december , at : am firefox just posted an article titled “ essential tips for safe online shopping”. tip no. reads in part: …”a green lock at the top corner of the url bar means the site is encrypted…”. as we know from this article, that’s absolutely untrue. brian, perhaps you can reach out to mozilla/firefox and let them know the truth about the green padlock, and suggest they issue a correction. i couldn’t find a good way to contact them. url: https://blog.mozilla.org/firefox/ -tips-for-safe-online-shopping/?utm_source=newsletter&utm_medium=email&utm_campaign=firefox- - - &utm_content=text-link chris december , at : am unfortunately, you’re making the same mistake that the article is warning you about: confusing “encrypted” with “trustworthy”. the firefox article is correct in saying that the site is encrypted. it then even goes on: “either way, make sure the site you’re on is trustworthy.” jimbob december , at : pm wow, did you even read brian’s article at all? dev december , at : am this is why dv ssl should be forbidden or at least that sites with dv should have a red padlock or something. today a normal user can’t see the difference between an organization validated certificate and a dv issued one.. jon smith december , at : pm the article and comments provide good awareness that the green lock does not indicate that a website is safe and the punycode configuration is a good step. does anyone have suggestions for verifying the trustworthiness of a site other than google’s “deceptive site ahead” or “the site ahead contains malware” warning? is there a plugin that would help? should the ssl cert be examined each time you visit a site and if so what are some red flags? acorn december , at : am https://krebsonsecurity.com/ / /how-to-shop-online-like-a-security-pro/comment-page- /#comment- i don’t know much about certs. however, i upgrade to more recent encryption levels. jon smith december , at : pm thank you – very helpful. jon smith december , at : pm thank you – very helpful. i was hoping there would be a plugin or something that would handle most of the manual querying but i guess that’s where google’s warnings come into play. i guess manual queries are fine because we all typically visit sites with which we’re already comfortable. acorn december , at : pm right. and, antivirus products help some too. i don’t use av; but on occassion do online lookups. qn december , at : pm might be worth outlining that ev certificates do provide some level of authenticity to the website. billy december , at : am is the brian krebs who supposedly authored this page ( http:// . . . ), the same brian krebs that authored this one? the address was included in some half-baked hacking attempt on my site and i arrived here during my investigations. ← older comments advertisement mailing list subscribe here recent posts facebook, instagram, tiktok and twitter target resellers of hacked accounts ‘validcc,’ a major payment card bazaar and looter of e-commerce sites, shuttered u.k. arrest in ‘sms bandits’ phishing service the taxman cometh for id theft victims arrest, seizures tied to netwalker ransomware all about skimmers click image for my skimmer series. spam nation a new york times bestseller! the value of a hacked pc badguy uses for your pc tools for a safer pc tools for a safer pc the pharma wars spammers duke it out badguy uses for your email your email account may be worth far more than you imagine. ebanking best practices ebanking best practices for businesses most popular posts sextortion scam uses recipient's hacked passwords ( ) online cheating site ashleymadison hacked ( ) sources: target investigating data breach ( ) trump fires security chief christopher krebs ( ) cards stolen in target breach flood underground markets ( ) reports: liberty reserve founder arrested, site shuttered ( ) was the ashley madison database leaked? ( ) true goodbye: 'using truecrypt is not secure' ( ) ddos-guard to forfeit internet space occupied by parler ( ) who hacked ashley madison? ( ) category: web fraud . innovations from the underground id protection services examined is antivirus dead? the reasons for its decline the growing tax fraud menace file 'em before the bad guys can inside a carding shop a crash course in carding. beware social security fraud sign up, or be signed up! how was your card stolen? finding out is not so easy. krebs’s rules… ...for online safety. © krebs on security. powered by wordpress. privacy policy meet meet real-time meetings by google. using your browser, share your video, desktop, and presentations with teammates and customers. brown university library digital technologies brown university library digital technologies bundler . . and homeless accounts this week we upgraded a couple of our applications to ruby . and bundler . . and one of the changes that we noticed was that bundler was complaining about not being able to write to the /opt/local directory. turns out this problem shows up because the account that we use to run our application is &# ; continue reading bundler . . and homeless accounts upgrading from solr to solr a few weeks ago we upgraded the version of solr that we use in our discovery layer, we went from solr . to solr . . although we have been using solr .x in other areas of the library this was a significant upgrade for us because searching is the raison d&# ;être of our discovery layer &# ; continue reading upgrading from solr to solr pypi packages recently, we published two python packages to pypi: bdrxml and bdrcmodels. no one else is using those packages, as far as i know, and it takes some effort to put them up there, but there are benefits from publishing them. putting a package on pypi makes it easier for other code we package up to &# ; continue reading pypi packages new riamco website a few days ago we released a new version of the rhode island archival and manuscript collections online (riamco) website. the new version is a brand new codebase. this post describes a few of the new features that we implemented as part of the rewrite and how we designed the system to support them. the &# ; continue reading new riamco website deploying with shiv i recently watched a talk called &# ;containerless django &# ; deploying without docker&# ;, by peter baumgartner. peter lists some benefits of docker: that it gives you a pipeline for getting code tested and deployed, the container adds some security to the app, state can be isolated in the container, and it lets you run the exact &# ; continue reading deploying with shiv checksums in the bdr, we calculate checksums automatically on ingest (fedora provides that functionality for us), so all new content binaries going into the bdr get a checksum, which we can go back and check later as needed. we can also pass checksums into the bdr api, and then we verify that fedora calculates the &# ; continue reading checksums exporting django data we recently had a couple cases where we wanted to dump the data out of a django database. in the first case (&# ;tracker&# ;), we were shutting down a legacy application, but needed to preserve the data in a different form for users. in the second case (&# ;deposits&# ;), we were backing up some obsolete data before &# ; continue reading exporting django data searching for hierarchical data in solr recently i had to index a dataset into solr in which the original items had a hierarchical relationship among them. in processing this data i took some time to look into the ancestor_path and descendent_path features that solr provides out of the box and see if and how they could help to issue searches based &# ; continue reading searching for hierarchical data in solr monitoring passenger’s requests in queue over time as i mentioned in a previous post we use phusion passenger as the application server to host our ruby applications. a while ago upon the recommendation of my coworker ben cail i created a cron job that calls passenger-status every minutes to log the status of passenger in our servers. below is a sample &# ; continue reading monitoring passenger&# ;s requests in queue over time looking at the oxford common filesystem layout (ocfl) currently, the bdr contains about tb of content. the storage layer is fedora , and the data is stored internally by fedora (instead of being stored externally). however, fedora is end-of-life. this means that we either maintain it ourselves, or migrate to something else. however, we don&# ;t want to migrate tb, and then have &# ; continue reading looking at the oxford common filesystem layout (ocfl) safety insights, data privacy, and spatial justice – the city is here for you to use skip to content the city is here for you to use safety insights, data privacy, and spatial justice posted bymita williams january , february , comments on safety insights, data privacy, and spatial justice i am a supporter of openmedia which is “a community-driven organization that works to keep the internet open, affordable, and surveillance-free”. this morning, i filled out the openmedia privacy act survey which has been adapted from the canadian department of justice’s -question survey (open until february th). here are some questions that it asks: . to what extent do you agree or disagree with the following: i have a right to know if federal government departments or agencies are using artificial intelligence to make a decision that affects me. . to what extent do you agree or disagree with the following: i should have a right to request human involvement in a decision-making process about me that relies on computerized automated processes, such as artificial intelligence. . to what extent do you agree or disagree with the following: a federal government department or agency should be free to collect, use and share personal information that is readily available to the public, including information on a social media website. . how comfortable are you with a federal government department or agency sharing your personal information without your explicit permission with the following entities in circumstances where doing so would help carry out the same purpose the information was originally collected for? d) with a private sector for-profit business. . how comfortable are you with federal government departments and agencies sharing your personal information without your explicit permission with the following entities to achieve a purpose that is different from what the information was originally collected for? d) with a private sector for-profit business. then, later on this morning, i thought i would dive a little deeper into the recent agreement between ford smart mobility canada (the other fordnation) and the city of windsor: the city of windsor is the first municipality in canada to sign an agreement with ford smart mobility canada company to access ford mobility’s safety insights platform. ford mobility is a business line within the ford motor company that works with cities to better understand their unique challenges and then design targeted solutions that help improve the quality of life for residents. ford mobility’s award-winning safety insights tool enables cities like windsor to streamline what can be a costly and time-consuming process of accessing and analyzing transportation data. by integrating this data into safety insights and supplementing it with simulations and solutions based on industry-standard best practices, cities can spend fewer resources on crunching data and focus on helping to improve the safety of their streets. windsor the first canadian city with ford safety insights platform, january , like, frazier fathers, my first thoughts were that the city of windsor already has both the data pertaining to car crashes and a gis team that already has the ability to make inferences from that data. given that the platform is really just an online dashboard it certainly seems like the weedc is paying a lot for insights that they could be or already are being generated. i am curious how many man hours will be saved by this investment of $ , for year. in the city released a report on the most accident prone intersections. this report on potential red light camera locations at the most dangerous intersections in the city. a few thoughts on week , january , . curious what possible insights ford’s safety insight product might deliver, i watched its : promo video. here are some screenshots. the first is of the system’s crash layer. what can we glean from this? well, it appears that in order for that type of information to be available to the city of windsor, the city of windsor is going to hand over the data that will identify what crash factors that has been recorded for each car crash to ford. this next screenshot caught my attention. how does ford know the volume of pedestrians walking down these streets? then i looked up what aadt stood for and learned that these are annual average daily traffic counts. unless ford is buying cell phone location data from brokers, this information might be coming from the city of detroit. addendum: this next slide gave me pause. it looks like the purpose of the safety insights dashboard to is generate calculations of the cost of future accidents versus the cost savings from investing in a countermeasure. i would like to believe that the human cost of a car accident is included in this calculation. i certainly hope so. this is my favourite screen capture of them all. ford safety insights is powered by machine learning. combining data provided by the city, fsip uses artificial intelligence (ai), machine learning as well as industry-standard algorithms to reveal crash-reduction predictions. the results, said the fsip reps, are actionable. smart city: windsor is first canadian city to launch ford safety insights platform to reduce crashes, n.f. mendoza, tech republic, january , , so at the very moment in which the federal government is asking canadians how we feel about handing over personal data to corporations and asking how we feel about artificial intelligence making decisions that may affect our safety, the city of windsor announces this finished agreement, without any consultation from either residents or city council. does this mean that the decision whether there has been enough near-fatal accidents to have occurred at the intersection at your child’s school is enough to justify a stop sign will be made by an algorithm that has been trained been on american drivers, some driving michigan lefts? does this mean that in future, the city of windsor can hand over some or all of its data pertaining to residents to any private company in exchange for a discount of that company’s product? i have yet to see any evidence that ford’s safety insights system can deliver what it promises and do better than what our city staff can already provide. is this project innovation or outsourcing? shouldn’t the city of windsor first develop its own policy on how it will use machine learning in civic decision making and in doing so, ensure that there is also some form of human intervention in the cases that the machine has learned incorrectly or has embodied the structural biases from its training data? these are not unreasonable questions to ask. there are historical precedents to these concerns. for example, early attempts at using geospatial data to make firefighting more cost-effective that started in directly resulted to the death of thousands in new york city over the next decade. this map was also published in : in her subsequent analysis, warren uses this map as a jumping off point to discuss spatial justice more generally for the black community. for example, most african-americans work in the factories which are situated several hours from their community so they leave for work at or am because the buses only run once an hour. those coming from the black community who do have cars are unable to get on the expressway between the hours of - pm due to the timing of the stoplights. she uses these examples and others to unequivocally demonstrate that detroit’s urban planning and transportation is inadequate and unjust for the black community and calls for the dgei to establish “black planning” for the city of detroit where commuters run over black children, detroit , june , , alex b. hill it is safe to say that every person has had someone close to them either die in a car accident or know someone who has had someone beloved die in a car accident as such, significant improvements to road safety should be greatly received and celebrated. unfortunately, the administration of the city of windsor seems oddly slow to adopt practices that been proven greatly reduce the number of deaths such as reducing residential speed limits or find ways to establish protected bike lanes. it makes you wonder if safety isn’t really their number one concern. share this: twitter facebook posted bymita williamsjanuary , february , posted incity, council, technology post navigation previous post previous post: weeknote , next post next post: weeknote , replies on “safety insights, data privacy, and spatial justice” pingback: weeknote ( ) – librarian of things pingback: weeknote , – the city is here for you to use comments are closed. pages about flooding in windsor essex police officer charged… the city of windsor’s budget the future we want don’t have an rss reader? subscribe to posts by email recent posts weeknote , safety insights, data privacy, and spatial justice weeknote , weeknote , weeknote , meta log in entries feed comments feed wordpress.org the city is here for you to use, proudly powered by wordpress. happy will ferrell gif - find & share on giphy github - hughrun/public_library_map: interactive geodata about public libraries in australia skip to content sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories → security → team enterprise explore explore github → learn & contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → nonprofit → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this user all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up {{ message }} hughrun / public_library_map watch star fork interactive geodata about public libraries in australia librarymap.hugh.run stars forks star watch code issues pull requests actions security insights more code issues pull requests actions security insights main branch tags go to file code clone https github cli use git or checkout with svn using the web url. work fast with our official cli. learn more. open with github desktop download zip launching github desktop if nothing happens, download github desktop and try again. go back launching github desktop if nothing happens, download github desktop and try again. go back launching xcode if nothing happens, download xcode and try again. go back launching visual studio if nothing happens, download the github extension for visual studio and try again. go back latest commit hughrun merge pull request # from hughrun/labels … e feb , merge pull request # from hughrun/labels rename labeller action e git stats commits files permalink failed to load latest commit information. type name latest commit message commit time .github rename labeller action feb , website merge csv data to topo feb , .gitignore updated gitignore feb , readme.md update docs & website jan , about.md update docs & website jan , boundaries.geo.json update topojson boundaries feb , contributing.md fix typos in contributing.md feb , contributors.txt update contributors feb , sources.md update docs & website jan , view code readme.md public library data map this project collects and maps data from public libraries across the australia and the external territories of christmas island and the cocos (keeling) islands. you should be able to find the location of every public library in australia territory, plus the standard loan period for each libary service, and whether they charge overdue fines (if known, in both cases). check out the map at librarymap.hugh.run. find more on this project at about. attributions see sources. want to help? see contributing, especially which files to update. contributors contributors (whether via github or another way) are listed at contributors.txt. licenses /website/data: cc-by-sa everything else: gpl- . -or-later about interactive geodata about public libraries in australia librarymap.hugh.run topics geodata public-libraries resources readme releases no releases published contributors languages html . % javascript . % css . % python . % © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. none none none job description template job description job title: bibliometrics and research impact librarian department: library reports to: associate university librarian, collections, technology, scholarly communication jobs reporting: none salary grade: usg - effective date: september primary purpose the bibliometrics and research impact librarian is accountable for developing, advancing, implementing and supporting a range of services around scholarly output, research impact and bibliometrics for the university of waterloo community. the incumbent works in close collaboration with institutional analysis and planning (iap), the office of research (or), and colleagues within the library to develop and support services around measurement of research impact for the campus, with a focus on bibliometrics. the librarian works with multiple stakeholders, including senior leadership, to establish and negotiate work priorities and deadlines based on institutional priorities. key accountabilities management of bibliometrics services and projects  lead and promote the library’s bibliometric services  prioritize, evaluate, negotiate and manage requests for bibliometric data and support  ensure timely and diplomatic communication, through appropriate channels, of developments and projects; communicate on any related events having potential to directly or indirectly impact normal services  lead and coordinate projects with the primary goal being to ensure best possible service  work closely with associate university librarian, collections, technology, scholarly communication and campus partners to develop policies, manage change, set future directions, and resolve problems outreach and education  work with faculties, departments, research centres and institutes, and others to understand appropriate use of bibliometrics and other tools that generate research impact measures, grounded in the university’s white paper on bibliometrics  serve as a resource to the university community, including library staff, on bibliometrics and research impact  monitor and disseminate literature on bibliometrics and research impact  provide individual and group training on bibliometrics software  design, develop and maintain effective online and print content related to bibliometrics and research impact  design and deliver instructional programs and materials related to bibliometrics and research impact  act as a resource person/expert consultant and train liaison librarians, faculty and graduate students in the tools and techniques used to identify and maintain accurate personal citation metrics, and provide advice with respect to individual bibliometrics, researcher profiling tools and other methods of measuring impact job description  serve on committees and groups, fostering collaboration, information sharing, partnership and expertise across campus and the broader community manage bibliometrics software for campus  manage bibliometrics software on behalf of campus  provide end-user support to faculty, students, and staff for bibliometrics software  manage positive relationships with software vendors and arrange for technical support and advanced training  work closely with software and data vendors to solve technical issues around research problems  work closely with the working group on bibliometrics (wgob) and others as appropriate with respect to the planning, implementation and ongoing support of the use bibliometrics software and appropriate data on campus  monitor the marketplace for advances in bibliometrics tools data analysis and management  advance operational processes that ensure the university’s publications are maximally attributed to the university of waterloo in bibliometric data sources  serve as a resource for groups on campus looking at research impact of specific subject areas  consult on best practices in working with bibliometric data  explore and answer methodological questions related to bibliometrics  extract and analyze bibliometrics data in support of strategic requests from leadership, including strategic planning, ranking activity, and other research impact reporting  develop and maintain an awareness of the trends and issues in university level rankings, such as qs and the, and research impact in order to:  analyze the bibliometric data of university ranking organizations  respond to questions, in collaboration with iap, about methodologies used by ranking organizations  analyse trends and patterns over time using benchmarks and comparative data in consultation with institutional analysts  prepare standard and customized reports and presentations to support research impact questions at the institutional and developmental levels  create and maintain data dictionaries functional direction and guidance  oversee and provide direction, guidance and mentorship to co-op students  provide functional direction to staff that may participate on projects the incumbent leads  participate in performance discussions and reviews as appropriate  participate and provide feedback on hiring committees professional development  connect with professional organizations in their area of expertise and actively contribute  actively participate on specialized teams or working groups to support the unique needs and strategic goals at the university of waterloo  ongoing, proactive acquisition, maintenance, and provision of leadership in the development of associated skills and knowledge in areas of expected expertise *all employees of the university are expected to follow university and departmental health and safety policy, procedures and work practices at all times. employees are also responsible for the completion of all health and safety training, as assigned. employees with staff supervision and/or management responsibilities will ensure that assigned staff abide by the above, and actively identify, assess and correct health and safety hazards, as required. required qualifications education  ala-accredited master of library & information science degree, or equivalent job description experience  demonstrated experience with different bibliometric data sources having citation tracking capabilities (e.g. incites, scival, web of science, scopus, google scholar etc.)  demonstrated experience with statistical analysis methods associated with bibliometrics, including data management, manipulation, interpretation and analysis skills  experience working with time-sensitive deadlines  asset: experience with data analysis on large data sets knowledge/skills/abilities  excellent communication and diplomacy skills  strong written and verbal skills for teaching, troubleshooting, procedural documentation and report writing  knowledge of and/or experience with bibliometric software (e.g. incites, scival etc.)  knowledge of basic and derivative bibliometrics (publication counts, citation counts, h-index, journal or discipline normalized metrics, other index metrics)  strong data literacy  understanding of the issues related to responsible research evaluation via bibliometric data; conversant with issues related to the quantification of research impact  ability to take initiative, create opportunities, and develop effective partnerships  knowledge of issues related to researcher identification and experience using researcher profiling systems (e.g. orcid, researcher id, etc.) nature and scope  contacts: : internally, communicates with employees in all groups and departments throughout the university community and at all levels to gather ideas, envision, articulate, update and inform on projects for which the incumbent is leading or otherwise accountable. externally, communicates frequently with vendors and other staff in order to execute work  level of responsibility: the position is responsible and accountable for leading projects and other work. the position may have supervisory responsibility for work-study, co-op or contract positions.  decision-making authority: : the incumbent is responsible for advancing, building, maintaining and providing services for bibibiometrics and research impact, and works independently and with others, as appropriate. as required, the position consults with the associate university librarian, members of the library managers group, and others across campus  physical and sensory demands: minimal exposure to disagreeable conditions typical of an office position.  working environment: minimal exposure to disagreeable conditions typical of an office position exposed to stress and pressure associated with those responsibilities none none public_library_map/public_library_locations.csv at main · hughrun/public_library_map · github skip to content sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories → security → team enterprise explore explore github → learn & contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → nonprofit → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this user all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up {{ message }} hughrun / public_library_map watch star fork code issues pull requests actions security insights more code issues pull requests actions security insights permalink main public_library_map/website/data/public_library_locations.csv go to file go to file t go to line l copy path cannot retrieve contributors at this time lines ( sloc) kb raw blame lat lng town address phone - . . victor harbor public library service bay road - . . lameroo school community library bews terrace - . . hamra centre library brooker terrace - . . strathalbyn branch colman terrace - . . park holme branch duncan avenue - . . greenock depot greenock road - . . enfield branch kensington crescent - . . cove civic centre branch ragamuffin drive - . . elizabeth (playford civic) branch playford boulevard - . . goodwood branch goodwood road - . . st peters branch payneham road - . . goolwa branch cadell street - . . aldinga branch central way - . . norwood branch the parade - . . port broughton area school community library east terrace - . . mount pleasant branch - melrose street - . . hindmarsh branch port road - . . flinders mobile library service fifth street (truck ) - . . ardrossan school community library second street - . . semaphore branch semaphore road - . . mitcham branch belair road - . . naracoorte public library service smith street - . . campbelltown public library service montacute road - . . south west community centre branch sturt street - . . woodcroft branch bains road - . . tynte st north adelaide branch tynte street - . . unley civic branch unley road - . . saddleworth library & community centre branch belvidere road - . . kadina library a doswell terrace - . . waikerie branch a strangman road - . . glenelg branch colley terrace - . . greenacres branch fosters road - . . port lincoln library service london street - . . minlaton school community library north terrace - . . eudunda school community library reserve road - . . payneham branch turner street - . . port adelaide branch - church street - . . mawson lakes branch - main street - . . brighton branch jetty road - . . blackwood branch main road - . . hutt st branch hutt street - . . loxton branch a east tce - . . woodside branch onkaparinga valley road - . . cultural centre library (oaklands park) diagonal road - . . lyndoch branch barossa valley way - . . mallala branch a wasleys road - . . port pirie branch wandearah road - . . stretton centre branch peachey road - . . clare branch old north road - . . community hub branch church street - . . henley beach branch seaview road - . . barmera public library service barwell avenue - . . port augusta library service mackay street - . . burnside library service greenhill rd - . . fullarton park depot fullarton road - . . penola school community library cameron street - . . kingscote branch dauncey street - . . gawler administration centre branch high street - . . nuriootpa branch - tanunda road - . . gumeracha branch albert street - . . kingston community school library east terrace / - . . parks branch trafford st - . . port macdonnell (mary lattin memorial) library service charles street - . . mount barker community library dumas street - . . wallaroo library john terrace - . . kapunda branch – main street - . . bordertown public library service woolshed street - . . tea tree gully library service montague road - . . auburn branch st vincent street - . . mount gambier library service watson terrace - . . two wells branch old port wakefield road - . . coventry stirling branch mt barker road - . . evanston gardens branch angle vale road - . . walkerville public library service walkerville terrace - . . tanunda branch - murray street - . . kimba school community library west terrace - . . burra school community library bridge terrace - . . freeling depot hanson street - . . whyalla public library service - ekblom street - . . civic woodville branch woodville road - . . angaston branch angaston town hall - . . hawker school community library arkaba street - . . mt compass library kiosk australia post lpo compass central - . . balaklava community library balaklava high school - . . ingle farm branch beovich road - . . berri library service berri library and information centre - . . leigh creek school community library black oak drive - . . moonta library blanche terrace - . . crystal brook branch bowman street - . . pinnaroo school community library bundey terrace - . . bute depot bute post office - . . cambrai school community library cambrai primary school - . . ceduna school community library ceduna area school - . . central yorke peninsula (maitland) community library central yorke school - . . cleve school community library cleve area school - . . west lakes branch cnr brebner dr & west lakes blvd - . . milang depot cnr coxe street & ameroo avenue - . . coffin bay depot coffin bay hall - . . coober pedy school community library coober pedy area school - . . coomandook school community library coomandook area school - . . cowell school community library cowell area school - . . roxby downs community library service cultural and leisure precinct - . . cummins school community library cummins area school - . . minlaton library curramulka depot curramulka institute - . . elliston depot elliston visitor centre - . . findon branch findon shopping centre - . . geranium community library depot geranium terrace - . . andamooka school community library government road - . . seaford branch grand boulevard - . . noarlunga branch hannah road - . . salisbury west branch hollywood boulevard - . . aberfoyle park (the hub) branch hub drive - . . jamestown school community library humphris terrace - . . renmark paringa public library service james avenue - . . penneshaw depot kangaroo island community education - . . penneshaw depot kangaroo island community education - . . murray bridge library service level , market place - . . city library (rundle place) branch level , rundle place - . . lock school community library lock area school / (school) - . . lucindale school community library lucindale area school - . . mannum school community library mannum community college - . . meningie school community library meningie area school - . . beachport library millicent road - . . morgan library morgan & districts community hub - . . robe public library service mundy terrace - . . gawler sport and community centre branch nixon terrace - . . karoonda school community library north terrace - . . orroroo school community library orroroo area school - . . riverton and district high school library oxford terrace - . . prospect public library service payinthi - . . karcultaby school community library phillips road - . . minlaton library port vincent depot port vincent institute - . . peterborough school and community library service queen street - . . quorn school community library quorn area school - . . millicent library ridge terrace - . . snowtown school community library snowtown primary school - . . willunga branch st peter's terrace - . . streaky bay school community library streaky bay area school - . . swan reach community library swan reach area school - . . tailem bend school community library tailem bend primary school / - . . yankalilla library service the centre - . . pt elliot depot the strand - . . pt elliot depot the strand - . . tintinara coonalpyn school community library tintinara area school - . . keith community library tolmer terrace - . . tumby bay school community library tumby bay area school - . . para hills branch wilkinson road - . . woomera community library service woomera area school - . . wudinna school community library wudinna area school - . . yorketown school community library yorketown area school - . . bloomfield library - - . . citylibraries home service - - . . citylibraries mobile library - - . . gold coast mobile library - - . . ipswich mobile library service - - . . mobile central route (mobi ) - - . . mobile library - - - . . mobile southern route (mobi ) - - . . moreton bay region mobile library - - . . noosa mobile library - - . . redland mobile library - - . . scenic rim regional council mobile library - - . . toowoomba region mobile library - - - . . brisbane mobile library services view mobile library stops and calendar " - . . south hurstville library allen street - . . regents park library amy street - . . point cook cheetham street - . . chelsea chelsea road chelsea - . . thornlie public library culross avenue - . . whitemark davies street, whitemark ( ) - . . tammin public library donnan street - . . woodford library elizabeth street - . . moe george street - . . cottesloe/peppermint grove/mosman park public library leake street - . . narembeen public library longhurst street - . . manly library market place - . . currie meech street, currie ( ) - . . brighton le sands library moate avenue - . . mundaring public library nichol street - . . ryde library pope street (cnr pope and devlin streets, within top ryde city shopping centre) - . . baulkham hills library railway street - . . eaglehawk sailors gully rd - . . huon skinner drive, huonville ( ) - . . strathpine library station road - . . sydenham (watergardens) station street, taylors lakes - . . auburn library susan street - . . bribie island library welsby parade - . . melville civic square public library almondbury road - . . bateau bay village library bay village road - . . nowra library berry street - . . millaa millaa library main street - . . mundijong public library paterson street - . . nebo library reynolds street - . . jeparit roy street - . . sea lake sutcliffe street - . . north lakes library the corso - . . murwillumbah library tumbulgum road - . . nightcliff - pavonia place, nightcliff nt - . . bonnyrigg library bonnyrigg avenue - . . corowa library edward street - . . newman community library kalgan drive - . . goondiwindi library marshall street - . . mortlake dunlop street mortlake - . . beechworth ford street - . . dimboola lloyd street dimboola - . . ferntree gully burwood highway ferntree gully - . . joondalup public library boas avenue - . . karingal place ashleigh ave, frankston - . . williamstown ferguson street williamstown - . . mayfield library hanbury street - . . springwood library macquarie road - . . sans souci library russell avenue - . . box hill whitehorse road box hill - . . hamilton brown street hamilton - . . mount claremont public library montgomery avenue - . . longreach library eagle street - . . wee waa library rose street - . . braybrook churchill ave - . . docklands (library at the dock) harbour promenade - . . coopers plains library orange grove road - . . burdekin library graham street - . . barraba library queen street - . . blackall library shamrock street - . . somerville frankston flinders road somerville - . . gulgong library herbert street - . . broadmeadows pascoe vale rd the age library broadmeadows - . . geeveston church street, geeveston ( ) - . . rainbow federal street - . . brunswick heads library fingal street - . . arncliffe library firth street - . . kondinin public library gordon street - . . kingston hutchins street, kingston ( ) - . . stratford library kamerunga road - . . collingwood stanton street abbotsford collingwood - . . jabiru tasman cres, jabiru nt - . . lismore library magellan street - . . glen waverley kingsway glen waverley - . . st george library victoria street - . . latrobe gilbert street, latrobe ( ) - . . broadford high street - . . engadine library e caldarra avenue - . . gosford library donnison street - . . glenquarie library brooks street - . . belconnen lbrary chandler street, belconnen - . . chester hill library & knowledge centre chester hill road - . . george town community hub elizabeth street, george town ( ) - . . canungra library kidston street - . . proserpine library main street - . . hyden public library mcpherson street - . . cheltenham stanley avenue cheltenham - . . kilmore sydney st kilmore - . . boyanup public library thomas street - . . malabar community library anzac parade - . . torres shire ngulaig meta municipal library douglas street - . . burpengary library station road - . . fairfield station street fairfield - . . east melbourne george st east melbourne - . . avoca main st avoca - . . altona queen street altona - . . imbil library - yabba road - . . seymour anzac avenue seymour - . . heathcote high street heathcote - . . malvern high street malvern - . . muswellbrook library bridge street - . . grafton library - pound street - . . maryborough library - bazaar street - . . fitzroy moor street fitzroy - . . coolamon library cowabbie street - . . trentham albert street - . . wynyard exhibition link, wynyard ( ) - . . bedourie library herbert street - . . newport mason street newport - . . dowerin public library stewart street - . . condobolin library bathurst street - . . charters towers excelsior library gill street - . . scone library liverpool street - . . padstow library & knowledge centre cahors road - . . elliott lewis street, elliott nt - . . smithton - nelson street, smithton ( ) - . . dudley denny city library victoria street - . . terang high street terang - . . tenterfield library manners street - . . tea gardens library marine parade - . . new farm library sydney street - . . drouin princes way drouin - . . devonport rooke street, devonport ( ) - . . artarmon rd - . . speers point library main road - . . bright ireland street bright - . . welshpool main street - . . ringarooma main street, ringarooma ( ) - . . mossman library mill street - . . cobram punt road - . . brighton wilson street brighton - . . campsie library & knowledge centre - amy street - . . cranbrook public library climie street - . . wynnum library florence street - . . greenbank library - teviot road - . . oakleigh drummond street oakleigh - . . kilcoy library kennedy street - . . ulverstone king edward street, ulverstone ( ) - . . newcastle city library laman street - . . halifax sub branch - hinchinbrook shire library macrossan street - . . mukinbudin public library maddock street - . . high wycombe public library markham road - . . corrimal district library short street - . . st kilda carlisle street st kilda - . . maffra johnson street maffra - . . tarneit (julia gillard) sunset views boulevard - . . city library abbott street - . . eastgardens library bunnerong road - . . mungindi library st george street - . . geelong west b pakington street geelong west - . . north rockhampton library berserker street - . . ashburton high street ashburton - . . local history library herries street - . . toowoomba city library herries street - . . yarram grant street yarram - . . lithgow library learning centre main street - . . inverloch a'beckett street - . . albany creek library ferguson street - . . cunnamulla library john street - . . carmila library music street - . . atherton library robert street - . . leonora public library tower street - . . mirani library victoria street - . . bentleigh jasper road bentleigh - . . hervey bay library old maryborough road - . . belmont high street belmont - . . bombala library maybe street - . . korumburra commercial street korumburra - . . ascot vale union road ascot vale - . . st ives library mona vale road - . . borroloola robinson road, borroloola nt - . . warrandyte yarra street - . . tumut library wynyard street - . . casuarina bradshaw terrace, casuarina nt - . . fairfield library kenyon street - . . dungog library mackay street - . . baldivis library and community centre settlers avenue - . . liverpool city library george street - . . colac queen street - . . edmonton library bruce highway - . . werribee watton street werribee cbd - . . ballarat doveton street n ballarat - . . west end library - boundary street - . . ringwood (realm) maroondah high, ringwood - . . green valley library - wilson road - . . bowral central library bendooley street - . . carnarvon public library egan street - . . numurkah mccaskill street - . . lakes entrance mechanics st - . . pingelly public library parade street - . . poowong ranceby road poowong - . . wandering public library watts street - . . drysdale - hancock st drysdale - . . kuranda library - arara street - . . jimboomba library - honora street - . . mildura deakin avenue mildura - . . north fitzroy (bargoonga nganjin) st georges road north fitzroy - . . goulburn mulwaree library - bourke street - . . glebe library glebe point road - . . raymond terrace library a sturgeon street - . . bothwell alexander street, bothwell ( ) - . . belmont library ernest street - . . esk library heap street - . . kinglake whittlesea road - . . quirindi library george street - . . caroline springs - caroline springs blvd - . . emerald hill (south melbourne) bank street emerald hill - . . safety bay public library safety bay road - . . northampton public library hampton road - . . gingin public library a lily king place - . . hampton d service street hampton - . . wongan hills public library elphin cresent - . . deloraine - emu bay road, deloraine ( ) - . . rosedale cansick street - . . balwyn north (greythorn library lounge) centre way, balwyn north - . . burwood library conder street - . . benalla fawckner drive - . . west ryde library graf avenue - . . canning bridge public library kintail road - . . wentworthville library lane street - . . altona meadows newham way altona meadows - . . narre warren (bunjil place) patrick north east drive ( ) - . . leongatha smith street leongatha - . . forbes library victoria lane - . . millmerran library - campbell street - . . armidale war memorial library / rusden street - . . muttaburra library bruford street - . . marble bar public library francis street - . . constitution hill library hollis street - . . windale library lake street - . . leeman public library morcombe road - . . eaton public library recreation drive - . . isisford library st mary street / - . . wingham library wynter street - . . maribyrnong (highpoint shopping centre) rosamond road highpoint shopping centre - . . north ryde library coxs road - . . carrum downs lyrebird drive carrum downs - . . southbank (boyd) city road - . . st marys library - queen street - . . wangaratta docker street wangaratta - . . lesmurdie school community library reid road - . . mirrabooka public library sudbury road - . . manilla library manilla street - . . lake bolac glenelg highway - . . the entrance library and council services a the entrance road - . . castlemaine barker street castlemaine - . . camperdown manifold street camperdown - . . irymple fifteenth street irymple - . . belmont public library wright street - . . merredin public library coronation street - . . russell island library high street - . . south west rocks library landsborough street - . . ilfracombe library main avenue - . . mount garnet library opal street - . . st lawrence library railway parade - . . bairnsdale service street bairnsdale - . . hampton park - stuart avenue hampton park ( ) - . . brunswick west (campbell turnbull library) melville road campbell turnbull library, brunswick west - . . mareeba library byrnes street - . . finley library murray st - . . dandenong lonsdale st - . . swansea library pacific highway - . . reservoir edwardes street - . . bridgewater green point road, bridgewater (behind brighton civic centre) ( ) - . . mathoura library moama street - . . leichhardt library norton street - . . nambucca heads library ridge street - . . mount isa city library west street - . . cohuna - king edward street cohuna - . . rockhampton regional library bolsover street - . . waurn ponds pioneer road - . . healesville maroondah highway healesville - . . griffith city library - banna avenue - . . stanton library (north sydney) miller street - . . subiaco public library rokeby road - . . ravenshoe library & customer service centre moore street - . . babinda library munro street - . . bexley north library shaw street - . . bexley north library shaw street - . . wickepin public library wogolin road - . . taroom library yaldwyn street - . . nyabing public library - richmond street - . . gundagai library sheridan street - . . broken hill city library - blende street - . . paddington library oxford street - . . sorell cole street, sorell ( ) - . . bayswater public library king william street - . . warrnambool liebig street warrnambool - . . barham noorong street - . . beerwah library peachester road - . . bannockburn - high street - . . melbourne (city library) flinders lane melbourne city library - . . ivanhoe upper heidelberg road ivanhoe - . . northbridge library sailors bay road - . . bendigo hargreaves street bendigo - . . oatley library letitia street - . . logan central library wilbur street - . . lindfield library pacific highway - . . preston gower street preston - . . kangaroo flat high street kangaroo flat - . . darkan public library burrowes street - . . gloucester library denison street - . . springsure library eclipse street - . . cygnet mary street, cygnet ( ) - . . clarkson public library ocean keys boulevard - . . goonellabah library oliver avenue - . . victoria park public library sussex street - . . carlton (kathleen syme) faraday street - . . calingiri public library cavell street - . . horsham mclachlan st horsham - . . hornsby library - george street - . . wycheproof broadway - . . stones corner library logan road - . . kelmscott public library albany highway - . . banyo library st vincents road - . . birdsville library burt street - . . bingara library maitland street - . . richmond library west market street - . . gunnedah shire library conadilly street - . . temora library hoskins street - . . wangi wangi library watkins road - . . lalor a may road lalor - . . mansfield b collopy street - . . kyneton baynton st - . . lidcombe library bridge street - . . kincumber library bungoona road - . . dardanup public library little street - . . wanneroo public library rocca way - . . forrestfield public library salix way - . . bermagui branch library - bunga street - . . burnie alexander street, burnie ( ) - . . wallsend library bunn street - . . orford charles street, orford ( ) - . . mundubbera library lyons street - . . goroke main street goroke - . . kawana library nanyima street - . . katoomba library parke street - . . sutherland library - belmont street - . . sunshine hampshire road sunshine - . . broomehill public library great southern highway - . . customs house library alfred street, circular quay - . . kilkivan library bligh street - . . narromine library dandaloo street - . . tara library day street - . . swansea franklin street, swansea ( ) - . . malanda library & customer service centre james street - . . st marys main street, st marys ( ) - . . melton mckenzie street melton - . . mount morgan library morgan street - . . corrigin public library walton street, corrigin post office - . . miranda library wandella road - . . echuca hare street - . . highett highett road highett - . . marrickville library and pavilion marrickville road - . . flemington racecourse road flemington - . . albert park montague streeet, albert park - . . dingley c marcus road dingley - . . urunga library bonville street - . . port macquarie library grant street (corner grant & gordon streets) - . . yarraville wembley avenue yarraville - . . northcote - separation street northcote - . . claremont public library stirling highway - . . nabawa public library chapman valley road - . . daylesford albert street daylesford - . . cooktown library helen street - . . rushworth high street rushworth - . . blaxland library hope street - . . westbury william street, westbury ( ) - . . mandurah public library pinjarra road - . . woodanilling public library robinson road - . . altone park public library benara road - . . port melbourne bay street port melbourne - . . balwyn whitehorse road balwyn - . . proston library blake street - . . dorothy jones library, tully bryant street - . . perry library heussman street - . . yackandandah high street - . . beresfield library lawson avenue - . . moora public library padbury street - . . traralgon - kay street traralgon - . . merriwa library - vennacher street - . . camberwell camberwell road - . . south yarra (toorak) toorak road toorak/south yarra - . . dandaragan public library dandaragan road - . . local studies / family history library a church street - . . marsden library chambers flat road - . . westall fairbank rd, clayton south - . . goomeri library moore street - . . thirroul district library - lawrence hargrave drive - . . green square library botany road - . . galston library galston road - . . lake cargelligo library a forster stret - . . moree library balo street - . . clunes fraser street clunes - . . mulgrave mackie rd - . . eidsvold library moreton street - . . dulwich hill library - new canterbury road - . . willetton public library burrendah boulevarde - . . collinsville library conway street - . . yalgoo public library gibbons street - . . mitchelton library heliopolis parade - . . geraldton regional library marine terrace - . . western heights college library vines rd - . . balmain library darling street - . . caringbah library - port hacking road - . . nunawading whitehorse road nunawading - . . warialda library hope street - . . nullagine public library gallop road - . . gladstone city library goondoon street - . . flinders shire public library gray street - . . cobar library marshall street - . . morisset library yambo street - . . narrandera library - east street - . . mill park plenty road mill park - . . forster library - breese parade - . . cardwell library balliol street - . . kambalda public library barnes drive - . . gin gin library dear street - . . barooga library golf course road - . . citylibraries aitkenvale petunia street - . . greenmount public library scott street - . . elsternwick staniland grove elsternwick - . . glenorchy terry street, glenorchy ( ) - . . five dock library - garfield street - . . watsonia - ibbottson street watsonia - . . culcairn library balfour street - . . camden library john street - . . wallerawang library main street - . . duncraig public library warwick road - . . ultimo library william henry street - . . emerald b belgrave-gembrook road emerald - . . surry hills library crown street - . . chatswood library victoria avenue - . . miriam vale library blomfield street - . . wollongong central library burrelli street - . . mount waverley miller crescent mount waverley - . . westonia public library wolfram street - . . jerilderie library - jerilderie street - . . shepparton - marungi street shepparton - . . springvale springvale road springvale - . . richmond church street richmond - . . dunedoo library bolaro street - . . heyfield macfarlane street heyfield - . . picton library menangle street - . . murgon library stephens street - . . knox burwood hwy knox - . . walcha library n derby street - . . rochester - mackay street rochester - . . emerald library borilla street - . . perenjori public library fowler street - . . derrinallum main street derrinallum - . . maroochydore library sixth avenue - . . rockdale library - princes highway - . . birchip campbell st - . . midland public library helena street - . . hamilton library james street - . . annerley library ipswich road - . . alstonville library commercial road - . . bourke library mitchell street - . . bassendean public library old perth road - . . tamworth library peel street - . . biggenden library edward street - . . murrurundi library mayne street - . . tin can bay library tin can bay road - . . redcliffe library oxley avenue - . . rosny bligh street, rosny ( ) - . . nanango library drayton street - . . greenwich library greenwich road - . . maitland library high street - . . niddrie keilor road niddrie - . . bundeena library r scarborough street - . . warwick library albion street - . . childers library churchill street - . . edenhope elizabeth street edenhope - . . alexandra grant street alexandra - . . innisfail library rankin street - . . wellstead public library windsor road - . . bundaberg library woondooma street - . . kenilworth library a elizabeth street - . . maleny library coral street - . . edgeworth library minmi road - . . turramurra library ray street - . . gascoyne junction scott street - . . woodvale public library trappers drive - . . lara walkers rd - . . monto library newton street - . . tuggerah library and council services wyong road - . . goomalling public library - railway terrace - . . wauchope library high street - . . scottsdale king street, scottsdale ( ) - . . geelong little malop street geelong - . . smithfield community library - oxford street - . . quilpie shire library brolga street - . . thomastown main street thomastown - . . berrigan library - chanter street - . . lightning ridge library pandora street - . . swan hill - campbell street swan hill - . . west footscray barkly street - . . bruce rock public library johnson street - . . gordon white library phillip street - . . patterson lakes thompsons rd - . . deniliquin library napier street - . . paynesville the esplanade paynesville - . . cobden victoria street cobden - . . longford wellington street, longford ( ) - . . frankland public library wingebellup road - . . inverell shire public library - campbell street - . . julia creek library burke street - . . stanthorpe library lock street - . . footscray paisley street footscray - . . everton park library south pine road - . . wetherill park library - polding street - . . west chatswood library mowbray road - . . brewarrina library bathurst street - . . tocumwal library deniliquin street - . . whittlesea laurel street - . . exeter main road, exeter ( ) - . . cunderdin public library main street - . . chilwell (newtown) russell street chilwell - . . manjimup public library - rose street - . . perth city public library hay street - . . beaudesert library brisbane street - . . clarinda viney street clarinda - . . hawthorn glenferrie road hawthorn - . . coolah library binnia street - . . tullamarine spring street tullamarine - . . avalon community library a old barrenjoey road - . . werris creek library a single street - . . forestville library darley street - . . wandoan library henderson road - . . buronga library midway drive - . . coolum beach library park street - . . girrawheen public library patrick court - . . gladesville library pittwater road - . . ballina library river street - . . queanbeyan library rutledge street - . . concord library flavelle street - . . frankston playne street frankston - . . nedlands public library - stirling highway - . . penrith city library high street - . . mosman library military road - . . st helens cecilia street, st helens ( ) - . . oak flats library central avenue - . . herberton library & customer service centre grace street - . . visability kitchener avenue - . . broadbeach library sunshine boulevard - . . cooma library vale street - . . cootamundra library - wallendoon street - . . bindoon public library great northern highway - . . euroa binney street euroa public library - . . surat library burrowes street - . . oakey library campbell street - . . lakemba library & knowledge centre the boulevard - . . arana hills library cobbity crescent - . . croydon library samwell street - . . morwell - elgin st morwell - . . penshurst library forest road - . . mudgee library market street - . . corinda library oxley road - . . cranbourne berwick-cranbourne road cranbourne ( ) - . . sarina library broad street - . . shark bay public library knight terrace - . . cessnock city library - vincent street - . . north melbourne errol street north melbourne - . . south tamworth library robert street - . . logan hyperdome library - mandew street - . . carlton rathdowne street carlton - . . maroubra (lionel bowen) library - anzac parade - . . casterton henty street casterton - - . . bowen library herbert street - . . riverton public library riley road - . . portland library williwa street - . . doncaster doncaster rd - . . jurien public library bashford street - . . warren library dubbo street - . . logan west library grand plaza drive - . . blackbutt library hart street - . . avondale heights military road, avondale - . . coolgardie public library sylvester street - . . seaford r broughton street - . . gemfields library burridge road - . . tomaree library & community centre community close - . . manildra library derowie street - . . morley public library dewar street - . . hastings high street hastings - . . emu park library and customer service centre hill street - . . denman library ogilvie street - . . kiama municipal library railway parade - . . carnegie shepparson avenue carnegie - . . mooroolbark station street mooroolbark - . . noosaville library wallace drive - . . kalamunda public library williams street - . . sale foster street - . . mallacoota maurice avenue mallacoota - . . stratford tyers street stratford - . . donald woods street - . . bathurst library - keppel street - . . mullumbimby library - station street - . . keilor b old keilor highway - . . launceston civic square, launceston ( ) - . . glen innes severn public & tafe library grey street - . . mulwala library melbourne street - . . agnes water library springs road - . . halls creek public library thomas street - . . st albans a alfrieda street st albans - . . beaufort neill st beaufort - . . dee why library pittwater road - . . hinchinbrook shire library - mcilwraith street - . . greystanes library merrylands road - . . glenroy pascoe vale road glenroy - . . zeehan main street, zeehan ( ) - . . nathalia blake street - . . craigieburn central park avenue - . . lambton library elder street - . . adamstown library victoria street - . . warragul victoria street warragul - . . winton library elderslie street - . . richmond library goldring street - . . moonee ponds - mt alexander road moonee ponds - . . cowra library darling street - . . campbell town high street, campbell town ( ) - . . home hill branch library - ninth avenue - . . waterloo library elizabeth street - . . bicheno burgess street, bicheno ( ) - . . seville grove public library champion drive - . . haberfield library dalhousie street - . . allora library herbert street - . . wondai library mackenzie street - . . grange library evelyn street - . . ouyen oke street - . . molong library – watson street - . . granville library carlton street - . . mount gravatt library creek road - . . jericho library darwin street - . . narrabri library doyle street - . . quambatook guthrie street - . . gisborne hamilton street gisborne - . . watsons bay library marine parade - . . caloundra library omrah avenue - . . little river rothwell rd - . . warilla library woolworths avenue - . . newtown library - brown street - . . singleton library - queen street - . . jerramungup public library - tobruk street - . . gympie library - mellor street - . . riverwood library & knowledge centre kentucky road - . . kenwick public library kenwick road - . . bencubbin public library monger street - . . bankstown library and knowledge centre rickard road - . . chinchilla library - heeney street - . . penguin main street, penguin ( ) - . . holland park library seville road - . . ulladulla library b princes highway - . . williamstown creative technology hub nelson place - . . boggabri central northern branch library wee waa street - . . yeppoon library john street - . . manor lakes manor lakes blvd - . . citylibraries thuringowa central thuringowa drive - . . wheelers hill ferntree gully road wheelers hill - . . sorrento kiosk - melbourne road - . . ashgrove library amarina avenue - . . hallidays point library high street - . . norseman public library - prinsep street - . . walgett library fox street - . . grenfell library (weddin shire) main street - . . gordonvale library norman street - . . goombungee library mocatta street - . . brookton public library robinson road - . . phillip island thompson avenue phillip island - . . castle cove library b deepwater road - . . deception bay library bayview terrace - . . spearwood public library coleville crescent - . . laurieton library laurie street - . . cooroy library maple st - . . mooroopna morell street - . . churchill phillip parade - . . osborne park public library royal street - . . toowong library sherwood road - . . stonehenge library stratford street - . . dongara public library waldeck street - . . west coast community hub - driffield street, queenstown ( ) - . . new norfolk / charles street, new norfolk ( ) - . . coolbellup public library cordelia avenue - . . ellenbrook public library main street - . . hobart murray street, hobart ( ) - . . junee library lorne street - . . gunning library yass street - . . kojonup public library albany highway - . . dapto district library - princes highway - . . charleville library alfred street - . . kellerberrin public library massingham street - . . new lambton library regent street - . . parkdale parkers road parkdale - . . beaumaris reserve road beaumaris - . . toodyay public library stirling terrace - . . guildford public library james street - . . romsey main street - . . kariong library mitchell drive - . . vincent public library loftus street - . . cambridge public library the boulevard - . . wyndham public library koolama street - . . boyup brook public library abel street - . . lake grace public library absolon street - . . nannup public library adam street - . . adelaide river adelaide river primary school memorial terrace adelaide river nt - . . blayney library adelaide street - . . cocos west island public library airforce road - . . creswick albert street creswick - . . sebastopol albert street sebastopol - . . menai library allison crescent - . . augusta public library allnut terrace - . . alyangula alyangula area school, flinders st, alyangula nt - . . lilydale anderson street lilydale - . . angurugu angurugu nt - . . kandos library angus avenue - . . cervantes public library aragon street - . . rutherford library arthur street - . . tambo library arthur street - . . barcaldine library ash street - . . paraburdoo public library ashburton avenue - . . katanning public library austral terrace - . . doveton autumn place doveton ( ) - . . mount druitt library ayres grove - . . kaniva baker street kaniva - . . dunwich library ballow road - . . amity point library ballow street - . . ararat barkly street ararat - . . barunga barunga school, bagala street, barunga nt - . . batchelor batchelor institute, corner of nurndina street & awilla street, batchelor nt - . . bauhinia library honorarium bauhinia downs station - . . torquay beach road torquay - . . beacon public library beacon central, rowlands street - . . beaconsfield beaconsfield primary school, grubb street, beaconsfield ( ) - . . bell library bell bunya community centre - . . yarrawonga belmore street yarrawonga public library - . . portland bentinck street portland - . . harrow blair street harrow - . . parkes library bogan street - . . bolgart public library bolgart memorial hall, george street - . . wundowie public library boronia avenue - . . holbrook library bowler street - . . guyra library bradley street - . . tweed heads library brett street - . . brisbane square libary brisbane square library and customer centre - . . latham public library britt street - . . williams public library brooking street - . . bulleen bulleen plaza manningham road bulleen - . . boulia library burke street - . . burrum heads library burrum heads road - . . orange library byng street - . . maidenwell library c.w.a hall - . . banana shire mobile library c/- banana shire library service - . . caboolture library caboolture hub - . . harrington library caledonia street - . . mitchell library cambridge street - . . southern cross public library canopus street - . . capalaba library capalaba place - . . gayndah library capper street - . . carindale library carindale regional shopping centre - . . murchison public library carnarvon-mullewa road - . . tatura casey street tatura public library - . . coonamble library castlereagh street - . . ti tree / anmatjere central desert regional council, service centre, spencer st, ti tree nt - . . local studies library central library building - . . tom price public library central road - . . epping library chambers court - . . charlton charlton hall supper room, armstrong street - . . smithfield library cheviot street - . . bullsbrook community library chittering road - . . inglewood library civic centre - . . tennant creek civic centre, peko road, tennant creek nt - . . diamond valley civic drive diamond valley - . . croydon civic square croydon - . . civic library civic square, london circuit, canberra city - . . nhill clarence street nhill - . . derby public library clarendon street - . . koorda public library cnr allenby and haig street - . . maryborough cnr alma and nolan streets maryborough - . . inglewood public library cnr beaufort street and tenth avenue - . . donnybrook public library cnr bentley and emerald streets - . . blackburn cnr blackburn & central roads blackburn - . . cleveland library cnr bloomfield and middle streets - . . nambour library cnr bury and currie streets - . . normanton library cnr caroline and lansbourough streets - . . burleigh waters library cnr christine avenue & galeen drive - . . woden library cnr corinna & furzer streets, phillip - . . oberon library cnr dart & fleming streets - . . iluka library cnr duke and micalo streets - . . falcon public library cnr flavia street and cobblers road - . . southport library cnr garden and lawson streets - . . caulfield cnr glen eira & hawthorn roads caulfield - . . tieri library cnr grasstree and anncrouye streets - . . broome public library cnr haas and hammersleys streets - . . bentley public library cnr hedley place and manning road - . . gungahlin library cnr hibberson & gozzard streets, gungahlin - . . woodend cnr high & forest streets woodend - . . onslow public library cnr hooley and mcgrath avenue - . . zillmere library cnr jennings street and zillmere road - . . clermont library cnr karmoo and herschel street - . . albury librarymuseum cnr kiewa & swift streets - . . ballajura public library cnr kingfisher and illawarra crescent - . . byron bay library cnr lawson & middleton street - . . bull creek public library cnr leichhardt street and hassell crescent - . . helensvale library cnr lindfield drive and sir john overall drive - . . springfield central library cnr main street & sirius drive - . . tamborine mountain library cnr main street and yuulong road - . . manning public library cnr manning road and goss avenue - . . whitford public library cnr marmion and banks avenues - . . lowood library cnr michel and main streets - . . montrose cnr mt dandenong tourist & swansea roads montrose - . . deer park cnr neale & station roads deer park - . . crows nest john french v.c. memorial library cnr new england highway and william street - . . middle park cnr nimmo and richardson streets middle park - . . mullewa public library cnr padbury and thomas street - . . lennox head library cnr park lane & mackney lane - . . murray public library cnr pinjarra road and forrest street - . . ocean grove cnr presidents ave and the avenue ocean grove - . . nerang library cnr price and white streets - . . mornington cnr queen & vancouver streets mornington - . . hamilton library cnr racecourse road and rossiter parade - . . the pines (doncaster east) cnr reynolds & blackburn roads the pines - . . kwinana public library cnr robbos way and chisham avenue - . . cannington public library cnr sevenoaks and wharf streets - . . menzies public library cnr shenton and brown streets - . . south perth public library cnr south terrace and sandgate street - . . logan north library cnr sports drive & springwood road - . . a h bracks public library cnr stock road and canning highway - . . coburg cnr victoria & louisa streets coburg - . . amherst village public library cnr warton road and holmes street - . . success public library cnr wentworth parade and beeliar drive - . . fremantle city library cnr william and newman streets - . . collie public library cnr wittenoom and steere streets - . . boyne island library cnr wyndham & hayes avenue - . . newdegate public library collier street - . . jandowae library community & cultural centre - . . woodgate beach library community centre - . . blackwater library community centre wey street - . . highfields library community court - . . badgingarra public library community hall - . . greenacre library & knowledge centre community place - . . ravensthorpe public library community resource centre, dunn street - . . yass library comur street - . . chiltern conness street chiltern - . . clayton cooke street clayton - . . panania library & knowledge centre corner anderson avenue & tower street - . . wagga wagga city library corner baylis & morrow streets - . . newcomb corner bellarine hwy & wilsons rd newcomb - . . st clair library corner bennett road & endeavour avenue - . . woy woy library corner blackwall road & oval avenue - . . bracken ridge library corner bracken and barrett streets - . . toronto library corner brighton avenue & penell street - . . castle hill library corner castle & pennant streets - . . leeton library corner dooley lane & sycamore street - . . eagle vale library corner emerald drive & feldspar road - . . blacktown library corner flushcombe road & alpha street - . . prahran / windsor corner greville and chapel streets prahran /windsor - . . eastwood library corner hillview road & west parade - . . earlwood library & knowledge centre corner homer & william streets - . . eden library corner imlay & mitchell streets - . . kurri kurri library corner lang & allworth streets - . . lawson library corner loftus street & san jose avenue - . . cardiff library corner main & macquarie roads - . . toukley library corner main road & victoria avenue - . . riverstone library corner market street & riverstone parade - . . carina library corner mayfield road and nyrang street - . . merrylands library corner miller & newman streets - . . moorebank library corner nuwarra road & maddecks avenue - . . stuarts point library corner ocean avenue & marine parade - . . biloela library corner of grevillea and melton streets - . . alice springs corner of leichardt terrace and gregory terrace nt - . . hurstville library corner of queens road and dora street - . . brunswick corner of sydney road and dawson street brunswick - . . bulimba library corner oxford street and riding road - . . wellington library corner percy & maughan streets - . . unanderra branch library corner princes highway & factory road - . . narellan library corner queen & elyard streets - . . pennant hills library corner ramsay & yarrara roads - . . upper coomera library corner reserve and abraham roads - . . charlestown library corner smith & ridley streets - . . st peters/sydenham library corner unwins bridge road & swain street - . . umina beach library corner west & bullion streets - . . kew cotham road and civic drive kew - . . tuggeranong library cowlishaw street, greenway - . . violet town cowslip street violet town public library - . . corio cox road corio - . . langwarrin cranbourne road and southgateway langwarrin shop - . . beenleigh library crete street - . . crookwell library crookwell memorial hall complex, denison street - . . cronulla library croydon street - . . quinalow library daly street - . . trangie library dandaloo street - . . karrinyup public library davenport street - . . omeo day avenue omeo - . . kootingal library denman avenue - . . dianella public library dianella plaza shopping centre, waverley street - . . dickson library dickson shopping centre, dickson - . . lilydale district school, main street, lilydale ( ) - . . rockingham central library dixon road - . . miles library dogwood crossing @ miles - . . calliope library don cameron drive - . . stanmore library douglas street - . . durong community library durong hall - . . walkerston library dutton street - . . harden library east street - . . ravenswood eastside village shopping centre, prossers forest rd, ravenswood ( ) - . . heywood edgar street heywood - - . . jackson library edward street - . . kempsey shire library elbow street - . . palm beach library eleventh avenue - . . moss vale library elizabeth street - . . duaringa library elizabeth street - . . eneabba public library eneabba drive - . . erina library erina fair shopping centre - . . fairfield libary fairfield gardens shopping centre - . . margaret river public library fearn avenue - . . narooma library field street - . . katherine first floor - randazzo centre, katherine terrace, katherine nt (opposite oasis shopping centre) - . . northam public library fitzgerald street - . . tiaro library forgan terrace - . . capel public library forrest road - . . narrogin regional library fortune street - . . lalor park library freeman street - . . galiwin'ku galiwin’ku nt - . . woolgoolga library ganderton street - . . garden city library garden city shopping centre - . . east maitland library garnett road - . . canowindra library gaskill street - . . christmas island public library george fam centre, murray road - . . bungendore library gibraltar street - . . wendouree gillies street - . . glenden library glenden shopping centre - . . kingaroy library glendon street - . . barwon heads golf links road barwon heads - . . lake haven library and council services goobarabah road - . . aramac library gordon street - . . palmerston goyder square, on the boulevard palmerston nt - . . casino library graham place - . . grass patch public library grass patch primary school - . . blackheath library great western highway - . . wentworth falls library great western highway - . . green head public library greenhead road - . . kalbarri public library grey street - . . lavington library griffith road - . . moranbah library grosvenor complex - . . toogoolawah library gunyah street - . . sandstone public library hack street - . . useless loop public library hackett square - . . batemans bay community library hanging rock place - . . corryong hansen street corryong - . . kippax library hardwick crescent, holt - . . darwin city harry chan avenue, darwin nt - . . dumbleyung public library harvey street - . . mascot library hatfield street - . . hay library hay library lachlan street - . . helensburgh library helensburgh branch library walker street - . . henty library henty library sladen street - . . queenscliff hesse street queenscliff - . . hibberd library hibberd drive - . . texas library high steet ( ) - . . dampier public library high street - . . boonah library high street - . . koroit high street koroit - . . rutherglen high street rutherglen - . . yarra junction hoddle street yarra junction - . . cocos home island public library home island campus, cocos islands district high school - . . wodonga hovell street wodonga - . . withers public library hudson road - . . injune library hutton street - . . bellingen library hyde street - . . inala library inala civic centre - . . casula library ingham drive - . . nundle library innes street - . . ipswich central library ipswich central library - . . red cliffs jamieson avenue red cliffs - . . balingup public library jayes road - . . york public library joaquina street - . . coonabarabran library john street - . . pakenham john street pakenham ( ) - . . kulin public library johnson street - . . dalwallinu public library johnston street - . . fawkner jukes road fawkner - . . jundah library jundah information centre - . . karama karama shopping centre, karama nt - . . kenmore library kenmore village shopping centre - . . leitchville king albert ave leitchville - . . stockton library king street - . . runaway bay library lae drive - . . laidley library laidley library and customer service centre - . . lajamanu lajamanu learning centre, lajamanu nt - . . kyabram lake road - . . mt beauty lakeside avenue mt beauty - . . victoria point library lakeside shopping centre - . . talbingo library lampe street - . . south hedland public library leake street - . . moonie library leichhardt highway - . . port hedland public library len tapin memorial centre, dempster street - . . woollahra library at double bay level , new south head road - . . kings cross library level , - darlinghurst road - . . warrawong district library level , - king street - . . citylibraries flinders street library level , northtown - . . coolangatta library level , the strand - . . ashfield library level , liverpool road - . . redbank plaza branch library level , redbank plaza shopping centre - . . indooroopilly library level , indooroopilly shoppingtown - . . lane cove library library walk - . . carlingford library lloyds avenue - . . gatton library lockyer valley cultural centre - . . kingscliff library lot turnock street - . . roebourne public library lot sholl street - . . northcliffe public library lot muirillup road - . . cue public library lot austin street - . . pingrup public library lot burston street - . . pemberton public library lot brockman street - . . walpole public library lot latham avenue - . . ongerup public library lot eldridge street - . . yanchep public library lower floor, phil renkin recreation centre, lisford avenue - . . mount barker public library lowood road - . . warracknabeal lyle street warracknabeal - . . sunbury macedon street sunbury - . . wyandra library macks street - . . carnamah public library macpherson street - . . exmouth public library maidstone crescent - . . merbein main avenue merbein - . . coorow public library main street - . . meekatharra public library main street - . . augathella library main street - . . bollon library main street - . . bacchus marsh main street bacchus marsh - . . foster main street foster - . . bridport main street, bridport ( ) - . . kununurra public library mangaloo street - . . tongala mangan street tongala - . . kukerin public library manser street - . . marlborough library marlborough historical museum - . . bremer bay public library mary street - . . yungaburra library maud kehoe park - . . erindale library mcbryde crescent, wanniassa - . . rosebud mcdowell street rosebud - . . clifton library meara place - . . middlemount library middlemount shopping mall - . . mckinlay library middleton street - . . milikapiti milikapiti nt - . . milingimibi milingimbi community education centre, gadapu road, milingimbi nt - . . dareton library millie street - . . moulamein library morago street - . . aberdeen library moray street - . . redbank plains library moreton avenue - . . mount coot-tha library mt coot-tha road - . . mount ommaney library mt ommaney centre - . . wonthaggi murray street - . . wentworth library murray street, adjoining the council chambers - . . kerang murray valley hwy and shadforth street kerang - . . dalby library myall - . . st arnaud napier street st arnaud - . . dunsborough public library naturaliste community centre, dunsborough lakes drive - . . mount magnet public library naughton street - . . neerim south neerim east road neerim south - . . lake king public library newdegate-ravensthorpe road - . . ngukurr ngukurr road, ngukurr nt - . . nhulunbuy nhulunbuy high school, matthew flinders way, nhulunbuy nt - . . dingo library normanby street - . . chermside library north regional business centre - . . oatlands oatlands district high school, high st, oatlands ( ) - . . mogumber public library old school building, mogumber yarrawindah road - . . pittsworth library olympia arcade - . . werribee plaza pacific werribee shopping centre shop mm , level , derrimut road - . . pannawonica public library pannawonica drive - . . eltham panther place eltham - . . sanctuary point library paradise beach road - . . australind public library paris road - . . burleigh heads library park avenue - . . boronia park crescent boronia - . . braidwood library park lane - . . mona vale library park street - . . bunbury public library parkfield street - . . vermont south pavey place vermont south - . . capella library peak downs street - . . dural library pellitt lane - . . karratha public library pilbara institute campus, lot dampier road - . . pine creek pine creek museum and library, railway terrace, pine creek nt - . . dorrigo library pine street - . . murrayville pioneer park, grey street - . . batlow library pioneer street - . . pirlangimpi pirlangimpi nt - . . rolleston library planet street - . . point lookout library point lookout community hall - . . boddington public library pollard street - . . port douglas library port douglas community hall - . . sylvania library port hacking road - . . quairading public library post office building, jennaberring road - . . nundah library primrose lane - . . tumbarumba library prince street - . . milton library princes highway - . . boorowa library pudman street - . . mittagong library queen street - . . theodore library queensland government buildings - . . mount larcom library raglan street - . . nungarin public library railway avenue - . . three springs public library railway road - . . trayning public library railway street - . . dirranbandi library railway street ( ) - . . guildford library railway terrace - . . wyalkatchem public library railway terrace - . . rainbow beach library rainbow beach community hall - . . manunda library raintrees - . . ramingining ramingining library and media centre, ramingining nt - . . byfield library raspberry creek homestead - . . endeavour hills raymond mcmahon blvd endeavour hills ( ) - . . moura library recreation reserve - . . mungallala library redford street - . . belgrave reynolds lane belgrave - . . mirboo north ridgway mirboo north - . . coffs harbour city library rigby house, corner coff & duke streets - . . ermington library river road - . . phoenix park (malvern east) rob roy road phoenix park - . . kalgoorlie-boulder public library roberts street - . . robina library robina town centre drive - . . roma library roma community arts centre - . . mataranka roper terrace, mataranka nt - . . rosebery rosebery district high, morrisby street, rosebery ( ) - . . highton roslyn road highton - . . orbost ruskin street orbost - . . albion park russell street - . . port fairy sackville street port fairy - . . meandarra library sara street - . . scarborough public library scarborough civic centre, gildercliffe street - . . cloncurry bob mcdonald library scarr street - . . bruny online school road, alonnah, bruny island, tasmania ( ) - . . wiluna public library scotia street - . . yea semi-circle yea - . . sandgate library seymour street - . . dysart library shannon crescent - . . sheffield sheffield district high, henry street, sheffield ( ) - . . dimbulah library shire hall - . . burketown library shire office - . . thargomindah library shire office - . . earlville library shop stockland - . . miller library shop a cartwright avenue - . . bayswater shop , mountain high shopping centre, high st - . . kingston library shop , giles street, kingston - . . laverton public library shop , laver place - . . moorabbin shop taylor st moorabbin - . . dalyellup public library shop / norton promenade - . . stawell sloane street stawell - . . myrtleford standish street and odonnell avenue myrtleford - . . busselton public library stanley street - . . maclean library stanley street - . . bridgetown public library steere street - . . howard library steley street - . . yuleba library stephenson street - . . strahan strahan primary school, bay street, strahan ( ) - . . kyogle library stratheden street - . . denmark public library strickland street - . . rowville stud rd rowville - . . dundas valley library sturt street - . . sunnybank hills library sunnybank hills shopping centre - . . warnbro community library swallowtail parade - . . wickham public library tamarind place - . . tambellup public library tambellup community resource centre, - norrish street - . . taminmin taminmin college, challoner circuit, humpty doo nt - . . tasman tasman district high, nubeena back road, nubeena ( ) - . . thornton library taylor avenue - . . gladstone park taylor drive gladstone park - . . cecil plains library taylor street - . . alpha library tennyson st - . . georgetown library terrestrial - . . waroona public library thatcher street - . . gosnells public library the agonis, d albany highway - . . berowra branch library the gully road - . . chillagoe library the hub - . . elanora library the pines shopping centre - . . maylands public library the rise, eighth avenue - . . yarraman library toomey street - . . fitzroy crossing public library tourist bureau, flynn drive - . . tallangatta towong street tallangatta - . . wagin public library trent street - . . wongaling beach library tully-mission beach drive - . . adelong library tumut street - . . rosanna turnham avenue rosanna - . . umbakumba umbakumba nt - . . hopetoun public library unit / veal street - . . mingenew public library victoria street - . . beverley public library vincent street - . . lancelin public library vins way - . . moruya library vulcan street - . . wadeye wadeye nt - . . karumba library walker street - . . wallumbilla library wallumbilla cultural hub - . . sandringham waltham street sandringham - . . timboon wark street - . . morven library warrego highway - . . gilgandra library warren road - . . balranald library we street - . . wallan wellington square wallan - . . shellharbour village library wentworth street - . . logan village library wharf street - . . cannonvale library whitsunday plaza shopping centre - . . hebel library william street - . . thallon library william street - . . esperance public library windich street - . . windorah library windorah information centre - . . morawa public library winfield street - . . willagee public library winnacott street - . . evans head library woodburn street - . . yamba library wooli street - . . mount evelyn wray crescent mount evelyn - . . albany public library york street - . . gnowangerup public library yougenup road - . . harvey public library young street - . . yowah library yowah rural transaction centre - . . yuna public library yuna hall, main street - . . bega shire library zingel place - . . ariah park branch library - . . ashford library - . . baradine library - . . belrose library - . . bidyadanga community library - . . binnaway library - . . bondi junction central library - . . bulahdelah library - . . bulli library - . . bundarra library - . . cassilis library - . . castlecrag library - . . clarence regional library macksville - . . clarence regional library - . . clive james library and service centre - . . coraki library - . . curlewis library - . . drake library - . . drummoyne library - . . emu plains library - . . euabalong library - . . gerringong library - . . greater taree city library - . . greg percival library ingleburn - . . gulargambone library - . . gwandalan library - . . harbord community library - . . high street community library - . . hillston library - . . hj daley library campbelltown - . . homebush library - . . ku-ring-gai gordon library - . . library link pyrmont - . . library link town hall - . . macquarie regional library - . . margaret martin library - . . mendooran library - . . merimbula library - . . minto community library - . . munglinup public library - . . murrumbateman library - . . nabiac branch library - . . naremburn library - . . niagara library - . . nymagee library - . . nyngan library - . . old bar community library - . . parramatta central library - . . peak hill library - . . pooncarie library - . . premer library - . . rouse hill library - . . rydal library - . . terrey hills community library - . . the whitlam library cabramatta - . . tilligerry library - . . toormina library - . . toronto library - . . tottenham library - . . trundle library - . . tullamore library - . . urbenville library - . . west wyalong library - . . windsor library - . . wyoming library - . . norfolk island library new cascade road, next to the radio station + - . . armadale public library shop , orchard avenue, armadale, wa go © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. introduction to response rate limiting (rrl) isc website download software public mailing lists rss feed contact isc for professional support a quick introduction to response rate limiting sep minutes to read contributors share this print share dark light contents x no matching results found loading.. a quick introduction to response rate limiting updated on sep minutes to read contributors print share dark light what is rrl? rrl, or response rate limiting, is an enhancement to implementations of the dns protocol that can help mitigate dns amplification attacks (see what is a dns amplification attack?). in such an attack, the attacker sends high volumes of forged dns queries to a large number of authoritative dns servers, using the victim computer's ip address as the source of the request. the victim computer sees huge numbers of replies to questions that it did not ask. the authoritative servers have no way of knowing whether any particular dns query is real or malicious, but can detect patterns and clusters of queries when they are abused at high volumes. if a goodly number of authoritative servers can all be tricked into sending high-volume replies to the same victim computer, it is quite likely to collapse from overload. rrl helps mitigate dns denial-of-service attacks by reducing the rate at which authoritative servers respond to high volumes of malicious queries. the rrl mechanism is part of bind . , and was available as a software build option in bind . . . the problem any internet protocol based on udp is suitable for use in a denial-of-service attack, but dns is especially well suited for such malevolence. there are three reasons: the user datagram protocol, or udp, which is the norm for dns traffic, was not designed with source validation in mind. dns server software such as bind cannot tell by examining a particular packet whether the source address in that packet is real or fraudulent. an attacker can therefore send dns queries forged to look like they came from the intended victim, causing the dns server to send the replies to that victim. this is a "reflected attack". most isps do not check for forged source addresses in outbound packets. this allows forged-address reflection attacks to be launched from almost anywhere. small dns queries can generate large responses, allowing the attacker to send a lot less traffic than the victim receives, thereby amplifying the attack. for example, an edns query for the name isc.org of type any is bytes long (not counting the wire headers) and triggers a response that is , bytes long. by using an authoritative dns server as an unwitting accomplice, an attacker can achieve a nearly -fold increase in the amount of traffic that being directed at the victim and they can conceal the source of the attack as well. a solution if one packet with a forged source address arrives at a dns server, there is no way for the server to tell it is forged. if hundreds of packets per second arrive with very similar source addresses asking for similar or identical information, there is a very high probability of those packets, as a group, being part of an attack. the rrl software has two parts. it detects patterns in arriving queries, and when it finds a pattern that suggests abuse, it can reduce the rate at which the replies are sent. the results operators of large authoritative servers have reported huge reductions in network traffic and server load after enabling rrl. additionally, these servers are no longer seen as participating in abusive network behavior as fewer illegitimate responses are reaching their intended targets. the impact on legitimate traffic has been minimal. for more information using the response rate limiting feature outlines how to use the rrl feature in bind . . as with all bind features, the complete documentation is in the bind administrators' reference manual, the arm. pdf and html versions of that manual are part of every release of bind and are also available here. tags rrl security best practices authoritative amplification ddos isc website join our active user mailing lists content is ©isc - problems with this site? email us login to edit change password! current password current password is required new password new password must be at least characters long. it should contain at least one upper and lower case letter, number, and a special character. confirm password confirm password must be at least characters long. it should contain at least one upper and lower case letter, number, and a special character. confirm password should be same as new password update cookie consent we use our own and third-party cookies to understand how you interact with our knowledgebase. this helps us show you more relevant content and ads based on your browsing and navigation history. accept all cookies none none none none logplex - queue time - hirefire knowledge base hirefire knowledge base toggle navigation contact contact logplex - queue time "queue time is the amount of time (in milliseconds) between the heroku router accepting the request and the web dyno starting to process the request." if you want to auto-scale based on queue time percentiles or averages, then you have to create a logdrain and point it at our secure endpoint: heroku drains:add https://logdrain.hirefire.io once created, take note of the drain token: heroku drains | grep hirefire the format of the token is d. - - - - . the metrics will become available one minute after adding the logdrain. ruby application integration you now have to configure your application to actually log the queue time to stdout so that the heroku logplex picks it up and forwards it to our logdrain. if you have a ruby application backed by rack (rails, hanami, ...), then you can use the hirefire-resource gem for easy setup. gem "hirefire-resource" if you're using rails, it'll automatically inject the required middleware. for any other rack-based application such as sinatra or hanami you'll have to manually add the hirefire middleware at the top of the middleware stack. open up your middleware configuration (usually located in config.ru) and add the following: use hirefire::middleware # at the top # any other middleware here run myapp # at the bottom now that the middleware has been set up, all that you have to do is to create a hirefire configuration file. in a rails application you can simply create an initializer. when working with non-rails applications, you can just create the file wherever you wish, as long as it's loaded in at application boot time. once you've created the file, add the following configuration to it. hirefire::resource.configure do |config| config.log_queue_metrics = true end your application, once deployed to heroku, should now be ready to autoscale using the web.logplex.queuetime strategy. non-ruby application integration for applications written in other languages, you have to write the following format to stdout on each http request that you receive inside of your application: [hirefire:router] queue={time}ms for example: [hirefire:router] queue= ms in order to figure out what time should be, you need two pieces of information: the current time in milliseconds. the value stored in the x-request-start header. this header is added to the http request by the heroku router and represents the time (as a timestamp with millisecond precision) that the request was accepted by the heroku router. this header will be available at the application layer. the following formula will give you the time metric you need: current_time_in_milliseconds - x-request-start in ruby this can be achieved like so: (time.now.to_f * ).to_i - env["x-request-start"].to_i do the equivalent in your programming language. it's important to do this as high up in your web stack as possible on every http request. if you have a middleware / pipeline layer of some kind, put this logic there (preferably at the top for the best results). note that when you write the [hirefire:router] queue={time}ms entry to stdout, be sure that you don't run it through a log formatter that prefixes additional data such as, for example, timestamps. simply puts , print , echo , write (etc) it directly. hirefire ui once you've made the necessary changes to your application, and the logdrain has been created, and you've taken note of the drain token, log in to hirefire and add your heroku application if you haven't already. in the application form you'll see a field labeled "logplex drain token". add the token there and save it. now proceed to create a manager, using web as its name (must be called web , all lowercase, to reflect the procfile entry web ), set the type to web.logplex.queuetime , configure the rest of the options to your liking and then save and enable the manager. once all of that's done, hirefire will be auto-scaling your web dynos. details this strategy allows you to stream your application/platform logs to hirefire's logdrain. the logdrain parses metric data out of the received logs to build a temporary data stream of metrics that hirefire's primary system can utilize to auto-scale your dyno formation. logs are consumed, parsed and stored in the logdrain within roughly a second after heroku emits them, providing hirefire with metrics accurate to the second when it requests them during the minutely manager checkups. hirefire does not extract any information from your logs other than what is necessary for auto-scaling, which are the metrics emitted by heroku's router (service, connect), application (queue), runtime metrics (load m, load m, load m). did this answer your question? thanks for the feedback there was a problem submitting your feedback. please try again later. yes no still need help? contact us contact us last updated on january , toggle search categories guides general information frequently asked questions manager options status legal no results found © final creation . powered by help scout contact us name email subject message upload file none members of ndsa dlf about overview calendar faq foundational principles leadership strategic plan membership join the ndsa member orientation members groups overview interest groups content infrastructure standards and practices active working groups communications and publications conference program innovation awards levels of preservation ndsa agenda publications overview levels of preservation ndsa agenda osf repository conference digipres conference past digipres conferences news members of ndsa browse geographically, or search and sort below! organization state digital preservation focus ndsa about members groups calendar social twitter itunes youtube news linkedin contact ndsa c/o clir+dlf south clark street, arlington, va e: ndsa@diglib.org ndsa the ndsa is proudly hosted by the digital library federation at clir. all content on this site is available for re-use under a cc by-sa . international license. dlf libraries – erin white libraries – erin white library technology, ux, the web, bikes, #rva trans-inclusive design at a list apart i am thrilled and terrified to say that i have an article on trans-inclusive design out on a list apart today. i have read a list apart for years and have always seen it as the site for folks who make websites, so it is an honor to be published there. coming out as nonbinary at work this week, after years of working at vcu libraries, i have been letting my colleagues know that i&# ;m nonbinary. response from my boss, my team, and my colleagues has been so positive, and has made this process so incredibly easy. i didn&# ;t really have a template for a coming-out message, so ended up writing [&# ;] what it means to stay seven years ago last month i interviewed for my job at vcu. i started work a few months later, assuming i&# ;d stick around for a couple of years then move on to my next academic library job. instead i found myself signing closing papers on a house on my sixth work anniversary, having decided to [&# ;] back-to-school mobile snapshot this week i took a look at mobile phone usage on the vcu libraries website for the first couple weeks of class and compared that to similar time periods from the past couple years. here&# ;s some data from the first week of class through today. note that mobile is . % of web traffic. to round [&# ;] recruiting web workers for your library in the past few years i&# ;ve created a couple of part-time, then full-time, staff positions on the web team at vcu libraries. we now have a web designer and a web developer who&# ;ve both been with us for a while, but for a few years it was a revolving door of hires. so let&# ;s just say i&# ;ve hired lots [&# ;] easier access for databases and research guides at vcu libraries today vcu libraries launched a couple of new web tools that should make it easier for people to find or discover our library&# ;s databases and research guides. this project&# ;s goal was to help connect “hunters” to known databases and help “gatherers” explore new topic areas in databases and research guides . our web redesign task force [&# ;] why this librarian supports the ada initiative this week the ada initiative is announcing a fundraising drive just for the library community. i&# ;m pitching in, and i hope you will, too. the ada initiative&# ;s mission is to increase the status and participation of women in open technology and culture. the organization holds adacamps, ally workshops for men, and impostor syndrome trainings; and [&# ;] a new look for search at vcu libraries this week we launched a new design for vcu libraries search (our instance of ex libris&# ; primo discovery system). the guiding design principles behind this project: mental models: bring elements of the search interface in line with other modern, non-library search systems that our users are used to. in our case, we looked to e-commerce websites [&# ;] library lessons from the new york times digital strategy report cross-posted from vcu libraries&# ; intranet with minor edits. last week a new york times employee leaked an internal digital strategy report that&# ;s a stark case for change in the organization. here are some of the things that resonated with me about how this report dovetails with our digital strategies at vcu libraries and in libraryland. takeaways automate → innovate → transform [&# ;] takeaways from code lib i enjoyed the hell out of code lib in raleigh, nc back in march. code lib is a group of library/ish web/software developers/aficionados. we are loosely organized around a listserv, an annual conference and regional sub-conferences, and an open-access journal. this year i took pretty-okay notes for most of the sessions, but i wanted to loop back a few weeks after the fact [&# ;] getting started - hirefire knowledge base hirefire knowledge base toggle navigation contact contact getting started hirefire provides the ability to auto-scale your web- and worker dynos. it supports a variety of metric types. these metric types are used as a unit of measurement to determine, in combination with your hirefire configuration, when and how to scale. for web-based dynos we support the following metrics types: response time (percentile, average) connect time (percentile, average) queue time (percentile, average) dyno load (average) requests per minute apdex score for worker-based dynos we support the following metric types: job queue dyno load (average) guides we've prepared a few handy guides that'll walk you through integrating hirefire with your web-type dynos. we personally recommend the logplex solutions over the (legacy) hirefire and new relic solutions. this collection also includes guides on how to integrate hirefire for job queue-based scaling for worker-type dynos various environments. integration while there are libraries for ruby and python, you don't necessarily require any of these. they are simply provided for convenience and are only useful when you're looking setup job queue-based auto-scaling. all of the other metric types mentioned above won't involve any libraries at all and are easy and quick to setup. did this answer your question? thanks for the feedback there was a problem submitting your feedback. please try again later. yes no still need help? contact us contact us last updated on september , toggle search categories guides general information frequently asked questions manager options status legal no results found © final creation . powered by help scout contact us name email subject message upload file none implementation of a research data repository using the oxford common file layout standard at the university of technology sydney - uts eresearch toggle navigation uts eresearch research computing research computing amazon web services data arena high performance computing nci nectar data management and storage about eresearch @ uts about eresearch @ uts contact us uts eresearch strategy update uts eresearch strategy update uts eresearch strategy update eresearch strategy training platforms and tools platforms and tools stash cloudstor cloudstor swan describo enotebooks eresearch storage gitlab omero qualtrics redcap archives implementation of a research data repository using the oxford common file layout standard at the university of technology sydney date mon july by ptsefton this is a presentation by michael lynch and peter sefton, delivered by peter sefton at open repositories in hamburg. my travel was funded by the university of technology sydney. this presentation will discuss an implementation of the oxford common file layout (ocfl) in an institutional research data repository at the university of technology. we will describe our system in terms of the conference themes of open and sustainable and with reference to the needs and user experience of data depositors and users (many have large data, and or large numbers of files). ocfl, which is an approach to repository implementation based on static data, was developed to deal with a number of issues with “traditional” repository design, many of which are particularly acute when dealing with research data. we will cover how this meets our user and institutional needs and is a sustainable approach to managing data. this was presented by peter sefton - so the “i” throughout is him. this presentation was in a session about ocfl so we didn’t need to explain the standard in detail. here’s what they say on the ocfl site: this oxford common file layout (ocfl) specification describes an application-independent approach to the storage of digital information in a structured, transparent, and predictable manner. it is designed to promote long-term object management best practices within digital repositories. specifically, the benefits of the ocfl include: completeness, so that a repository can be rebuilt from the files it stores parsability, both by humans and machines, to ensure content can be understood in the absence of original software robustness against errors, corruption, and migration between storage technologies versioning, so repositories can make changes to objects allowing their history to persist storage diversity, to ensure content can be stored on diverse storage infrastructures including conventional filesystems and cloud object stores also moises sacal worked with us on this. fez fedora repository could be very finicky, but then in those days the repositories were “full stack” services whereas ocfl is just about the heart and soul of a repository, the storage. this talk is not just about ocfl, it reflects on the role of repositories and how we should view them in our infrastructure. my first “confession”. many, many years ago i was the technical manager for the rubric project - regional universities building research infrastructure collaboratively. this project helped a group of about nine partner unis get their first publications repositories up and running and our team morphed into the the national repository support service for australia. as part of this work we built a variety of migration tools to assist people in getting data from existing systems such as endnote, or ingesting marc (library) metadata into a repository. to do this we used the dspace archive format, a simple directory-plus-metadata file format that was not unlike ocfl. it occurred to me at the time that we could build a simple static repository system that used something like the dspace archive format, with separate ingest workflows, and build a portal using something like apace solr (though at that stage there really wasn’t anything else like apache solr). i didn’t follow up on those thoughts, as i thought they were a bit heretical - the monolithic repository architecture was the orthodoxy, though i did flirt with this idea in this article about discovery portals and persistent identifiers. anyway, it seems like those wrong thoughts are something i can talk about, now that we have ocfl. confession when i was working on the australian government funded nectar virtual lab project, alveo i suggested we start with a hydra/fedora repository as the ‘heart’ of the virtual lab, the repository component. that worked well in that we got up and running very fast with a discovery portal for data - but ran into problems when it came to data access - for example making item lists - data-sets - performed terribly because of the overhead inherent in the hydra architecture. i delivered a presentation at open repositories , written with other alveo staff, that explored some of the problems with using a “full stack” repository at scale. initially (and throughout the project) steve cassidy was suggesting a more ocfl-like way of loading data where it would be arranged on disk by some kind of curation human or machine orchestrated process and then indexed. steve was right. i was wrong. i’d love to come back to the alveo lab architecture for another go, maybe in a future round of ardc investment? like i said, creating a repository is this simple. but what is a repository? there are a variety of ways to look at a “repository” - all of these are facets of what a repository can be. repositories are no longer just places to put stuff (if they ever were) - they’re positively bristling with services. over the years of this conference, repositories have become shall we say baroque. and in some cases slow. they also tend to have problems when you present them with a terabyte file, or ten million kilobyte files. ocfl is, we think, in part a reaction from our community to those realities. using ocfl as the storage layer in a repository is a radical separation of con- -cerns. this is a view of our eresearch architecture - stash is our data management service, which is an implementation of redbox. the point of showing the diagram is not to go through it all in detail, but to show how complex the ecosystem of research data services can be - and where data archive / repository core sits. the ocfl components (the publication and archive file-systems) are shown in red, while agents (the repository adaptor that writes content from research workspace services in a repository, the public and private data portales) that access ocfl are shown in blue. this is our work-in-progress discovery portal - this is not built in to the repository like it would be with, say dspace, it’s a separate service that indexes an ocfl repository (in this case the public data). here’s the same diagram stripped back a bit more ... and even more stripped back, to show only the core repository service (in red) and a couple of services that interact with it (in blue). and here’s the actually repository part - the storage layer. the idea of ocfl is that there are services that all access this core layer but they’re disposable and/or interchangeable, and you just leave your precious data where it is, on disk (or some virtual disk-like view of whatever the storage solution de jour happens to be). here’s a screenshot of what an ocfl object looks like - it’s a series of versioned directories, each with a detailed inventory. no this slide is not about a zoological repository. it’s about the elephants in the repository aka the risks that come with an ocfl implementation. the main issue in implementation is making sure that you have to think about the design of software to update the repository at the same time. you don’t want to end up with (a) corrupt files or (b) transactions that didn’t complete. ocfl is also very new (it’s not at version yet) so there’s some risk that the services we are hoping for won’t arrive - but given that our data are still safe on our own storage infrastructure and it possible to code-up an ocfl consuming application in a matter of days this is not a serious risk. ocfl needs some explaining. i’ve had a couple of conversations with developers where it takes them a little while to get what it’s for. but they do get it. mike lynch’s summary - this is a modern take on “how to i organise my stuff” - in this sense ocfl is like a framework for data - and we all use frameworks for code these days, we have some good news to announce - we have a grant to continue our ocfl work from the australian research data commons. (i’ve used the new research organisation registry (ror) id for ardc, just because it’s new and you should all check out the ror). we’re going to be demonstrating large-scale use of ocfl, with research objects described using ro-crate metadata. (see also my presentation on datacrate which introduces ro-crate). so what do we think? working with ocfl has been really great for the team at uts - it’s well designed, and just when you think “hey, how do i do file-locking” you find that there’s a hint in the design (a /deposit directory in this case) that points towards a solution. over at the ro-crate project stian and eoghan and i really liked the ocfl approach of having a clear spec with implementation notes, and we’re going to try to emulate that as we work on merging research object and datacrate into one general purpose way of describing and packaging research data. see also my presentation on datacrate which introduces ro-crate. categories blog datacrate datacrate, repositories, eresearch event hacky hour preservation project repositories © uts · powered by gitlab and pelican - - at : am version: d back to top none lita blog lita blog empowering libraries through technology jobs in information technology: august , new this week coordinator of digital scholarship and programs, marquette university libraries, milwaukee wi digital scholarship coordinator, unc charlotte, charlotte, nc visit the lita jobs site for additional job openings and information on submitting your own job posting. jobs in information technology: august , new this week information systems manager (pdf), the community library association, ketchum, id children&# ;s librarian, buhl public library, buhl, id technology integration librarian, drexel university libraries, philadelphia, pa visit the lita jobs site for additional job openings and information on submitting your own job posting. your core community update much has been happening behind-the-scenes to prepare for core’s upcoming launch on september st, so we want to update you on the progress we’ve made. at the ala virtual conference council meetings, the ala council approved the creation of core, so we’re official! it’s been a difficult summer for everyone given the global situation, but this was a milestone we’re excited to reach. what we’ve been doing in may, the core transition committee (the division presidents plus senior staff) formed working groups of members from all divisions to make recommendations about how to proceed with our awards/scholarships, budget/finance, committees, communications, conference programming, continuing education, fundraising/sponsorships, interest groups, member engagement, nominations for president-elect, publications, and standards. these groups have done an amazing amount of work in a very short time period, and we’re grateful to these members for their commitment and effort. we’re happy to report... free lita webinar ~ library tech response to covid- ~ august th sign up for this free lita webinar: library tech response to covid- libraries are taking the necessary precautions to create a safe environment during the pandemic. social distancing isn’t the only solution, but providing access to loanable technologies, including handling and quarantine of equipment, cleaning, and other safety and health concerns are just some of the measures put in place. with the ongoing disruption to library services caused by covid- , what reopening planning policies should be considered for usage? in this free -minute presentation, our presenters will share tips that might be helpful to other librarians before they reopen. the presenters will also talk about the&# ;evolution of the phased plan from the establishment of a temporary computer lab in the library as covid- began to spread in march , to the current phased approach for gradual reopening. justin will also offer insight into managed access, technology and services, workflows, messaging,... jobs in information technology: july , new this week library director, walpole town library, walpole, nh visit the lita jobs site for additional job openings and information on submitting your own job posting. core call for ala annual program proposals submit an ala annual conference program proposal for ala’s newest division, core: leadership, infrastructure, futures, which will begin on september , . proposals are due september , , and you don’t need to be a core member to submit a proposal. submit your idea using this proposal form. core welcomes topics of interest to a wide range of library professionals in many different areas, including… . access and equity advocacy in areas such as copyright, equity of access, open access, net neutrality, and privacy preservation week equity, diversity, and inclusion, both within the division and the profession, as related to core’s subject areas . assessment emphasizing the role of assessment in demonstrating the impacts of libraries or library services assessment tools, methods, guidelines, standards, and policies and procedures . leadership and management developing leaders at every level best practices for inclusion by using an equity lens to examine leadership... core call for webinar proposals submit a webinar proposal for ala’s newest division, core: leadership, infrastructure, futures, which will begin on september , . proposals are due september , , and you don’t need to be a core member to submit a proposal. early submissions are encouraged and will be considered for september and october presentations. submit your idea using this proposal form. core webinars reach a wide range of library professionals in many different areas, including… .&# ;access and equity advocacy in areas such as copyright, equity of access, open access, net neutrality, and privacy preservation week equity, diversity, and inclusion, both within the division and the profession, as related to core’s subject areas .&# ;assessment emphasizing the role of assessment in demonstrating the impacts of libraries or library services assessment tools, methods, guidelines, standards, and policies and procedures .&# ;leadership developing leaders at every level best practices for inclusion by using an equity lens to examine... core virtual forum is excited to announce our keynote speakers! core virtual forum welcomes our keynote speakers, dr. meredith d. clark and sofia leung! both speakers embody our theme in leading through their ideas and are catalysts for change to empower our community and move the library profession forward. dr. clark is a journalist and assistant professor in media studies at the university of virginia. she is academic lead for documenting the now ii, funded by the andrew w. mellon foundation. dr. clark develops new scholarship on teaching students about digital archiving and community-based archives from a media studies perspective. she will be a - fellow with data &# ; society. she is a faculty affiliate at the center on digital culture and society at the university of pennsylvania. and, she sits on the advisory boards for project information literacy, and for the center for critical race and digital studies at new york university. clark is an in-demand media consultant... catch up on the june issue of information technology and libraries the june issue of information technology and libraries (ital) was published on june . editor ken varnum and lita president emily morton-owens reflect on the past three months in their letter from the editor, a blank page, and lita president’s message, a framework for member success, respectively. kevin ford is the author of this issue’s “editorial board thoughts” column, seeing through vocabularies. rounding out our editorial section, the june “public libraries leading the way” section offers two items. chuck mcandrew of the lebanon (new hampshire) public libraries describes his leadership in the imls-funded libraryvpn project. melody friedenthal, of the worcester (massachusetts) public library talks about how she approached and teaches an intro to coding using python course. peer-reviewed content virtual reality as a tool for student orientation in distance education programs: a study of new library and information science students dr. sandra valenti, brady lund, ting wang virtual reality... jobs in information technology: july , new this week dean of libraries, san jose state university, san jose, ca deputy library director, city of carlsbad, carlsbad, ca visit the lita jobs site for additional job openings and information on submitting your own job posting. jobs in information technology: july , new this week web services librarian, chester fritz library, university of north dakota, grand forks, nd visit the lita jobs site for additional job openings and information on submitting your own job posting. jobs in information technology: june , new this week metadata librarian, librarian i or ii, university of northern british columbia, prince george, british columbia, canada visit the lita jobs site for additional job openings and information on submitting your own job posting. jobs in information technology: june , new this week information technology librarian,&# ;university of maryland, baltimore county, baltimore, md associate university librarian for research and learning,&# ;columbia university libraries, new york, ny library technology/programmer analyst iii,&# ;virginia beach public library,&# ;virginia beach, va visit the lita jobs site for additional job openings and information on submitting your own job posting. core virtual happy hour social ~ june our joint happy hour social at midwinter was such a success that next week we’re bringing happy hour to you online—and registration is free! we invite members of alcts, lita, and llama to join us on friday, june , : - : pm central time for virtual happy hour networking and/or play with your peers in a game of scattergories.&# ;wear your favorite pop culture t-shirt, bring your best zoom background, grab a beverage, and meet us online for a great time! attendees will automatically be entered to win free registration to attend the core virtual forum.&# ;winner must be present to redeem prize.&# ;registration is required. register now at: bit.ly/ nenprh michael carroll awarded lita/christian larew memorial scholarship michael carroll has been selected to receive the &# ;lita/christian larew memorial scholarship ($ , ) sponsored by the library and information technology association (lita) and baker &# ; taylor. this scholarship is for master’s level study, with an emphasis on library technology and/or automation, at a library school program accredited by the american library association. criteria for the scholarship includes previous academic excellence, evidence of leadership potential, and a commitment to a career in library automation and information technology. the larew scholarship committee was impressed by&# ;what michael has already accomplished and look forward to seeing what he will accomplish after graduation in .&# ;michael&# ;has already shown&# ;a strong interest in digitization projects.&# ;he currently manages&# ;a&# ;team of students working with digitization.&# ;previously, he has scanned and cataloged many collections.&# ;he has also assisted the presbyterian historical society&# ;in creating&# ;sustainable processes for digitization.&# ;michael has also shown his willingness and ability to work&# ;with a wide variety of&# ;projects and technologies that span&# ;both&# ;technical&# ;and non-technical&# ;including... we are back on twitter friday for #litachat the fourth in this series of #litachats will start on friday, june from - central standard time on twitter. we will be asking you to chat with us about self-care. what are you doing to take care of yourselves during this time? how do you unplug without feeling guilty?&# ; we hope you’ll join us for #litachat and chat about self-care techniques and figuring out how to better take care of ourselves during these tough times. we&# ;re looking forward to hearing from you! join lita on twitter catch up on the last #litachat join us for alcts/lita/llama e-forum! please join us for a joint alcts/lita/llama e-forum discussion. it’s free and open to everyone! registration information is at the end of the message, along with subscription management options for existing listserv members. continuing to manage the impact of covid- on libraries june - , moderated by alyse jordan, steven pryor, nicole lewis and rebecca uhl please join us for an e-forum discussion. it’s free and open to everyone! registration information is at the end of the message. each day, discussion begins and ends at: pacific: a.m. – p.m. mountain: a.m. – p.m. central: a.m. – p.m. eastern: a.m. – p.m. over the past several months, covid- has significantly impacted libraries and library technical service units and departments, including requiring staff to work remotely and determining what services they can provide. as states begin to reopen, libraries face challenges as they determine... together against racism ala and core are committed to dismantling racism and white supremacy. along with the ala executive board, we endorse the&# ;black caucus of the american library association (bcala)’s may statement&# ;condemning the brutal murder of george floyd at the hands of minneapolis police department officers. in their statement, bcala cites floyd’s death as “the latest in a long line of recent and historical violence against black people in the united states.” not only does core support the sentiments of bcala, we vow to align our values regarding equity, diversity, and inclusion with those of bcala and other organizations that represent marginalized communities within ala. we also stand strong with the asian/pacific american community, which has been the target of xenophobia and racism in the wake of the outbreak of covid- , and support the&# ;asian/pacific american librarians association (apala) and their statement&# ;that, “there is no excuse for discriminatory sentiments and actions towards asians... we are back on twitter tomorrow for #litachat are you ready for the next twitter #litachat? join the discussion on friday, may , from - pm central time. we will be asking you to tell us about challenges with working from home. are there things you can’t do and wish you could? are there issues with your home setup in general?&# ;anne pepitone will lead the discussion. we invite you to join us tomorrow to share your experiences and chat with your colleagues. follow lita on twitter catch up on the last #litachat we&# ;re looking forward to hearing from you! -the lita membership development committee lita job board analysis report – laura costello (chair, assessment & research) lita assessment & research and diversity & inclusion committees background &# ; data this report comes from a joint analysis conducted by lita&# ;s assessment &# ; research and diversity &# ; inclusion committees in fall . the analysis focused on the new and emerging trends in skills in library technology jobs and the types of positions that are currently in demand. it also touches on trends in diversity and inclusion in job postings and best practices for writing job ads that attract a diverse and talented candidate pool.&# ; the committees were provided with a list of job postings from the lita job board between - . data included the employer information, the position title, the location (city/state) the posting date. some postings also included a short description. the assessment &# ; research committee augmented the dataset with job description, responsibilities, qualifications, and salary information for a % sample of the postings from each year using archival job posting information. committee members also assigned... congratulations to dr. jian qin, winner of the lita/oclc kilgour research award dr. jian qin has been selected as the recipient of the &# ;frederick g. kilgour award for research in library and information technology, sponsored by oclc and the library and information technology association (lita). she&# ;is the professor and director at the ischool, syracuse university.&# ;&# ;the kilgour award honors research relevant to the development of information technologies, especially work which shows promise of having a positive and substantive impact on any aspect(s) of the publication, storage, retrieval and dissemination of information, or the processes by which information and data are manipulated and managed. it recognizes a body of work probably spanning years, if not the majority of a career. the winner receives $ , , and a citation. dr. qin’s recent research projects include metadata modeling for gravitational wave research data management and big metadata analytics using genbank metadata records for dna sequences, both with funding from nsf. she also collaborated with a colleague to develop a capability maturity model... lita/ala survey of library response to covid- the library and information technology association (lita) and its ala partners are seeking a new round of feedback about the work of libraries as they respond to the covid- crisis, releasing a survey and requesting feedback by : p.m. cdt, monday, may , . please complete the survey by clicking on the following link: https://www.surveymonkey.com/r/libraries-respond-to-covid- -may- .&# ; lita and its ala partners know that libraries across the united states are taking unprecedented steps to answer the needs of their communities, and this survey will help build a better understanding of those efforts. lita and its ala partners will use the results to advocate on behalf of libraries at the national level, communicate aggregated results with the public and media, create content and professional development opportunities to address library staff needs, and share some raw, anonymized data elements with state-level staff and library support organizations for their own advocacy needs.&# ; additional information about... #coreforum is now a virtual event! join your ala colleagues from across divisions for the forum, which is now a virtual event!&# ; where: in light of the covid- public health crisis, leadership within lita, alcts, and llama made the decision to move the conference online to create a safe, interactive environment accessible for all. what: call for proposals have been extended to friday june , .&# ; when: forum is scheduled november and , how: share your ideas and experiences with library projects by submitting a talk for the inaugural event for core:&# ; https://forum.lita.org/call-for-proposals for more information about the lita, alcts, llama (core) forum, please visit https://forum.lita.org&# ; jobs in information technology: may , new this week web services librarian, fairfield university, fairfield, ct visit the lita jobs site for additional job openings and information on submitting your own job posting. wfh? boost your skill set with lita ce! reserve your spot and learn new skills to enhance your career with lita online continuing education offerings. buying strategies for information technologywednesday, may&# ; , , : - : pm central timepresenter:&# ;michael rodriguez, collections strategist at the university of connecticut in this -minute webinar, you’ll learn&# ;best practices, terminology, and concepts for effectively negotiating contracts for the purchase of information technology (it) products and services. view details&# ;and&# ;register here. using images from the internet in a webpage: how to find and citewednesday, june&# ; , , : - : pm central timepresenter:&# ;lauren bryant, priority associate librarian of ray w. howard library in this -minute&# ;webinar, you’ll learn&# ;practical ways&# ;to quickly find and filter creative commons licensed images online, learn how to hyperlink a citation for a website, and&# ;how to use creative&# ;commons images for thumbnails in videos and&# ;how to cite the image in unconventional situations like this. view details&# ;and&# ;register here. troublesome technology trends: bridging the learning dividewednesday, june , , : - : pm... may / twitter #litachat last week, anne pepitone kicked off the discussion with zoom virtual backgrounds, shared her favorites, and provided tips on how to use them. the next twitter #litachat will be on friday, may , from - pm central time when we&# ;ll talk about apps that help you work from home. what do you use to help with project management, time management, deadlines, or to just stay focused? we invite you to join us tomorrow to share, learn, and chat about it with your colleagues. follow lita on twitter. we&# ;re looking forward to hearing from you! -the lita membership development committee jobs in information technology: april , new this week two associate dean positions, james madison university libraries, harrisonburg, va visit the lita jobs site for additional job openings and information on submitting your own job posting. data privacy while working from home today&# ;s guest post is brought to you by our recent presenter, becky yoose. special thanks to becky for being willing to answer the questions we didn&# ;t have time for during our webinar! hello everyone from your friendly neighborhood library data privacy consultant! we covered a lot of material earlier this month in &# ;a crash course in protecting library data while working from home,&# ; co-sponsored by lita and oif. we had a number of questions during the webinar, some of which were left unanswered at the end. below are three questions in particular that we didn’t get to in the webinar. enjoy! working from home without a web-based ils we don&# ;t have a web-based version of our ils and our county-based it department says they can&# ;t set up remote desktop (something to do with their firewall)… do you have any recommendations on how to advocate for remote desktop? if i have... strategies for surviving a staffing crisis library staff are no strangers to budget and staffing reductions. most of us have way too much experience doing more with less, covering unfilled positions, and rigging solutions out of the digital equivalent of chewing gum and bailing wire, because we can’t afford to buy all the tools we need. in the last two years, my department at northern arizona university’s cline library operated with roughly half the usual amount of staff. in this post, i’ll share a few strategies that helped us get through this challenging time. first, a quick introduction. my department, content, discovery &# ; delivery services, includes the digital services unit (formerly library technology services) as well as collection management (including electronic resources management), acquisitions, cataloging, physical processing, interlibrary loan and document delivery, and course reserves. we are a technology-intensive department, both as users and implementers/supporters of technology. here are some of the strategies we used to... april / twitter #litachat a lot has changed since we had our last twitter #litachat, core passed and then covid happened. we are all navigating new territory in our jobs and life overall. so we wanted to bring you a weekly set of litachats discussing our shared experiences during these strange times.&# ; the first in this series of litachats will start on friday, april from - pm central standard time. we will be asking you to show us your zoom virtual backgrounds! we know that zoom conferencing has been popular among many workplaces so we thought what would be better than showcasing some of the creative backgrounds everyone has been using. if you don’t have a background no worries, you can share about the best backgrounds you have seen from colleagues. don’t know how to turn on zoom virtual backgrounds? we will cover that too! we hope you’ll join us on twitter for... congratulations to samantha grabus, winner of the lita/ex libris student writing award samantha grabus has been selected as the winner of the student writing award sponsored by ex libris group and the library and information technology association (lita) for her paper titled “evaluating the impact of the long s upon th-century encyclopedia britannica automatic subject metadata generation results.” grabus is a research assistant and phd student at drexel university metadata research center. &# ;this valuable work of original research helps to quantify the scope of a problem that is of interest not only in the field of library and information science, but that also, as grabus notes in her conclusion, could affect research in fields from the digital humanities to the sciences,&# ; said julia bauder, the chair of this year&# ;s selection committee. when notified she had won, grabus remarked, “i am thrilled and honored to receive the lita/ex libris student writing award. i would like to extend my gratitude to the award committee... jobs in information technology: april , new this week web and digital scholarship technologies librarian, marquette university libraries, milwaukee, wi ceo / library director, orange county library system, orlando, fl visit the lita jobs site for additional job openings and information on submitting your own job posting. ala lita emerging leaders: inventing a sustainable division in january , the latest cohort of emerging leaders met at ala midwinter to begin their projects. lita sponsored two emerging leaders this year: kelsey flynn, adult services specialist at white oak library, and paige walker, digital collections &# ; preservation librarian at boston college. kelsey and paige are part of emerging leaders group g, &# ;inventing a sustainable division,&# ; in which they’ve been charged with identifying measures that lita can take to improve its fiscal and environmental sustainability. as a first step in their assessment, the group distributed a survey to lita members that will quantify interest in sustainable measures such as virtual conferences and webinars. want to help? complete the survey to give feedback that may shape the direction of our chapter. group g is fortunate to have several other talented library workers on its team:&# ; kristen cooper, plant sciences librarian at university of minnesota tonya ferrell, oer coordinator at... latest in lita elearning so much has changed since covid- . online learning is in greater demand and we are working hard to provide you with resources and more professional development opportunities that strengthens the library community. we hope you are well and staying safe. there&# ;s a seat waiting for you. register today! digital inception: building a digital scholarship/humanities curriculum as a subject librarian wednesday, april , : &# ; : p.m. central time presenter: marcela isuster, education and humanities librarian, mcgill university this presentation will guide attendees in building a digital scholarship curriculum from a subject librarian position. it will explore how to identify opportunities, reach out to faculty, and advertise your services. it will also showcase activities, lesson plans, and free tools for digital publication, data mining, text analysis, mapping, a section on finding training opportunities and strategies to support colleagues and create capacity in your institutions. in this -minute webinar, you&# ;ll learn:... join us this fall for #coreforum – proposal deadline extended! call for proposals have now been extended to friday, may , . share your&# ;ideas and experiences about library technology, leadership, collections, preservation, assessment, and metadata at the inaugural meeting of core, a joining of lita/alcts/llama. we welcome your session proposal. for more information about the call for proposals and our theme of exploring ideas and making them reality, visit the forum website: https://forum.lita.org&# ; event details november - , baltimore, md renaissance baltimore harborplace hotel covid- planning the lita/alcts/llama forum planning committee is currently evaluating a contingency plan, should the covid- public health crisis impact forum in november. core is approved! we’re thrilled to announce that core: leadership, infrastructure, futures is moving forward, thanks to our members. the three existing divisions’ members all voted to approve the bylaws change that will unite alcts, lita, and llama to form core: alcts: % yes lita: % yes llama: % yes the presidents of the three divisions, jennifer bowen, alcts, emily morton-owens, lita, and anne cooper moore, llama, shared the following statement: “we first want to thank our members for supporting core. their belief in this vision, that we can accomplish more together than we can separately, has inspired us, and we look forward to working with all members to build this new and sustainable ala division. we also want to thank the core steering committee, and all the members who were part of project teams, town halls and focus groups. we would not have reached this moment without their incredible work.” ala executive... free lita webinar: protect library data while working from home a crash course in protecting library data while working from home presenter: becky yoose, founder / library data privacy consultant, ldh consulting services thursday, april , : &# ; : pm central time there’s a seat waiting for you…&# ;register for this free lita webinar today! libraries across the u.s. rapidly closed their doors to both public and staff in the last two weeks, leaving many staff to work from home. several library workers might be working from home for the first time in their current positions, while many others were not fully prepared to switch over to remote work in a matter of days, or even hours, before the library closed. in the rush to migrate library workers to remote work and to migrate physical library programs and services to online, data privacy and security sometimes gets lost in the mix. unfamiliar settings, new routines, and increased reliance on vendor... jobs in information technology: march , new this week head of library technology services, east carolina university, greenville, nc visit the lita jobs site for additional job openings and information on submitting your own job posting. march ital issue now available the march issue of information technology and libraries (ital) is available now. in this issue, ital editor ken varnum shares his support of lita, alcts, and llama merging to form a new ala division, core. our content includes a message from lita president, emily morton-owens. “a framework for member success,“ morton-owens discusses the current challenges of lita as a membership organization and reinvention being the key to survival. also in this edition, laurie willis discusses the pros and cons of handling major projects in-house versus hiring a vendor in &# ;tackling big projects.&# ; sheryl cormicle knox and trenton smiley discuss using digital tactics as a cost-effective way to increase marketing reach in &# ;google us!&# ; featured articles: “user experience methods and maturity in academic libraries,” scott w. h. young, zoe chao, and adam chandler this article presents a mixed-methods study of the methods and maturity of user experience (ux) practice in... learn how to build your own digital scholarship/humanities curriculum with this lita webinar are you a subject librarian interested in building digital scholarships? join us for the upcoming webinar &# ;digital inception: building a digital scholarship/humanities curriculum as a subject librarian,&# ; on wednesday, april , from : &# ; : pm cst. digital scholarship is gaining momentum in academia. what started as a humanities movement is now present in most disciplines. introducing digital scholarship to students can benefit them in multiple ways: it helps them interact with new trends in scholarship, appeals to different kinds of learners, helps them develop new and emerging literacies, and gives them the opportunity to be creative. this -minute&# ;presentation will guide attendees in building a digital scholarship curriculum from a subject librarian position. it will explore how to identify opportunities, reach out to faculty, and advertise your services. it will also showcase activities, lesson plans, and free tools for digital publication, data mining, text analysis, mapping, etc. finally, the presentation will... jobs in information technology: march , new this week project manager for resource sharing initiatives,&# ;harvard university,&# ;cambridge, ma research data services librarian,&# ;university of kentucky libraries,&# ;lexington, ky digital archivist,&# ;rice university, fondren library,&# ;houston, tx associate director, technical services,&# ;yale university,&# ;new haven, ct visit the lita jobs site for additional job listings and information on submitting your own job posting. congratulations to alison macrina, winner of the lita/library hi tech award the lita/library hi tech awards committee is pleased to select alison macrina as the recipient of the lita/library hi-tech award. macrina led the tor relay initiative in new hampshire, is the founder and executive director of the library freedom project, and has written and taught extensively in the areas of digital privacy, surveillance, and user anonymity in the context of libraries and librarianship. in this role, macrina was instrumental in creating the library freedom institute, which trained its first cohort in and will train its third cohort in . macrina has also spoken on digital privacy and the work of the library freedom project across the united states and published&# ;anonymity, the first book in ala&# ;s library futures series, in . the committee was fortunate to receive several outstanding nominations for the award. macrina stood out in this strong pool of candidates for the broad reach and impact... nominate yourself or someone you know for the next lita top tech trends panel of speakers lita is looking for dynamic speakers with knowledge about the top trends in technology and how they intersect with information security and privacy. library technology is quickly evolving with trends such as vr, cloud computing and ai. as library technology continues to impact our profession and those that we serve, security and privacy are quickly becoming top concerns. we hope this panel will provide insight and information about these technology trends for you to discuss within your own organization. if you or someone you know would be a great fit for this exciting panel, please submit your nomination today.&# ;&# ; submit your nominations – the deadline is april , . the session is planned for sunday, june , , : – : pm, at the ala annual conference in chicago, il. a moderator and several panelists will each discuss trends impacting libraries, ideas for use cases, and practical approaches for... jobs in information technology: march , new this week wilson distinguished professorship, university of north carolina at chapel hill, chapel hill, nc coordinator of library technical services, berea college, berea, ky ui/ux designer, university of rochester libraries, rochester, ny technical support and hardware specialist &# ; openings, st. lawrence university, canton, ny software engineer, library systems, stanford health care, palo alto, ca visit the lita jobs site for additional job listings and information on submitting your own job posting. hebah emara is our - lita/oclc spectrum scholar lita and oclc are funding hebah emara&# ;s participation in the ala spectrum scholars program as part of their commitment to help diversify the library technology field. emara&# ;is a second year distance student at the university of missouri – columbia school of information science and learning technologies mlis program. she is interested in the ways libraries and technology intersect. her background in it and love of learning about technology, computers, and programming drew her to working in library technology. libraries’ ability to bridge the digital divide and their use of technology to provide opportunities to their communities and solve problems are also of particular interest to emara. her decision to apply to the spectrum scholarship was fueled by a desire to learn from a community of peers and mentors.&# ; emara&# ;is currently the co-chair of a tech unconference to be held in april and organized by mentornj in collaboration with the... share your ideas and library projects by submitting a session proposal for the forum! forum call for proposals submission deadline: march , november - , baltimore, maryland renaissance baltimore harborplace hotel do you have an idea or project that you would like to share? does your library have a creative or inventive solution to a common problem? submit a proposal for the lita/alcts/llama forum! submission deadline is march th. our library community is rich in ideas and shared experiences. the forum theme embodies our purpose to share knowledge and gain new insights by exploring ideas through an interactive, hands-on experience. we hope that this forum can be an inspiration to share, finish, and be a catalyst to implement ideas… together. we invite those who choose to lead through their ideas to submit proposals for&# ;sessions or preconference workshops, as well as&# ;nominate keynote speakers. this is an opportunity to share your ideas or unfinished work, inciting collaboration and advancing the library profession... early-bird registration for the exchange ends in three days! the march early-bird registration deadline for the exchange is approaching. register today and save! there&# ;s still time to register for the exchange at a discount, with early-bird registration rates at $ for alcts, lita, and llama members; $ for ala individual members; $ for non-members; $ for student/retired members; $ for groups; and $ for institutions. early-bird registration ends march . taking place may , , and , the exchange will engage a wide range of presenters and participants, facilitating enriching conversations and learning opportunities in a three-day, fully online, virtual forum. programming includes keynote presentations from emily drabinski and rebekkah smith aldrich, and sessions focusing on leadership and change management, continuity and sustainability, and collaborations and cooperative endeavors. in addition to these sessions, the exchange will offer lightning rounds and virtual poster sessions. for up-to-date details on sessions, be sure to check the exchange website as new information... jobs in information technology: february , new this week back end drupal web developer,&# ;multnomah county library, portland, or distance education &# ; outreach librarian,&# ;winona state university,&# ;winona, mn senior systems specialist,&# ;prairiecat, library consortium,&# ;coal valley, il training and outreach coordinator,&# ;prairiecat, library consortium,&# ;coal valley, il visit the lita jobs site for additional job listings and information on submitting your own job posting. deadline extended to march – submit a proposal to teach for lita the deadline to submit lita education proposals has been extended to march th. we&# ;re seeking instructors passionate about library technology topics to share their expertise and teach a webinar, webinar series, or online course for lita this year. instructors receive a $ honorarium for an online course or $ for a webinar, split among instructors. check out our list of current and past course offerings to see what topics have been covered recently. be part of another slate of compelling and useful online education programs this year! submit your lita education proposal today! for questions or comments related to teaching for lita, contact us at lita@ala.org or ( ) - . the census starts in two weeks — are your computers ready? post courtesy of gavin baker, ala office of public policy and advocacy, deputy director, public policy and government relations on march , millions of american households will begin receiving mailings inviting them to respond to the census. to get an accurate count, everyone has to respond – if they don’t, our libraries and communities will lose needed funding. as the mailings arrive, patrons may come to your library with questions – and, with a new option to respond online, to complete the questionnaire using the library’s computers or internet. to help you prepare, ala has a new, two-page tip sheet, &# ;libraries and the census: responding to the census,&# ; that provides key dates, options for responding, and advice for libraries preparing for the census. for instance, the tip sheet explains these important facts: ways to respond: households can respond to the census online, by phone, or by mail... news regarding the future of lita after the core vote dear lita members, we&# ;re writing about the implications of lita’s budget for the upcoming - fiscal year, which starts september , . we have reviewed the budget and affirmed that lita will need to disband if the core vote does not succeed. since the great recession, membership in professional organizations has been declining consistently. lita has followed the same pattern and as a result, has been running at a deficit for a number of years. each year, lita spends more on staff, events, equipment, software, and supplies than it takes in through memberships and event registrations. we were previously able to close our budgets through the use of our net asset balance which is, in effect, like a nest egg for the division. of course, that could not continue indefinitely. our path towards sustainability has culminated in the proposal to form core: leadership, infrastructure, futures. the new division would come with... boards of alcts, lita and llama put core on march ballot the boards of the association for library collections &# ; technical services (alcts), library information technology association (lita) and the library leadership &# ; management association (llama) have all voted unanimously to send to members their recommendation that the divisions form a new division, core: leadership, infrastructure, futures.&# ; alcts, lita and llama will vote on the recommendation during the upcoming american library association (ala) election. if approved by all three memberships, and the ala council, the three long-time divisions will end operations on august , , and merge into core on september . members of the three boards emphasized that core will continue to support the groups in which members currently find their professional homes while also creating new opportunities to work across traditional division lines. it is also envisioned that core would strengthen member engagement efforts and provide new career-support services. if one or more of the division memberships do not... jobs in information technology: february , new this week librarian (emphasis in user experience and technology), chabot college, hayward, ca librarian ii (ils admin &# ; tech services), duluth public library, duluth, mn distance education &# ; outreach librarian, winona state university, winona, mn head, digital initiatives &# ; tisch library, tufts university, medford, ma online learning and user experience librarian, ast or asc professor, siu edwardsville, edwardsville, il discovery and systems librarian, hamilton college, clinton, ny visit the lita jobs site for additional job listings and information on submitting your own job posting. early-bird registration ends march st for the exchange with stimulating programming, including discussion forums and virtual poster sessions, the exchange will engage a wide range of presenters and participants, facilitating enriching conversations and learning opportunities in a three-day, fully online, virtual forum. programming includes keynote presentations from emily drabinski and rebekkah smith aldrich, and sessions focusing on leadership and change management, continuity and sustainability, and collaborations and cooperative endeavors. the exchange will take place may , , and . in addition to these sessions, the exchange will offer lightning rounds and virtual poster sessions. for up-to-date details on sessions, be sure to check the exchange website as new information is being added regularly. early-bird registration rates are $ for alcts, lita, and llama members, $ for ala individual members, $ for non-members, $ for student members, $ for groups, and $ for institutions. early-bird registration ends march . want to register your group or institution? groups watching the... jobs in information technology: february , new this week upper school librarian (pdf), st. christopher&# ;s school, richmond, va diversity and engagement librarian, ast or asc professor, siu edwardsville, edwardsville, il repository services manager, washington university, saint louis, mo information technology librarian, albin o. kuhn library &# ; gallery (umbc), baltimore, md visit the lita jobs site for additional job listings and information on submitting your own job posting. lita blog call for contributors we&# ;re looking for new contributors for the lita blog! do you have just a single idea for a post or a series of posts? no problem! we&# ;re always looking for guest contributors with new ideas. do you have thoughts and ideas about technology in libraries that you&# ;d like to share with lita members? apply to be a regular contributor! if you&# ;re a member of lita, consider either becoming a regular contributor for the next year or submitting a post or two as a guest. apply today! learn the latest in library ux with this lita webinar there’s a seat waiting for you… register for this lita webinar today! how to talk about library ux &# ; redux presenter: michael schofield librarian / director of engineering, whereby.us wednesday, march , : – : pm central time the last time we did this webinar was in &# ; and a lot&# ;s changed. the goal then was to help establish some practical benchmarks for how to think about the user experience and ux design in libraries, which suffered from a lack of useful vocabulary and concepts: while we might be able to evangelize the importance of ux, libuxers struggled with translating their championship into the kinds of bureaucratic goals that unlocked real budget for our initiatives. it&# ;s one thing to say, &# ;the patron experience is critical!&# ; it&# ;s another thing to say, &# ;the experience is critical &# ; so pay for optimalworkshop, or hire a ux librarian, or give me a... joint working group on ebooks and digital content in libraries john klima, the lita representative to the working group on ebooks and digital content, recently agreed to an interview about the latest update from ala midwinter . watch the blog for more updates from john about the working group in the coming months! what is the mission and purpose of the working group on ebooks and digital content? quoting from the minutes of the ala executive board fall meeting in october of : [the purpose of this working group is] to address library concerns with publishers and content providers specifically to develop a variety of digital content license models that will allow libraries to provide content more effectively, allowing options to choose between one-at-a-time, metered, and other options to be made at point of sale; to make all content available in print and for which digital variants have been created to make the digital content equally available to libraries without... forum call for proposals lita, alcts and llama are now accepting proposals for the forum, november - at the renaissance baltimore harborplace hotel in baltimore, md. intention and serendipity: exploration of ideas through purposeful and chance connections submission deadline: march , our library community is rich in ideas and shared experiences. the forum theme embodies our purpose to share knowledge and gain new insights by exploring ideas through an interactive, hands-on experience. we hope that this forum can be an inspiration to share, finish, and be a catalyst to implement ideas…together. we invite those who choose to lead through their ideas to submit proposals for&# ;sessions or preconference workshops, as well as&# ;nominate keynote speakers. this is an opportunity to share your ideas or unfinished work, inciting collaboration and advancing the library profession forward through meaningful dialogue. we encourage diversity in presenters from a wide range of background, libraries, and experiences. we deliberately... lita announces the excellence in children’s and young adult science fiction notable lists the lita committee recognizing excellence in children’s and young adult science fiction presents the excellence in children’s and young adult science fiction notable lists. the lists are composed of notable children’s and young adult science fiction published between november and october and organized into three age-appropriate categories. the annotated lists will be posted on the website at&# ;www.sfnotables.org. the golden duck notable picture books list is selected from books intended for pre-school children and very early readers, up to years old. recognition is given to the author and the illustrator: field trip to the moon by john hare. margaret ferguson books hello by aiko ikegami. creston books how to be on the moon by viviane schwarz. candlewick press out there by tom sullivan. balzer + bray the babysitter from another planet by stephen savage. neal porter books the space walk by brian biggs. dial books for young... jobs in information technology: february , new this week (tenure-track) senior assistant librarian, sonoma state universityrohnert park, ca data services librarian for the sciences, harvard universitycambridge, ma visit the lita jobs site for additional job listings and information on submitting your own job posting. teach for lita: submit proposals by february reminder: the deadline to submit lita education proposals is february th. please share our cfp with your colleagues. we are seeking instructors passionate about library technology topics to share their expertise and teach a webinar, webinar series, or online course for lita this year. all topics related to the intersection of technology and libraries are welcomed, including: machine learning it project management data visualization javascript, including: jquery, json, d .js library-related apis change management in technology big data, high performance computing python, r, github, openrefine, and other programming/coding topics in a library context supporting digital scholarship/humanities virtual and augmented reality linked data implementation or participation in open source technologies or communities open educational resources, creating and providing access to open ebooks and other educational materials managing technology training diversity/inclusion and technology accessibility issues and library technology technology in special libraries ethics of library technology (e.g., privacy concerns, social justice implications) library/learning management... jobs in information technology: january , new this week stem, instruction, and assessment librarian,&# ;mcdaniel college, westminster, md data science/analysis research librarian,&# ;hamilton college, clinton, ny electronic resources librarian, brown university, providence, ri systems librarian, brown university, providence, ri head, technical services, brown university, providence, ri network and systems administrator,&# ;st. lawrence university, canton, ny visit the lita jobs site for additional job listings and information on submitting your own job posting. emily drabinski, rebekkah smith aldrich to deliver keynotes at the exchange virtual forum the association for library collections and technical services (alcts), the library information technology association (lita) and the library leadership and management association (llama) have announced that emily drabinski and rebekkah smith aldrich will deliver keynote addresses at the exchange virtual forum. the theme for the exchange is &# ;building the future together,&# ; and it will take place on the afternoons of may , and . each day has a different focus, with day exploring leadership and change management; day examining continuity and sustainability; and day focusing on collaborations and cooperative endeavors. drabinski&# ;s keynote will be on may , and smith aldrich&# ;s will be on may . emily drabinski is the critical pedagogy librarian at mina rees library, graduate center, city university of new york (cuny). she is also the liaison to the school of labor and urban studies and other cuny masters and doctoral programs. drabinski&# ;s research includes... jobs in information technology: january , new this week information technology and web services (itws) department head, auraria library, denver, co visit the lita jobs site for additional job listings and information on submitting your own job posting. advice for the new systems librarian – building relationships . advice for the new systems librarian &# ; building relationships, part previous articles in this series: building relationships, helpful resources, a day in the life i am at the two-year mark of being in my role as systems librarian at jacksonville university, and i continue to love what i do. i am working on larger-scale projects and continuing to learn new things every week. there has not been a challenge or new skill to learn yet that i have been afraid of. my first post in this series highlighted groups and departments that may be helpful in learning your new role. now that i’m a little more seasoned, i have had the opportunity to work with even more departments and individuals at my institution on various projects. some of these departments may be unique to me, but i would imagine you would find counterparts where you work. the academic technology... jobs in information technology: january , new this week performing and visual arts librarian, butler university, indianapolis, in librarian, the college of lake county, grayslake, il user experience (ux) librarian, unc charlotte, j. murrey atkins library, charlotte, nc southeast asia digital librarian, cornell university, ithaca, ny head of digital infrastructure services at uconn library, university of connecticut, storrs, ct visit the lita jobs site for additional job listings and information on submitting your own job posting. lita education call for proposals for what library technology topics are you passionate about? have something you can help others learn? lita invites you to share your expertise with an international audience! our courses and webinars are based on topics of interest to library technology workers and technology managers at all levels in all types of libraries. taught by experts, they reach beyond physical conferences to bring high quality continuing education to the library world. we deliberately seek and strongly encourage submissions from underrepresented groups, such as women, people of color, the lgbtqa+ community, and people with disabilities. submit a proposal by february th to teach a webinar, webinar series, or online course for winter/spring/summer/fall . all topics related to the intersection of technology and libraries are welcomed, including: machine learning it project management data visualization javascript, including: jquery, json, d .js library-related apis change management in technology big data, high performance computing python, r, github, openrefine,... jobs in information technology: january , new this week web services &# ; discovery manager, american university library, washington, dcsenior research librarian, finnegan, washington, dc electronic resources and discovery librarian, auburn university, al discovery &# ; systems librarian, california state university, dominguez hills, carson, ca visit the lita jobs site for additional job listings and information on submitting your own job posting. ux “don’ts” we still need from erika hall the second edition of erika hall’s just enough research dropped october ; although this excellent volume was previously unknown to me i am taking the opportunity now to consume, embody, and evangelize hall’s approach to user research. or, as hall might put it, i’m a willing convert to the gospel of “enoughening”. hall is a seasoned design consultant and co-founder of mule design studio but her commercial approach is tempered by a no-nonsense attitude that makes her solutions and suggestions palatable to a small ux team such as my own at indiana university bloomington libraries. rather than conduct a formulaic book review of just enough research, i want to highlight some specific things hall tells the reader not to do in their ux research. this list of five “don’ts” summarize hall’s tone, style, and approach. it will also highlight the thesis of the second edition’s brand new chapter on surveys.... jobs in information technology: december , new this week vice provost and university librarian, university of oregon,&# ;eugene, or data migration specialist (telecommuting position), bywater solutions,&# ;remote position research librarian, oak ridge national laboratory, oak ridge, tn visit the lita jobs site for additional job listings and information on submitting your own job posting. announcing the new lita elearning coordinator we are proud to announce that kira litvin will be the new lita elearning coordinator. litvin has been the continuing education coordinator at the colorado school for public health for the past six months. she provides distance/online learning library services and instruction and works regularly with other librarians, instructional designers, faculty, and educators to collaborate on instructional delivery projects. &# ;i am passionate about being a librarian and working with people in an online environment! &# ;for the past nine years i have worked with libraries that are exclusively online. my roles include administering and managing electronic library systems, including springshare products, and providing virtual reference and instruction to students, faculty and staff. more recently i have transitioned to working as an elearning instructional designer which means i design and develop instructional content available for asynchronous learning and professional development. as online learning continues to grow, i believe that libraries need to... submit a nomination for awards and scholarships hugh c. atkinson memorial award the award honors the life and accomplishments of hugh c. atkinson by soliciting nominations and recognizing the outstanding accomplishments of an academic librarian who has worked in the areas of library automation or library management and has made contributions (including risk taking) toward the improvement of library services or to library development or research. nomination deadline: january , winner receives a cash award and a plaque. learn more about the requirements for the atkinson memorial award. ex libris student writing award the lita/ex libris student writing award is given for the best unpublished manuscript on a topic in the area of libraries and information technology written by a student or students enrolled in an ala-accredited library and information studies graduate program. application deadline: february , winner receives a $ , cash and a plaque. learn more about the requirements for the ex libris student... submit a nomination for the hugh c. atkinson memorial award lita, acrl, alcts, and llama invite nominations for the hugh c. atkinson memorial award. please submit your nominations by january , . the award honors the life and accomplishments of hugh c. atkinson by recognizing the outstanding accomplishments of an academic librarian who has worked in the areas of library automation or library management and has made contributions (including risk taking) toward the improvement of library services or to library development or research. winners receive a cash award and a plaque. this award is funded by an endowment created by divisional, individual, and vendor contributions given in memory of hugh c. atkinson. the nominee must be a librarian employed in one of the following during the year prior to application for this award: university, college, or community college library non-profit consortium, or a consortium comprised of non-profits that provides resources/services/support to&# ; academic libraries the nominee must have a minimum... core update – / / greetings again from the steering committee of core: leadership, infrastructure, futures, a proposed division of ala. coming up this friday, december is the last of four town halls we are holding this fall to share information and elicit your input. please join us! register for town hall today. alcts, lita, and llama division staff will lead this town hall with a focus on core’s mission, vision, and values; benefits organizationally; benefits to members; and opportunities in the future. our speakers will be jenny levine (lita executive director), julie reese (alcts deputy executive director), and kerry ward (llama executive director and interim alcts executive director). we’re excited to share an updated core proposal document for ala member feedback and review, strengthened by your input. we invite further comments on this updated proposal through sunday, december . meanwhile, division staff will incorporate your comments and finalize this proposal document for... jobs in information technology: december , new this week senior specialist &# ; makerspace,&# ;middle tennessee state university, walker library,&# ;murfreesboro, tn user experience librarian,&# ;auburn university,&# ;auburn university, al visit the lita jobs site for additional job listings and information on submitting your own job posting. announcing the new lita blog editor we are proud to announce that jessica gilbert redman will be the new editor of the&# ;lita&# ;blog.&# ; gilbert redman has been the web services librarian at the university of north dakota for the past three years. she coordinates and writes for the library blog and maintains the library website. she has completed a post-graduate certificate in user experience and always seeks to ensure that end users are able to easily find the information they need to complete their research. additionally, she realizes communication is the key component in any relationship, be it between libraries and their users or between colleagues, and she always strives to make communication easier for all involved. &# ;i am excited to become more involved in lita, and i think the position of lita blog editor is an excellent way to meet more people within lita and ala, and to maintain a finger on the pulse of new... jobs in information technology: december , new this week digital discovery librarian/assistant librarian, miami university, oxford, oh visit the lita jobs site for additional job listings and information on submitting your own job posting. jobs in information technology: november , new this week web and digital scholarship technologies librarian,&# ;marquette university libraries, milwaukee, wi digital access and metadata librarian,&# ;marquette university libraries, milwaukee, wi librarian (san ramon campus),&# ;contra costa community college district,&# ;san ramon, ca visit the lita jobs site for additional job listings and information on submitting your own job posting. support lita scholarships this #givingtuesday it’s almost #givingtuesday, so we’re highlighting the difference that lita scholarships can make, and inviting you to join us in increasing access to lita events by donating to our scholarship fund today. you can help us to provide more scholarships to events like avramcamp and lita forum, as well as sponsor emerging leaders, with your donation today! your donation of $ could open up untold opportunities for other library technology professionals. “the lita scholarship afforded me the opportunity to present at the avramcamp and ala conference. it was an incredible opportunity to network with dozens of information professionals, build connections with people in the field, ask them all of my questions and exchange our technical acumen and job experiences. as a result, i have been offered two interviewing opportunities that were an incredibly valuable experience for my career development. i am very grateful to lita for the opportunity to... jobs in information technology: november , new this week metadata specialist iii, metadata services, the new york public library, new york, ny eresources librarian,&# ;university of maryland, baltimore county, baltimore, md multiple librarian positions,&# ;george washington university, washington dc information technology analyst,&# ;san mateo county libraries, san mateo county, ca visit the lita jobs site for additional job listings and information on submitting your own job posting. call for blog coordinator for the exchange: an alcts/lita/llama collaboration the exchange: an alcts/lita/llama collaboration brings together experiences, ideas, expertise, and individuals from the three ala divisions. broadly organized around the theme of “building the future together,” the exchange will examine the topic in relation to collections, leadership, technology, innovation, sustainability, and collaborations. participants from diverse areas of librarianship will find the three days of presentations, panels, and lightning rounds both thought-provoking and highly relevant to their current and future career paths. the exchange will engage a wide range of presenters and participants, facilitating enriching conversations and learning opportunities. divisional members and non-members alike are encouraged to register and bring their questions, experiences, and perspectives to the events. as part of the conference experience, the exchange plans to host regular blog posts in advance of the conference. blog posts will serve multiple purposes: generate excitement and interest in content, encourage participation outside of simply watching presentations, and provide an avenue... the exchange call for proposals and informational webinar alcts, lita, and llama are now accepting proposals for the exchange: building the future together, a virtual forum scheduled for may , , and , . the twelve hour virtual event will take place over three afternoons, featuring the following themes and topics: day &# ; leadership and change management day &# ; continuity and sustainability day &# ; collaborations and cooperative endeavors session formats the exchange will feature the following session formats: full-session proposals presenters prepare content for a -minute session, with an additional -minute q&# ;a period for all presenters. full-session proposals may include multiple presentations with content that is topically related. lightning round each participant is given five minutes to give a presentation. at the end of the lightning round, there will be a - -minute q&# ;a period for all presenters in the session. topics for lightning rounds related to innovative projects or research are encouraged. proposals will be... registration is now open for the exchange in may , join alcts, lita, and llama for an exciting and engaging virtual forum. registration is now open! &# ; the exchange: an alcts/lita/llama collaboration brings together experiences, ideas, expertise, and individuals from the three ala divisions. broadly organized around the theme of “building the future together,” the exchange will examine the topic in relation to collections, leadership, technology, innovation, sustainability, and collaborations. participants from diverse areas of librarianship will find the three days of presentations, panels, and lightning rounds both thought-provoking and highly relevant to their current and future career paths. the exchange will engage a wide range of presenters and participants, facilitating enriching conversations and learning opportunities. divisional members and non-members alike are encouraged to register and bring their questions, experiences, and perspectives to the events. “building on the rich educational traditions of the three divisions, the exchange provides the opportunity to break down silos and explore synergies... core call for comment greetings again from the steering committee of&# ;core: leadership, infrastructure, futures, a proposed division of ala. the steering committee welcomes comments on the&# ;draft division proposal documentation&# ;through november th. please join the conversation! your perspectives and input are shaping the identity and priorities of the proposed division. we’re asking for you to respond to the documents with key questions in mind, including: does this make sense to someone new to alcts/ lita/ llama? does this piece of the plan reflect how members want the new division to function? are there any points that are cause for concern? if you’re interested in helping us in the review process or other work ahead, please&# ;consider volunteering&# ;for&# ;core.&# ;we’re eager to collaborate with you! we’re working hard to ensure everyone can participate in the&# ;core&# ;conversation, so please&# ;let us know&# ;what could make&# ;core&# ;a compelling and worthy division home for you.&# ;keep the feedback and input coming! full details for all our&# ;upcoming events&# ;are... lis students: apply for the larew scholarship for tuition help the library and information technology association (lita) and baker &# ; taylor are accepting applications for the lita/christian (chris) larew memorial scholarship for those who plan to follow a career in library and information technology, demonstrate potential leadership, and hold a strong commitment to library automation. the winner will receive a $ , check and a citation.&# ;the application form is open through march , . criteria for the scholarship includes previous academic excellence, evidence of leadership potential, and a commitment to a career in library automation and information technology. candidates should illustrate their qualifications for the scholarships with a statement indicating the nature of their library experience, letters of reference and a personal statement of the applicant’s view of what they can bring to the profession.&# ;winners must have been accepted to&# ;a master of library science (mls) program recognized by the american library association. references, transcripts, and other documents must be postmarked no... jobs in information technology: november , new this week full time faculty &# ; non tenure track,&# ;sjsu school of information, san jose, ca digital collections librarian, union college,&# ;schenectady, ny web services librarian,&# ;university of oregon libraries, eugene, or galileo programmer/analyst,&# ;university of georgia libraries, athens, ga visit the lita jobs site for additional job listings and information on submitting your own job posting. lita opens call for innovative lis student writing award for the library and information technology association (lita), a division of the american library association (ala), is pleased to offer an award for the best unpublished manuscript submitted by a student or students enrolled in&# ;an ala-accredited graduate program. sponsored by&# ;lita&# ;and&# ;ex&# ;libris, the award consists of $ , , publication in&# ;lita’s&# ;referred journal,&# ;information technology and libraries (ital), and a certificate. the deadline for submission of the manuscript is february , . the award recognizes superior student writing and is intended to enhance the professional development of students. the manuscript can be written on any aspect of libraries and information technology. examples include, but are not limited to, digital libraries, metadata, authorization and authentication, electronic journals and electronic publishing, open source software, distributed systems and networks, computer security, intellectual property rights, technical standards, desktop applications, online catalogs and bibliographic systems, universal access to technology, and library consortia. to be eligible, applicants must follow&# ;these&# ;guidelines&# ;and fill out&# ;the application form&# ;(pdf).... jobs in information technology: november , new this week open educational resources production manager, oregon state university &# ; ecampus, corvallis, or user experience librarian, northwestern university, evanston, il institute for clinical and translational research (ictr) librarian, university of maryland, baltimore, baltimore, md director of collections &# ; access, wheaton college, norton, ma visit the lita jobs site for additional job listings and information on submitting your own job posting. nominate a colleague doing cutting edge work in tech education for the lita library hi tech award nominations are open&# ;for the &# ;lita/library hi tech award, which is given each year to an individual or institution for outstanding achievement in educating the profession about cutting edge technology within the field of library and information technology. sponsored by the&# ;library and information technology association&# ;(lita) and library hi tech, the award includes a citation of merit and a $ , stipend provided by&# ;emerald publishing, publishers of library hi tech. the deadline for nominations is december , . the award, given to either a living individual or an institution, may recognize a single seminal work or a body of work created during or continuing into the five years immediately preceding the award year. the body of work need not be limited to published texts but can include course plans or actual courses and/or non-print publications such as visual media. awards are intended to recognize living persons rather than to honor the deceased; therefore,... propose a topic for the ital “public libraries leading the way” column information technology and libraries (ital), the quarterly open-access journal published by ala’s library information technology association, is looking for contributors for its regular “public libraries leading the way” column. this column highlights a technology-based innovation or approach to problem solving from a public library perspective. topics we are interested in include the following, but proposals on any other technology topic are welcome. -d printing and makerspaces civic technology drones diversity, equity, and inclusion and technology privacy and cyber-security virtual and augmented reality artificial intelligence big data internet of things robotics geographic information systems and mapping library analytics and data-driven services anything else related to public libraries and innovations in technology to propose a topic, use this brief form, which will ask you for three pieces of information: your name your email address a brief ( - word) summary of your proposed column that describes your library, the technology you wish to... alcts, lita and llama collaborate for virtual forum the association for library collections &# ; technical services (alcts), the library and information technology association (lita) and the library leadership &# ; management association (llama) have collaborated to create the exchange, an interactive, virtual forum designed to bring together experiences, ideas, expertise and individuals from these american library association (ala) divisions. modeled after the alcts exchange, the exchange will be held may , may and may in with the theme “building the future together.” as a fully online interactive forum, the exchange will give participants the opportunity to share the latest research, trends and developments in collections, leadership, technology, innovation, sustainability and collaborations. participants from diverse areas of librarianship will find the three days of presentations, panels and activities both thought-provoking and highly relevant to their current and future career paths. the exchange will engage an array of presenters and participants, facilitating enriching conversations and learning opportunities.... submit your annual meeting request by feb the lita meeting request form is now open for the ala annual conference in chicago, il. all lita committee and interest group chairs should use it to let us know if you plan to meet at annual. we&# ;re looking forward to seeing what you have planned. the deadline to submit your meeting request is friday, february , . we&# ;re going to change how we&# ;ve listed meetings in the past. if you do not submit this form, your group will not be included in the list of lita session on our website, the online scheduler, or the print program. while we&# ;ll still hold the joint chairs meeting on saturday from : - : am and use that same room for committee and ig meetings from : - : am, your group will only be listed if you submit this form. you should also use it if you want to request a meeting on a different day... submit a nomination for the prestigious kilgour technology research award lita and oclc invite nominations for the frederick g. kilgour award for research in library and information technology. submit your nomination no later than december , . the kilgour research award recognizes research relevant to the development of information technologies, in particular research showing promise of having a positive and substantive impact on any aspect of the publication, storage, retrieval, and dissemination of information or how information and data are manipulated and managed. the winner receives $ , cash, an award citation, and an expense-paid trip (airfare and two nights lodging) to the ala annual conference in chicago, il. nominations will be accepted from any member of the american library association. nominating letters must address how the research is relevant to libraries; is creative in its design or methodology; builds on existing research or enhances potential for future exploration; and/or solves an important current problem in the delivery of... core update – october , greetings again from the steering committee of core: leadership, infrastructure, futures, a proposed division of ala. thank you for all of your questions and feedback about the proposed new division!&# ;the steering committee has been revising core documents based on what we’ve heard from you so far in order to share draft bylaws and other information with you soon. we want you to know that we are continuing to listen and incorporate the feedback you’re providing via town halls, twitter chats, the core feedback form, and more.&# ; in our next steering committee meeting, we will be discussing how we can support the operational involvement of interested volunteers. if you have ideas on how members should be involved, please share them with us through&# ;the feedback form.&# ; we’re working hard to ensure everyone can participate in the core conversation, so please&# ;let us know&# ;what could make core a compelling and worthy division home for... jobs in information technology: october , new this week metadata &# ; research support specialist, open society research services, open society foundations, new york, ny head of public services in the daniel library, the citadel, the military college of south carolina, charleston, sc engineering and science liaison, mit, cambridge, ma head of technical services &# ; library, the citadel, the military college of south carolina, charleston, sc analyst programmer , oregon state university libraries and press, corvallis, or collection information specialist, isabella stewart gardner museum, boston, ma visit the lita jobs site for additional job listings and information on submitting your own job posting. jobs in information technology: october , new this week metadata librarian for distinctive collections, mit, cambridge, ma electronic access librarian, university of rochester, rochester, ny dean, university libraries, university of northern colorado, greeley, co administrative/metadata specialist, asr international corp., monterey, ca core systems librarian, university of oregon libraries, eugene, or visit the lita jobs site for additional job listings and information on submitting your own job posting. september ital issue now available the september issue of information technology and libraries (ital) is available now. in this issue, ital editor ken varnum announces six new members of the ital editorial board. our content includes a recap of emily morton-owens&# ; president&# ;s inaugural message, &# ;sustaining lita&# ;, discussing the many ways lita strives to provide a sustainable member organization. in this edition of our &# ;public libraries leading the way&# ; series, thomas lamanna discusses ways libraries can utilize their current resources and provide ideas on how to maximize effectiveness and roll new technologies into operations in &# ;on educating patrons on privacy and maximizing library resources.&# ; featured articles: &# ;library-authored web content and the need for content strategy,&# ; courtney mcdonald and heidi burkhardt increasingly sophisticated content management systems (cms) allow librarians to publish content via the web and within the private domain of institutional learning management systems. “libraries as publishers”may bring to mind roles in scholarly communication and... jobs in information technology: october , new this week information research specialist, harvard business school, boston, ma - library residency program (provost’s postdoctoral fellowship), new york university, division of libraries, new york, ny executive director, library connection, inc, windsor, ct associate university librarian, cornell university, ithaca, ny visit the lita jobs site for additional job listings and information on submitting your own job posting. new vacancy listings are posted on wednesday afternoons. latest lita learnings there&# ;s a seat waiting for you&# ; register today for a lita webinar! guiding students through digital citizenship presenter: casey davis instructional designer (it), arizona state university wednesday, october , : &# ; : pm central time as academic librarians, we help build our students into digital citizens.&# ;it&# ;s our duty to make sure students have the tools and resources to be savvy tech users, become information literate, and understand the permanence of their digital actions. in this -minute webinar,&# ;you&# ;ll learn research-based best practices you can implement using the framework of the hero&# ;s journey&# ;without creating an additional burden on faculty, staff, and students. learning objectives for this program include: • an expanded understanding of digital citizenship within the context of college/university life •&# ;examining areas where increased awareness and practice is needed within the college/university community • creating authentic training for increasing digital citizenship within the college/university community view details&# ;and&# ;register here. in-house vs.... none marcedit . .x/macos . .x timelines – terry's worklog skip to content home about me marcedit homepage github page privacy policy terry's worklog marcedit . .x/macos . .x timelines by reeset / on january , / in marcedit i sent this to the marcedit listserv to provide info about my thoughts around timelines related to the beta and release. here’s the info. dear all, as we are getting close to feb. (when i’ll make the . beta build available for testing) – i wanted to provide information about the update process going forward. feb. : marcedit . download will be released. this will be a single build that includes both the and bit builds, dependencies, and can install if you have admin rights or non-admin rights. i expect to be releasing new builds weekly – with the goal of taking the beta tag off the build no later than april . marcedit . .x i’ll be providing updates for . .x till . comes out of beta. this will fold in some changes (mostly bug fixes) when possible. marcedit macos . .x i’ll be providing updates for macos . .x till . is out and out of beta marcedit macos . .x beta once marcedit . .x beta is out, i’ll be looking to push a . .x beta by mid-late feb. again, with the idea of taking the beta tag off by april (assuming i make the beta timeline) march marcedit macos . .x beta will be out and active (with weekly likely builds) marcedit . .x beta – testing assessed and then determine how long the beta process continues (with april being the end bookend date) marcedit . .x – updates continue marcedit macos . .x – updates continue april marcedit . .x comes out of beta marcedit . .x is deprecated marcedit macos . .x beta assessed – end bookend date is april th if above timelines are met may marcedit macos . .x is out of beta marcedit macos . .x is deprecated let me know if you have questions. post navigation marcedit . change/bug fix list marcedit . .x/ . .x (beta) updates leave a reply cancel reply your email address will not be published. required fields are marked * comment name * email * website notify me of follow-up comments by email. notify me of new posts by email. search for: terry's worklog terry's worklog © . - created by slicejack. job queue - ruby (on rails) - hirefire knowledge base hirefire knowledge base toggle navigation contact contact job queue - ruby (on rails) integrating ruby on rails with hirefire job queue is quick and easy when using the hirefire-resource rubygem. add it to your gemfile: gem "hirefire-resource" this library injects middleware into your stack, exposing an http get endpoint that hirefire will periodically request in order to get the latest information on your job queue(s). in addition to that, it also ships with a bunch of convenient macros for common worker libraries in the ruby ecosystem, including: sidekiq delayed job resque goodjob qu que queue classic bunny the library is open source, you can view the source here. integration for this example we'll use the popular sidekiq library. assuming that you have the following worker entry in your procfile: worker: bundle exec sidekiq you'll need to setup a corresponding worker resource for hirefire. create an initializer at app_root/config/initializers/hirefire_resource.rb and add the following code: hirefire::resource.configure do |config| config.dyno(:worker) do hirefire::macro::sidekiq.queue end end the sidekiq macro provided by hirefire-resource simply counts all the jobs in all of your queues and returns an integer representing that number of jobs. when hirefire requests the http get endpoint at the following url: http://your-domain.com/hirefire//info this library will transform your resource configuration to a json response in the following format: [{"name":"worker","quantity": }] hirefire will then read this information in order to determine the current queue size of your worker dynos. using this, in combination with your auto-scaling configuration in the hirefire user interface, hirefire will determine how and when to scale your worker dynos. the hirefire_token mentioned in the above code snippet can be found in the hirefire ui when creating/updating a dyno manager. it's just an environment variable that you'll need to add to your heroku application later on. multiple queues what if you have multiple queues? this is quite common, and with hirefire you can deal with each queue separately by scaling multiple procfile entries individually. for example, let's say your procfile not only contains a worker entry, but also an urgent_worker entry: worker: bundle exec sidekiq -q default urgent_worker: bundle exec sidekiq -q urgent all that you have to do to make this work is configure the following resources in your initializer: hirefire::resource.configure do |config| config.dyno(:worker) do hirefire::macro::sidekiq.queue(:default) end config.dyno(:urgent_worker) do hirefire::macro::sidekiq.queue(:urgent) end end with this in place the library will now transform your resources in a json response with the following format: [{"name":"worker","quantity": }, {"name":"urgent_worker","quantity": }] confirm that it works with all this in place, start the local development server and access the following url: http://localhost: /hirefire/development/info you should now see a json response containing your queue sizes. if you do, congratulations, you're done! deploy it to heroku. hirefire ui now that you've integrated hirefire-resource into your application and deployed it to heroku, log in to hirefire and create two managers named worker and urgent_worker and configure them based on your auto-scaling requirements. don't forget to add the previously mentioned hirefire_token environment variable to your heroku application. did this answer your question? thanks for the feedback there was a problem submitting your feedback. please try again later. yes no still need help? contact us contact us last updated on september , toggle search categories guides general information frequently asked questions manager options status legal no results found © final creation . powered by help scout contact us name email subject message upload file posts | mark a. matienzo skip to content w c svg main navigation menu about now posts notes music projects publications presentations press posts perfecting a favorite: oatmeal chocolate chip cookies publish date: november , tags: recipes food by mark a. matienzo i have a horrible sweet tooth, and i absolutely love oatmeal chocolate chip cookies. i tend to bake as a means to cope with stress, and of course, more often then that means making these cookies. after making many iterations, i’ve settled upon this recipe as the ultimate version to which all compare. (read more …) in memoriam and appreciation of rob casson ( - ) publish date: october , tags: code lib personal by mark a. matienzo the world lost one of its brightest and most charming lights earlier this week, rob casson. many of us knew rob through the code lib community and conferences and his work at miami university libraries. we miss his generosity, patience, sense of humor, and genuine kindness. those of us who got the chance to socialize with him also remember his passion for music, and some of us were even lucky to see live shows in the evenings between conference sessions and other social activities. on sunday, october at : pm pacific/ : pm eastern, those of us who knew him through code lib and the world of libraries are encouraged to gather to share our memories of him and to appreciate his life and work. please join me and my co-organizers, mike giarlo and declan fleming on zoom (registration required). robert casson (robcasson), jan - sep . photo: declan fleming. (read more …) first sota activation publish date: september , tags: ham radio by mark a. matienzo about a month ago, i got my ham radio license, and soon after i got pretty curious about summits on the air (sota), an award scheme focused on safe and low impact portable operation from mountaintops. while i like to hike, i’m arguably a pretty casual hiker, and living in california provides a surprising number of options within minutes driving time for sota newbies. (read more …) optimizing friction publish date: august , tags: indieweb music plan food by mark a. matienzo over and in response to the last few months, i’ve been reflecting about intentionality, and how i spend my time creating things. i have tried to improve the indiewebbiness of my site, and understanding what it means to “scratch my own itch”. this resonates particularly lately because it’s leading me to mull over which parts should be hard and easy. unsurprisingly, much of that is personal preference, and figuring out how i want to optimize from the perspective of user experience. friction in ux can be a powerful tool, part of what i’m trying to find is where i want to retain friction as it helps me remain intentional. (read more …) a hugo shortcode for embedding mirador publish date: july , tags: iiif hugo by mark a. matienzo i spent a little time over the last day or so trying to bodge together a shortcode for hugo to embed an instance of mirador. while it’s not quite as simple (or full-featured) as i’d like, it’s nonetheless a starting point. the shortcode generates a snippet of html that gets loaded into hugo pages, but (unfortunately) most of the heavy lifting is done by a separate static page that gets included as an ×  about library carpentry library carpentry is a global community teaching software and data skills to people working in library- and information-related roles. more services contact rss atom sitemap.xml links our code of conduct our privacy policy our style guide cc by . license the carpentries home - ilda about about ilda strategic areas community gender and inclusion developing technologies transparency and governance projects femicide data standard artificial intelligence global data barometer resources papers reports tools blog contact español english português do brasil we work towards an open, equal and data-driven region featured projects proyectos status: active ilda: the next generation proyectos status: active empatía proyectos status: active femicide data standardization proyectos status: active global data barometer proyectos status: active open data barometer (latin america and the caribbean edition – lac) news posts / / open data standards design behind closed doors? recursos / / data for development – a road ahead recursos / / flow to identify femicides dirección legal rincon / montevideo - uruguay impact hub av. , entre calle y , san pedro san josé - costa rica home researches projects blog contacto suscribite a nuestro newsletter: leave this field empty if you're human: contactanos seguinos seguinos en: apoyan: maisonbisson maisonbisson recent content on maisonbisson every journalist ryu spaeth on the dirty job of journalism: [e]very journalist […] at some point will have to face the morally indefensible way we go about our business: namely, using other people to tell a story about the world. not everyone dupes their subjects into trusting them, but absolutely everyone robs other people of their stories to tell their own. every journalist knows this flushed feeling, a mix of triumph and guilt, of securing the story that will redound glory unto them, not the subject. the three tribes of the internet authors primavera de filippi, juan ortiz freuler, and joshua tan outline three competing narratives that have shaped the internet: libertarian, corporate, and nationalist. this matters because our physical lives are now deeply intertwined with and codependent on our internet activities. the latest information about covid regulations in many communities is first released on twitter, for example. a declaration is a political act, which describes what should be done. a narrative is a political tool, which elaborates on why it should be done. happy d.b. cooper day d.b. cooper day is celebrated on this day, the saturday following thanksgiving, every year. vitaminwater's #nophoneforayear contest back in the before times, vitaminwater invited applicants to a contest to go a full year without a smartphone or tablet. it was partly in response to rising concerns over the effect of all those alerts on our brains. over , people clamored for the chance, but author elana a. mugdan’s entry stood out with an amusing video, and in february the company took away her iphone s and handed her a kyocera flip phone. membership-driven news media from the membership guide’s handbook/manifesto: journalism is facing both a trust crisis and a sustainability crisis. membership answers to both. it is a social contract between a news organization and its members in which members give their time, money, energy, expertise, and connections to support a cause that they believe in. in exchange, the news organization offers transparency and opportunities to meaningfully contribute to both the sustainability and impact of the organization. political bias in social media algorithms and media monetization models new reports reveal yet more structural political biases in consumption and monetization models. media monetization vs. internet advertising structural problems the internet is structured in favor of ad networks. ad spend grows approximately at the rate of inflation, but the inventory of pages on which those ads can appear grows with each new instagram post (about mm per day). internet advertising is far more automated than print, but the benefit goes to intermediaries and buyers. on average, publishers receive only about half of what advertisers pay for the advertising that appears in their publications. the argument against likes: aim for deeper, more genuine interactions it’s worth revisiting the infamous definition of social software as software that facilitates social encounters: “social software” is about making it easy for people to do other things that make them happy: meeting, communicating, and hooking up. […] the trick you want to accomplish is that when one person is using your software, it suddenly provides value to that person and their entire circle of friends, without the friends having had to do anything at all. paid reactions: virtual awards and tipping likes and reactions can stimulate more signal, leading to more user-activity on a site, but reactions that members pay to give to creators and other members on the site can be a revenue source. reddit introduced reddit gold in in an announcement that was surprisingly candid about their need to raise money. the original reddit gold was a combination of both premium, ad-free subscription and a type of reaction that allowed premium members to “gild” a post. reactions reactions in twitter dms. likes are a most perfect binary, but the meaning of a like can vary. consider the following interpretations of likes on instagram: this photo is incredibly inspiring to me and i want it hanging on my wall i like it when you like my photos and comments, so i will like your work as part of the social contract we have settled into i appreciate your comment on my photo and i want to recognize your participation it’s difficult, however, to “like” something with painful or negative emotions. “likes” vs. “faves” wikipedia credits vimeo for introducing the first like button as a more casual alternative to favorites. facebook introduced the feature in early , but twitter’s story is an interesting investigation into the differences a word or an icon can make. twitter switched from faves to likes on november . “you might like a lot of things, but not everything can be your favorite” explained twitter’s announcement. they continued: [w]e know that at times the [fave] star could be confusing, especially to newcomers. all my flickr photos, for indexing and archiving links to all my photos in flickr. photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size photo page, original size, large size honey cocktails: eau de lavender liquor.com’s recipe for eau de lavender, from a larger collection of cocktails with honey. they all look and sound delightful, but i can vouch for the eau de lavender. ingredients / oz tequila / oz fresh lemon juice / oz honey syrup egg white dash scrappy’s lavender bitters garnish: lavender sprig steps add all ingredients into a shaker and dry-shake (without ice). add ice and shake again to emulsify thoroughly. satellite tracking if you’re not reading skyriddles blog, then you’re not tracking the sky above. and you might have missed the re-discovery of a satellite launched in and lost for nearly years. as it turns out, there’s a lot of stuff that’s been forgotten up there, and quite a bit that some are trying to hide. the blog is an entertaining view into the world satellites, including communication, spy, weather, research, and the occasional probe going further afield. i'm missing restaurants now @nakedlunchsf was notable for having both a strong contender for the best burger in the city, and the best veggie sando. they kept the menu short and focused, and changed it up every few days based on what was in season and interesting. it was great food, but not fancy. the food, warm atmosphere, and a welcoming front of house team made the place a favorite for me and many others. when unzip fails on macos with utf unzip can fail on macos when utf- chars are in the archive. the solution is to use ditto. via a github issue: ditto -v -x -k --sequesterrsrc --rsrc filename.zip destinationdirectory tiktok vs. instagram connie chan: rather than asking users to tap into a video thumbnail or click into a channel, the app’s ai algorithms decide which videos to show users. the full-screen design of tiktok allows every video to unveil both positive and negative signals from users (positive = a like, follow, or watching until the end; negative = swipe away, press down). even the speed at which users swipe a video away is a relevant signal. swipegram template benjamin lee’s instructions and downloadable template to make panoramic carousel instagrams (aka #swipegram), as illustrated via his animation above. “it is clear that the books owned the shop... “it is clear that the books owned the shop rather than the other way about. everywhere they had run wild and taken possession of their habitat, breeding and multiplying, and clearly lacking any strong hand to keep them down.” words by agatha christie in photo: ️⃣ macleod’s books, vancouver, british columbia #penderstreet #downtownvancouver #mustbevancouver ️⃣ carlson & turner antiquarian books, portland, maine #portlandmaine #lovemaine at instagram. “life is like riding a bicycle... “life is like riding a bicycle. to keep your balance, you must keep moving.” —wisdom by albert einstein the bosch autoparts shop behind these commuters is now converted to an organic restaurant that anchors the northeast corner of copenhagen’s fashionable meatpacking district. at instagram. notes about spotify creator features spotify often gets bashed by top creators. the service pays just $ . per stream, but with million users listening to an average of hours per month, those streams can add up for creators who can get the listener’s attention. spotify verifies artists who then get additional benefits on the platform. some artists find success the traditional route, some optimize their work for the system, others work the system…and some really work it. exiftool examples i use for encoding analog camera details i’m a stickler for detail and love to add exif metadata for my film cameras to my scanned images. these are my notes to self about the data i use most often. i only wish exif had fields to record the film details too. random notes on instagram delete your photos deleting your old photos is recurring advice to photograpehers. jp danko suggests deleting photos just for simplicity of management. similarly, eric kim recommends it for decluttering as well. from another side, mike dixon deletes photos as part of his reflection and self-improvement efforts. and caleb kerr argues emotional attachment to old photos is bad for your portfolio and can be a barrier to creating better work. rebrand a number recommend starting from scratch. every media has its tastemakers and influencers every media, network, or platform has would-be influencers or promoters who can help connect consumers with creators. don’t mistake the value of these tastemakers, and be sure to find a place for them to create new value for your platform. storehouse: the most wonderful story sharing flop ever storehouse shuttered in summer , just a couple years after they launched, but the app and website introduced or made beautiful a few features that remain interesting now. “he had ridden his horse into the saloon on a dare... “he had ridden his horse into the saloon on a dare—his practice was always to accept dares; it spiced life up a little.” words: larry mcmurtry at instagram. editorial efforts at scale anywhere you can find content—even user-generated content—you’ll find a content strategy and editors ensuring that content aligns to strategy (and to community standards). somewhere, something incredible is waiting... “somewhere, something incredible is waiting to be known” words commonly misattributed to carl sagan, but most likely written by reporter sharon begley the eight-dish submillimeter array on mauna kea in hawaii was one of a global federation of radio telescopes used to produce the world’s first images of a black hole earlier this year. from wikipedia: “the radio frequencies accessible to this telescope range from – gigahertz ( . don’t make it dull... don’t make it dull if thou can’t make it colorful words by arrow at instagram. about that table of “hidden rules among classes” the following table has been circulating recently. i sourced it to framework for understanding poverty: a cognitive approach by ruby payne, phd, who sells educational materials and consulting services through her company, aha! process. poor middle class wealthy possessions people. things. one-of-a-kind objects, legacies, pedigrees. money to be used, spent. to be managed. to be conserved invested. personality is for entertainment. the couple in the booth next door... the couple in the booth next door, just been up all night smoking cigarettes and talking about life as the waitress hovers with nothing else to do but daydream about the cop she wants to screw words by david e oprava silver crest donut shop, san francisco #americansquares at instagram. maybe life is all about twirling under one of those midnight skies... maybe life is all about twirling under one of those midnight skies, cutting a swathe through the breeze and gently closing your eyes. words by sanober khan at instagram. design exercises for product leadership in a way, my career in tech started with graphic design. and as a not very good graphic designer, i eagerly looked for ways to improve my work. nothing beats inspiration and skillful effort, but sometimes finding inspiration is a matter of changing how you look at the subject. there are some exercises that can help with that and sometimes offer a shortcut to inspiration when all else fails. consider an illustration project in which you need to represent a subject. sai morgan you say rolls i say royce you say god give me a choice you say lord i say christ i don’t believe in peter pan frankenstein or superman sai rode by on his bike and i invited him over for a photo. i’ve tried to send him the photos to the email address he gave me (s morgan@[redacted]), but i haven’t heard back. before i built a wall i’d... before i built a wall i’d ask to know what i was walling in or walling out, and to whom i was like to give offense. words by bob frost at instagram. normcore, mysticore, streetwear, and other words for “fashion” normcore normcore, at its most basic level, is fashionable people choosing to dress unfashionably, which is hardly a new idea. a case could be made that normcore has existed since the popularization of ready-to-wear clothing in the early s. any clothing that is not made by hand or commissioned specifically for a person is ready-to-wear. almost immediately after the creation of ready-to-wear fashion, it became a trend to wear what everyone else was wearing, especially if you were a wealthy person not used to sharing clothes with the commoners. how big is s ? tl;dr: somewhere between - exabytes. up in the air i go flying again/up in the air and down! how do you like to go up in a swing, up in the air so blue? up in the air i go flying again, up in the air and down! words by robert louis stevenson music cc-by-nc-sa: “cocek” by the underscore orkestra the swing is an installation at the #bombaybeachbiennale titled “the water ain’t that bad, it’s just salty” by @damonjamesduke and @ssippi with the bombay bunny club glitter, glitter, everywhere near the entrance, metal shelves taller than a man were laden with over one thousand jumbo jars of glitter samples arranged by formulation, color, and size: emerald hearts, pewter diamonds, and what appeared to be samples of the night sky collected from over the atlantic ocean. there were neon sparkles so pink you have only seen them in dreams, and rainbow hues that were simultaneously lilac and mint and all the colors of a fire. it's , and we need to fight for the future of the internet there are obviously conflicting opinions about how to piece together new and complex regulation, legislation, or tech innovation. but this has been true throughout history whenever a new idea begins to be broadly adapted. before the internet, we had to figure out how to manage cars and electricity and steam power and even the use of the written word (which many, including socrates, actually argued against). the internet is no different. the myth of the rv the myth of an rv is that you can go anywhere and bed down wherever you end up. the reality is that you can’t go just anywhere, and bedding down is not much more comfortable or convenient than tenting. astrophotography in san francisco from the space tourism guide: can you see the milky way in the bay area? unfortunately, it is very difficult to see the milky way in san francisco. between the foggy weather and the light pollution from million people, you can imagine that the faint light of our galaxy is lost to view. but c. roy yokingco argues: some people say the milky way cannot be photographed within miles of a major metropolitan area. well, this photo of the milky way was captured linear miles south of downtown san francisco, california. vijay selvaraj @iamvijayselvaraj looking like he’s modeling the new eos r for @canonusa while we were playing with strobes. at instagram. on building the plane while flying it “building a plane while flying it” or some variation has been used to describe situations in education ( ), education ( ). education ( ), health care, medicine, ride-hailing startups, business strategy, even fluffier business stories, and…this. and long before earning broad criticism for its use in tech, the phrase was vividly illustrated in an ad for electronic data systems (eds) that has since been appropriated for all the circumstances named above, as well as building churches: ed zak, photographer i found that hanging at red’s java house and wanted to learn more about ed zak. i mean, with ad copy like this, how can you not want to know more? find out why you should fly our to san francisco to shoot with a photographer who will make you eat at red’s java house and drive you around in this car. a photo shoot with ed zak is a photo shoot like no other. competing approaches to deadlines and excellence some people see deadlines as guidelines to aim for, not absolute dates by which a deliverable is expected by this view of deadlines as flexible guidelines can be seen throughout western culture, as exemplified by the ongoing, oft delayed brexit negotiations. however, deadlines also compete against other factors in any project. consider the three constraints in the project management triangle: a mathematical theory and evidence for hipster conformity in four parts academic publishes mathematical theory for conformance among hipsters: https://arxiv.org/pdf/ . .pdf mit tech review covers it, with a fancy photo illustration using a stock photo of a hipster-looking male: https://www.technologyreview.com/s/ /the-hipster-effect-why-anti-conformists-always-end-up-looking-the-same/ a hipster-looking male contacts mit tech review to loudly complain about their using a picture of him without asking: https://twitter.com/glichfield/status/ it turns out the hipster-looking male in the photo isn’t the same as the one who complained: https://twitter.com/glichfield/status/ the problem with content management systems in three tweet storms exhibit a: a series of tweets by gideon lichfield, editor of mit technology review and formerly of quarz, who asked: the legal case for emoji emoji are showing up as evidence in court more frequently with each passing year. between and , there was an exponential rise in emoji and emoticon references in us court opinions, with over percent of all cases appearing in , according to santa clara university law professor eric goldman, who has been tracking all of the references to “emoji” and “emoticon” that show up in us court opinions. inter-az cloud network performance archana kesavan of thousandeyes speaking at nanog reports that network traffic between azs within a single region is generally “reliable and consistent,” and that tested cloud providers offer a “robust regional backbone for [suitable for] redundant, multi-az architectures.” thousandeyes ran tests at ten minute intervals over days, testing bidirectional loss, latency, and jitter. kesavan reported the average inter-az latency for each tested cloud: aws azure gcp . default fonts that could have been i learned about serif and sans serif typefaces, about varying the amount of space between different letter combinations, about what makes great typography great. it was beautiful, historical, artistically subtle in a way that science can’t capture, and i found it fascinating. from steve jobs in stanford graduation address, explaining how he fell in love with typography during his time at reed college. he studied calligraphy like a monk, but…. spectre is here to stay as a result of our work on spectre, we now know that information leaks may affect all processors that perform speculation…. since the initial disclosure of three classes of speculative vulnerabilities, all major [cpu] vendors have reported affected products…. this class of flaws are deeper and more widely distributed than perhaps any security flaw in history, affecting billions of cpus in production across all device classes. from ross mcilroy, jaroslav sevcik, tobias tebbi, ben l. titzer, and toon verwaest (all of google) in spectre is here to stay; an analysis of side-channels and speculative execution. they continue: bare metal clouds are hard the problem, explains eclypsium, is that a miscreant could rent a bare-metal server instance from a provider, then exploit a firmware-level vulnerability, such as one in uefi or bmc code, to gain persistence on the machine, and the ability to covertly monitor every subsequent use of that server. in other words, injecting spyware into the server’s motherboard software, which runs below and out of sight of the host operating system and antivirus, so that future renters of the box will be secretly snooped on. indeed, the researchers found they could acquire, in the softlayer cloud, a bare-metal server, modify the underlying bmc firmware, release the box for someone else to use, and then, by tracking the hardware serial number, wait to re-provision server to see if their firmware change was still intact. and it was. bmc is the baseband management controller, the remote-controllable janitor of a server that has full access to the system. taking net promoter scores too far pick somebody in your life and send them a message asking them how their day is going on a scale of one to . that’s from author and game designer jane mcgonigal, quoted in reader’s digest. helvetica vs. univers univers was intrinsically superior to helvetica. it had a much larger family at the outset, with members compared to four in . more importantly, its family was logically designed with consistent weights and widths, something that helvetica never achieved until its redesign as neue helvetica in . univers’ characters, stripped of “unnecessary” elements such as the beard on ‘g’ or the curve on the tail of ‘y,’ were also more rationally designed. spielberg on the theater experience there’s nothing like going to a big dark theater with people you’ve never met before, and having the experience wash over you. steven spielberg, quoted in chaim gartenberg’s coverage of his speech at the cinema audio society’s cas awards. amusingly, according to gartenberg, spielberg has nothing against the streaming industry, he just really loves the theater experience and worries about what might happen to it. still, it’s hard not to imagine the filmmaker being a little bit swayed by the talk of hollywood irrelevance in the face of netflix. how pixar dominated the last three decades of special effects pixar’s renderman is the visual effects software hollywood didn’t think they needed (seriously, george lucas sold off the lucasfilm computer division in ). years later, after producing landmark visual effects for films such as terminator and jurassic park and many more, the academy of motion picture arts and sciences honored pixar and the creators of renderman with an award of merit in “for their significant advancements to the field of motion picture rendering as exemplified in pixar’s ‘renderman. there are no architects at facebook we get there through iteration. we don’t try to build an architecture that is failproof. building an architecture and worrying about it for months and months at a time before you actually go deploy it tends to not get us the result we want because by the time we’ve actually deployed something the problem has moved or there are more technologies available to solve different problems. we take it seriously enough to say “there are no architects on the team. the problem with economies of scale economies of scale quickly become economies of hassle from jessamyn, amplifying the exasperation people feel when daily activities are made more complex by poor application of technology. in the example given, the phone app reduces costs for the provider, but doesn’t improve the experience for the customer. people may not expect parking to be delightful, but that’s not an excuse for making it frustrating. wither hardware startups? [i]t’s getting harder to find independent hardware startups that can scale up to something big without getting bought. from dieter bohn on the collective disappointment so many people feel about the eero acquisition. the rise of product ecosystems is increasing the costs and risks for independent hardware startups in every category. (perhaps that’s why remarkable positions itself as the intentionally unconnected alternative to our phones.) turning off exposure preview on my fuji x-e nanda kusumadi has quite a number of tips for configuring a fuji x-e . those tips include using raw photo recording and turning on k video capture (they’re off by default), and one i hadn’t considered: enabling adobe rgb color space with its wider than srgb gamut. i prefer not to use some of other the suggestions, such as enabling electronic shutter (it reduces dynamic range). one setting not mentioned in nanda’s tips is turning off exposure preview. something from nothing: a dog park, a parade, and... on a lark, jaime kornick created patrick’s park. then she created a dog parade, then…. iheart mentioned the dog parade on the radio, local publications wrote about it, and the rsvps started rolling in. in total, more than people said they were coming. that’s when i realized i needed to get a permit. then she got a call: i told them the panel would consist of thought leaders within the canine community, bull shitting. market risks and opportunities for linux distro vendors ibm’s acquisition of red hat got me thinking about how the market for commercially supported linux distros is changing. ibm is trying to find a foothold in a maturing market dominated by aws while the market for enterprise data centers is shrinking. so, where is linux being used (or will be used), and what’s changing in those spaces? to be clear: this is about commercial linux distros, not upstack offerings like openstack, openshift, kubernetes, etc. kubesprawl this leads to the emerging pattern of “many clusters” rather than “one big shared” cluster. its not uncommon to see customers of google’s gke service have dozens of kubernetes clusters deployed for multiple teams. often each developer gets their own cluster. this kind of behavior leads to a shocking amount of kubesprawl. from paul czarkowski discussing the reasons and potential solutions for the growing number of kubernetes clusters. hard solutions to container security the vulnerability allows a malicious container to (with minimal user interaction) overwrite the host runc binary and thus gain root-level code execution on the host. from aleksa sarai explaining the latest linux container vulnerability. to me, the underlying message here is: containers are linux. from scott mccarty washing his hands of it. kata containers is an open source project and community working to build a standard implementation of lightweight virtual machines (vms) that feel and perform like containers, but provide the workload isolation and security advantages of vms. on asking the right questions long before digital cameras killed film, kodak and fuji were locked in a desperate battle for market share. film camera and mm film sales climbed steadily through most of the th century, and in , kodak dominated with % share of the film market, but then things started changing: kodak was said to have done a survey to determine whether its color films were what pro and amateur photographers really wanted. explore for inspiration, then test and focus cultivate exploration: as a leader, you want to encourage people to entertain “unreasonable ideas” and give them time to formulate their hypotheses. demanding data to confirm or kill a hypothesis too quickly can squash the intellectual play that is necessary for creativity. then ruthlessly prioritize for focus: [force] teams to focus narrowly on the most critical technical uncertainties and [rapidly experiment for] faster feedback. the philosophy is to learn what you have gotten wrong early and then move quickly in more-promising directions. government drinking game the department of agriculture [had] an annual budget of $ bn and was charged with so many missions critical to the society that the people who worked there played a drinking game called does the department of agriculture do it? someone would name a function of government, say, making sure that geese don’t gather at us airports, and fly into jet engines. someone else would have to guess whether the agriculture department did it. it just looks better that way in old english the past tense of “can” did not have an “l” in it, but “should” and “would” (as past tenses of “shall” and “will") did. the “l” was stuck into “could” in the th century on analogy with the other two. from arika okrent, in a mentalfloss piece about the weird history of some spellings. the piece has other examples of spelling changes to conform words to some aesthetic or another, even when those changes were inconsistent with the history and etymology of the word. on building a culture of candid debate a good blueprint for [building a culture of candid debate] can be found in general dwight d. eisenhower’s battle-plan briefing to top officers of the allied forces three weeks before the invasion of normandy. as recounted in eisenhower, a biography by geoffrey perret, the general started the meeting by saying, “i consider it the duty of anyone who sees a flaw in this plan not to hesitate to say so. subtitling videos there are plenty of people and companies offering human or automated speech-to-text services for video captioning, but embedding those captions in a video was a curiosity to me. bitfield ab’s isubtitle is a straightforward choice that does exactly what you expect and adds no complications. however, google drive doesn’t import captions embedded in videos, and instead you have to upload them separately. shuffle sharding in dropbox's storage infrastructure first, some terms and context: [we aggregate blocks] into gb logical storage containers called buckets. [buckets] are aggregated together and erasure coded for storage efficiency. we use the term volume to refer to one or more buckets replicated onto a set of physical storage nodes. osds [are] storage boxes full of disks that can store over a petabyte of data in a single machine, or over pb per rack. parts of a network you should know about if you’re running infrastructure and applications on aws then you will encounter all of these things. they’re not the only parts of a network setup but they are, in my experience, the most important ones. the start of graham lyons’ introduction to networking on aws, which (though the terms may change) is a pretty good primer for networking in any cloud environment. though cloud infrastructure providers have to deal with things at a different later, graham’s post covers the basics—vpcs, subnets, availability zones, routing tables, gateways, and security groups—that customers need to manage when assembling their applications. we're gonna need a bigger prng cycle length... the general lesson here is that, even for a high quality prng, you can’t assume a random distribution unless the generator’s cycle length is much larger than the number of random values you’re generating. a good general heuristic is — if you need to use n random values you need a prng with a cycle length of at least n². from a post by mike malone on prngs vs. on uber eats nobody knows your restaurant is a popup for independent or family-owned restaurants with less traffic, douglass points to the pop-up restaurant. not to be confused with popup restaurants, which are dining concepts open for a limited time. popups are cooking stations within the main kitchen of a restaurant dedicated to fulfilling delivery-only orders. eater recently profiled a dallas, tx-based chain called sushiyaa, which owns five physical locations but houses a couple dozen brands within them. the virtual brands are only available through uber eats. interconnected, machine readable data, at scale the nga provides a free database with no regulations on its use. maxmind takes some coordinates from that database and slaps ip addresses on them. then ip mapping sites, as well as phone carriers offering “find my phone” services, display those coordinates on maps as distinct and exact locations, ignoring the “accuracy radius” that is supposed to accompany them. “we assume the correctness of data, and often these people who are supposed to be competent make mistakes and those mistakes then are very detrimental to people’s daily lives,” said olivier. interfaces, surface area, durability a dos program can be made to run unmodified on pretty much any computer made since the s. a javascript app might break with tomorrow’s chrome update — joe groff (@jckarter) july , a dos program can be made to run unmodified on pretty much any computer made since the s. a javascript app might break with tomorrow’s chrome update from joe groff, who wonders if developers will choose old platforms running in emulators over more complex and volatile modern platforms. in praise of refactoring under the right conditions refactoring provides a sort of express lane to becoming a master developer. […] through refactoring, a developer can develop insights, skills, and techniques more quickly by addressing a well understood problem from a more experienced perspective. practice make perfect. if not the code, maybe the coder. from patrick goddi, who argues refactoring is about more than code quality. the day-to-day drudgery of state sponsored hacking after a review of bids and testing the capabilities of some of the exploits offered, the team decided to build its own malware. “this is the only inexpensive way to get to the iphone, except for the [israeli] solution for million and that’s only for whatsapp,” explained one team member in a message. “we still need viber, skype, gmail, and so on.” the same was true of the android and windows malware and the back-end tools used to manage the campaign. who controls the menu? when people are given a menu of choices, they rarely ask: “what’s not on the menu?” “why am i being given these options and not others?” “do i know the menu provider’s goals?” “is this menu empowering for my original need, or are the choices actually a distraction?” (e.g. an overwhelmingly array of toothpastes) from tristan harris, co-founder of the center for humane technology. it’s the first of ten magic tricks he pointed to that technology companies use to hijack users’ minds and emotions. apple cloudkit uses foundationdb record layer together, the record layer and foundationdb form the backbone of apple’s cloudkit. we wrote a paper describing how we built the record layer to run at massive scale and how cloudkit uses it. today, you can read the preprint to learn more. from an anonymous foundationdb blog post introducing relational database capabilities built atop foundationdb’s key-value store. the paper about cloudkit (pdf) is also worth a read. cloudkit is apple’s free at any legitimate scale back-end as a service for all ios and macos apps. you can identify a dog on the internet, but will you bother to? you can construct any [effing] narrative by scouring the internet for people claiming something. it doesn’t make it relevant. it doesn’t make it true. from agri ismaïl’s media criticism (start here). this isn’t an issue of not knowing the dogs on the internet, it’s a matter of not caring who’s a dog in the interest of either clicks or political interest. technology choices, belonging, and contempt i was taught to be contemptuous of the non-blessed narratives, and i was taught to pay for my continued access to the technical communities through perpetuating that contempt. i was taught to have an elevated sense of self-worth, driven by the elitism baked into the hacker ethos as i learned to program. by adopting the same patterns that other, more knowledgable people expressed i could feel more credible, more like a real part of the community, more like i belonged. rollback buttons and time machines adding a rollback button is not a neutral design choice. it affects the code that gets pushed. if developers incorrectly believe that their mistakes can be quickly reversed, they will tend to take more foolish risks. […] mounting a rollback button within easy reach […] means that it’s more likely to be pressed carelessly in an emergency. panic buttons are for when you’re panicking. from dan mckinley, speaking about the complications and near impossibility of rolling back a deployment. don't let requests linger in practice, we have fixed whole classes of reliability problems by forcing engineers to define deadlines in their service definitions. from ruslan nigmatullin and alexey ivanov on dropbox’s migration to grpc. also consider request replication. polarization vs. judgement in a polarized climate, opponents would jeer even eloquence from an unwelcome source; partisans would chant lovingly for public incontinence if delivered on behalf of the home team. from politico editor-in-chief john f. harris, talking about trump, but the point seems to apply far more broadly. shooting down star wars as a vehicle for exploring human relationships with future technologies into the ongoing fight between those who dismiss star wars as a shallow space opera vs. those who who would elevate the movies to a position of broader significance (so-called hard science fiction) strolls jeremy hsu, who points out: regardless of writer-director rian johnson’s intentions for “the last jedi,” his story transformed the adorable robotic sidekick into a murder droid with a will of its own. that would normally have huge implications in a science fiction story that wants to seriously explore a coherent and logical futuristic world setting. incident postmortems: customer communication incidents happen. the question is whether or not we’re learning from them. there are a bunch of postmortem resources collected here to help teams maximize the learning and service reliability improvements they can gain from an incident. however, there’s a separate question about how to communicate about incidents with customers. this definitely involves communications during the incident, but i’m especially interested in customer-facing communications after an incident. these seem to be the key questions customers need answers to: pid controllers are way cooler than the wikipedia article lets on the wikipedia entry on pid controllers is perfectly accurate, but it seems to bury the elegance of the technology and theory. meanwhile, the article on gyroscopic autopilot (both maritime and aeronautical) makes no mention of pid controllers, despite that being the field in which the theory of pid controllers was developed. pid controllers are all around us. they make elevators accelerate and decelerate without knocking passengers to the floor or pinning them to the ceiling, they stabilize video for pros and consumers alike, they make anti-lock brakes work, and nearly every other automated task in the software and physical world where the control needs to be adjusted based on observed conditions. wikipedia quotes: mathematical models of vagueness and ignorance [f]uzzy logic uses degrees of truth as a mathematical model of vagueness, while probability is a mathematical model of ignorance. from wikipedia on fuzzy logic. ipads as primary computers: never say never this twitter thread has some points worth considering for those interested in how our expectations and relationship with “business tools” changes over time: and, in case that tweet disappears, here’s the key text and the referenced gui review: i’m fascinated by the technical “class” obsession w/ ipads replacing laptops. this review of gui and mouse is what i think some of the review of the ipad will look like in years. common root causes of intra data center network incidents at facebook from to from a large scale study of data center network reliability by justin meza, tianyin xu, kaushik veeraraghavan, and onur mutlu, the categorized root causes of intra data center incidents at fabook from to : category fraction description maintenance % routine maintenance (for example, upgrading the software and firmware of network devices). hardware % failing devices (for example, faulty memory modules, processors, and ports). the entirely rational, yet surprising relationship between timecode broadcasts and sputnik many us folks just changed their clocks for daylight saving time, and here in california we’re voting on a proposition that might lead to changes in california’s time standards, so quite a number of people have time on their minds. meanwhile, on a national level, trump intends to defund one of the mechanisms we use to to synchronize time across the country. the national institute for standards and technology operates timecode radio stations. republics, power, and populism: their rise and fall mike duncan, writing in the washington post on the fall of the roman republic: some in the roman leadership could see clearly by the s and s b.c. that this socioeconomic dislocation was becoming an acute problem. they could see that, out in the countryside, families were losing their land, and in the cities, grain shortages were leading to panic and starvation. these poor families were certainly not sharing the benefits of rome’s imperial wealth and power. pour one out for the sears catalog, the original market disrupter whet moser pointed out this enlightening twitter thread that explains an aspect of sears i hadn’t considered before: by disrupting retail stores with mail-order, it was empowering a demographic that was often underserved in their communities: the sears catalog succeeded because it got the goods to people who couldn’t get to stores. one of those demographics? african-americans. in a lengthy twitter thread, cornell historian louis hyman writes that it freed up black southerners from going to general stores, which was often (at best) a humiliating experience. donut tours everywhere i’m a big enough fan of donuts that i’ve planned tours to explore and celebrate them: : the lowell donut tour : donut tour : this time it’s personal those tours focused on massachusetts, but it turns out that isn’t the only state with a strong donut heritage. the butler county visitors bureau promotes a donut trail, including map, passport, and faq. those who complete the passport can receive an exclusive donut trail t-shirt. how to date your foodstuffs whet moser, suddenly making sell-by dates on food products relevant to me: about a quarter of us methane emissions comes from food rotting in landfills. the dates on our packaged food products look so authoritative, but the way moser tells it, they were invented by marketing folks to increase sales at the cost of disposing of otherwise good products that have an expired sell-by date. fuji instax back for hasselblad isaac blankensmith writing in petapixel about building an instax instant film back for a hasselblad : instant photos are magical. they develop before your eyes. you can share them, gift them, spill water on them, draw on them. the only problem is that most instant cameras are pretty cheap — that’s why i’ve always wanted to hack my medium format camera to take instant photos with shallow depth of field and sharpness. can we train ourselves out of color blindness? which one of the boxes has an irregular color? a screenshot of the igame color vision test. i’m very color blind by traditional tests, but my score in this one has improved over time. am i learning the test, or…? psa reminder about takt time from wikipedia a common misconception is that takt time is related to the time it takes to actually make the product. in fact, takt time simply reflects the rate of production needed to match the demand. said again: it’s the required rate, not the actual rate. notes on observing the milky way notes from kevin palmer at dark site finder and matt quinn at petapixel. what is it? cc-by-nc-nd by bryce bradford kevin palmer: every star you can see with the unaided eye is located within the milky way. […] but when most people talk about “seeing the milky way”, they are talking about the core of the galaxy. located in the constellation sagittarius, this is the brightest part of the milky way. restaurants, hotels, mustaches, wages matthew taub, writing in atlas obscura around the same time, the first modern restaurants were rising around paris. these establishments, primarily for the wealthy, sought to recreate the experience of dining in an upscale home. the experience was about more than food. waiters had to retain the appearance of domestic valets, who were forbidden to wear mustaches as a sign of their rank. diners were “paying to humiliate people in an almost institutional way,” says historian gil mihaely, who has published extensively on the subject of french masculinity. a cold day in coaldale a cold day in the desert. coaldale, nevada music cc-by-nc-sa: dan warren, “the debate” at instagram. bad maps are ruining american broadband karl bode in the verge: in policy conversations, isp lobbyists lean heavily on the fcc’s flawed data to falsely suggest that american broadband is dirt cheap and ultra competitive, despite real-world evidence to the contrary. isps also use this false reality to imply meaningful consumer protections aren’t necessary because the market is healthy (as we saw during the fight over net neutrality). s and cloudfront configuration frustration it turns out that the interaction between s , cloudfront, and route can be bumpy when setting up buckets as cdn origins. it’s apparently expected that a cloudfront url will read data from the wrong bucket url and redirect browsers there for the first hour or more. the message from aws is “just wait,” which makes for a crappy experience. time synchronization is rough cloudflare on the frustrations of clock skew: it may surprise you to learn that, in practice, clients’ clocks are heavily skewed. a recent study of chrome users showed that a significant fraction of reported tls-certificate errors are caused by client-clock skew. during the period in which error reports were collected, . % of client-reported times were behind by more than hours. ( . % were ahead by more than hours.) this skew was a causal factor for at least . parents in vs. this thread from breanne boland, which starts with a screenshot of another tweet: your parents in : don’t trust anyone on the internet. your parents in : freedom eagle dot facebook says hillary invented aids. twin beech, beatty, nv just outside beatty nevada you’ll find a weathered sign promising the services of a long-closed brothel, and next to it, an aircraft covered in generations of tags. the plane, a twin beach, made an abrupt and final landing in the s as the unexpected end to a marketing stunt—or perhaps a dare—gone wrong. sit at the bar in town for a while and you’ll get a number of stories. windows was mb. today we have web pages heavier than that! the title is a quote from nikita prokopov, who is wallowing in disenchantment. claim chowder from : computational photography way back in i wrote: i’m sure somebody will eventually develop software to automatically blur the backgrounds of our smartphone photos, but until then, this is basic physics. the new camera system in the iphone xs seems to have moved computational photography from the world of parlor tricks to the mainstream. update this blog post from the developer of halide, a premium camera app for ios, goes into a lot more detail about all the computation going on in the new cameras. the color of copenhagen the color of #copenhagen is it yellow, brown, mustard? i love all the shades. at instagram. the real goldfinger: the london banker who broke the world goldfinger, the bond film, is based on a premise that is incredibly foreign to today’s audiences: moving gold between countries was illegal. oliver bullough in the guardian asks us all to think about that a bit more: the us government tried to defend the dollar/gold price, but every restriction it put on dollar movements just made it more profitable to keep your dollars in london, leading more money to leak offshore, and thus more pressure to build on the dollar/gold price. git foo a few git commands i find myself having to look up: resolve git merge conflicts in favor of their changes during a pull: git pull -xtheirs git checkout --theirs the/conflicted.file source viewing unpushed git commits git log origin/master..head you can also view the diff using the same syntax: git diff origin/master..head or, “for a little extra awesomeness” git log --stat origin/master..head updated since it was first posted: starting with git . things that make us dumber: air pollution, full bladders air pollution is making us dumber, study shows: the team found that both verbal and math scores “decreased with increasing cumulative air pollution exposure,” with the decline in verbal scores being particularly pronounced among older, less educated men. study links urge to pee with impairment: snyder and his team ran the study on eight individuals, who each drank milliliters of water every minutes until they reached their “breaking point,” where they could no longer hold their urine. maintenance and renewal abby sewell, with photographs by jeff heimsath, in the national geographic: every spring, communities gather to take part in a ceremony of renewal. working together from each side of the river, the villagers run a massive cord of rope, more than a hundred feet long and thick as a person’s thigh, across the old bridge. soon, the worn structure will be cut loose and tumble into the gorge below. over three days of work, prayer, and celebration, a new bridge will be woven in its place. hash rings, sharding, request replication balancing data and activity between shards your consistent hash ring leads to inconsistent performance: the basic consistent hashing algorithm presents some challenges. first, the random position assignment of each node on the ring leads to non-uniform data and load distribution. second, the basic algorithm is oblivious to the heterogeneity in the performance of nodes. from https://www.cs.cornell.edu/projects/ladis /papers/lakshman-ladis .pdf, which explains that cassandra addresses that common problem by “analyz[ing] load information on the ring and have lightly loaded nodes move on the ring to alleviate heavily loaded nodes. steven dean mcclellan, bombay beach steven dean mcclellan, bombay beach at instagram. improving automated fault injection automated failure analysis is hard, manual failure analysis requires great expertise. why this painting of dogs playing poker has endured for over years jackson arn in artsy: the “dogs playing poker” paintings, by cassius marcellus coolidge, belong to that pantheon of artworks—michelangelo’s david, da vinci’s mona lisa, botticelli’s the birth of venus, van gogh’s starry night, hopper’s nighthawks— that are immediately recognizable to people of all ages and backgrounds, including those who don’t readily admit to enjoying art. so how, pray tell, did a pack of dogs playing poker outlast so many other “serious” paintings? willie in christiana willie has lived in christiana since it was founded in at instagram. product managers, project managers, delivery managers, and engineering managers, according to quora i’m trying to write some job descriptions, so of course i found myself in quora. what is the difference between program manager and delivery manager? as delivery manager, we ensure the projects are delivered on time and on budget. we are a slightly higher level work that project managers in the sense that we try not to escalate issues as much as resolving them and letting upper management know of relevant issues. twin beech, beatty this beautiful old twin beech lies wrecked and abandoned near beatty, nv. locals tell stories of how the plane was used to shuttle guests from las vegas to the town’s brothel in the s, but things went wrong with a publicity stunt, or perhaps a dare, and the plane made its final landing here. at instagram. love locks, copenhagen love locks in copenhagen toldbodgade bridge over nyhavn inlet. at instagram. campanology, noun the cambridge dictionary tells us that “campanology” means “the art or skill of ringing church bells.” it doesn’t give us a collective noun, however, but i’m sure this is it: a group of bell ringers? that’s a “pubfull” with more at pinterest. bar velo, brooklyn bar velo, brooklyn #mediumformat #fujigw iii at instagram. transamerica pyramid, from columbus avenue #ispytransamericapyramid from the center of columbus avenue at broadway at instagram. tantallon castle, scotland tantallon castle, scotland at instagram. vxlan routing recommendations from cumulous networks vxlan routing recommendations from cumulous networks, which offers switch software (but not client software). https://cumulusnetworks.com/blog/vxlan-designs-part- / vxlan routing is the process in which a vtep receives a vxlan packet destined to itself, removes the vxlan header and then performs a layer route lookup on the inner decapsulated packet. since the vtep has to perform two sets of lookups, first on the encapsulated vxlan traffic then on the decapsulated inner packet, it requires special hardware asic to perform both lookups in a single pass all in hardware. flight of the bumblebee flight of the bumblebee music: jazzy ashes, cc by-nc-sa the underscore orkestra at instagram. steven dean mcclellan, bombay beach steven dean mcclellan, bombay beach at instagram. birdsong birdsong, mural by joshua coffy at instagram. flickr get photo page from image name let’s say you have an old-style flickr photo url like the following: http://www.flickr.com/photos/ _ c f .jpg now let’s say you want to find the page on flickr for that photo? put the photo id in a url like this: https://www.flickr.com/photo.gne?id= poulsen welding shop, susanville, ca poulsen welding shop, susanville, ca growing up, i remember welding and fabrication shops being common. not so much anymore. there are just over , self-employed welders in the us today, according to the bureau of labor statistics, but getting historical data from them is approximately impossible. looking for more, i found assembling magazine’s retrospective on how welding has changed in the past half century or so: new processes, such as electron beam welding, friction welding, plasma arc welding, friction stir welding, explosion welding and laser beam welding, have increased the range of materials and components that can be welded. object storage prior art and lit review this list is not exhaustive. instead, it is a selection of object storage implementations and details that appear interesting. some themes that it many or all of these comparators struggled with include: new systems to meet scaling needs facebook, google, and yahoo are all very open about having reinvented their object storage solutions to address evolving needs (typically cost and availability) as they scaled. those players dramatically reinvented their systems without strong regard for backwards compatibility, but evidence suggests s has gone through similarly dramatic changes as well, but without breaking api compatibility. naming things is hard. naming people is harder. michael sherrod and matthew rayback scoured american census records searching for atrocious baby names. the results are compiled in an amusing little book called bad baby names: the worst true names parents saddled their kids with—and you can too!. among the names they discovered were “toilet queen,” “leper,” “cholera,” “typhus,” “stud duck,” “loser,” “fat meat,” “meat bloodsaw,” “cash whoredom,”“headless,” “dracula,” “lust,” “sloth,” “freak skull,” “sexy chambers,” “tiny hooker,” “giant pervis,” “acne fountain,” “legend belch,” and “ghoul nipple. yongma land just a creepy fiberglass clown head at an abandoned amusement park outside seoul at instagram. stereotypical photo of the brooklyn bridge gray skies at the #brooklynbridge at instagram. yarn bombed, san francisco city hall yarn-bombed trees outside san francisco city hall at instagram. observing an abandoned building and open landscape, coaldale, nevada open floor plan, coaldale junction, nevada music: cc-by-nc-sa dan warren at instagram. feature flags gone wrong btw – if there is an sec filing about your deployment, something may have gone terribly wrong. from doug seven explaining how, in , that’s exactly what happened. rain, san francisco much-needed rain soaks the tables at san francisco’s ferry building. at instagram. spencer wynn: hello project spencer wynn’s hello project is everything i need right now. johnathan little i first met johnathan little on us route , about miles due north of pahrump, nv. he’d been walking since he left oklahoma one day a while back. #vanlife gets a lot of love on instagram, but johnathan joined the #walkinglife to regain his self-respect and lose some weight, and he seems on a path to do both. this photo was from the second time i met him, on my way back from beatty, nv. the kpa soldier guarding the door to north korea the door behind this kpa soldier exits to north korea. in addition to needing a stolid face, kpa soldiers must be expert martial artists, according to wikipedia. at instagram. no groceries, mina, nevada “grocery, sundries, ice cream” in mina, nevada at instagram. the paradox of tolerance less well known is the paradox of tolerance: unlimited tolerance must lead to the disappearance of tolerance. if we extend unlimited tolerance even to those who are intolerant, if we are not prepared to defend a tolerant society against the onslaught of the intolerant, then the tolerant will be destroyed, and tolerance with them. — in this formulation, i do not imply, for instance, that we should always suppress the utterance of intolerant philosophies; as long as we can counter them by rational argument and keep them in check by public opinion, suppression would certainly be unwise. aws regions, azs, and vpcs, nics, ips, and performance jump to section: availability zones and regions vpcs elastic ips and elastic network interfaces network performance resources by scope connectivity by scope availability zones and regions aws’ primary cloud is available in regions, each with two to six availability zones, not including separately operated regions (with independent identity) for govcloud and china. most aws services operate independently in each region (though identity is shared across regions in the primary cloud), and each service has its own (often region-specific) endpoint (many libraries and the aws cli simply insert the region name in the endpoint url). claim chowder: cloud storage ten years ago apple was still doing macworld expo keynotes, and that year they introduced time capsule. my response was this: forget time capsule, i want a space ship: so here’s my real question: why hasn’t apple figured out how to offer me a storage solution that puts frequently used items on local disk, and less-frequently used items on a network disk? seamlessly. ten years later: cloud storage is definitely the norm. dalhousie castle sunrise to sunset at @dalhousiecastle music: “the moments of our mornings” cc-by-nc kai engel at instagram. the make us proud and yld offices just another awesome day at the make us proud and yld offices (find them on twitter). @tomholloway is the star of this one, but you’ll see some others on the team working on a project for @joyent. shot with an @alpinelabs radian music is cc-by-nc-sa dexter britain at instagram. good enough, satisficing, and meeting market demand nanda kusumadi: companies tend to over-serve customers in their products to the point that the surplus of performance metrics cannot be consumed. this leads to waste in r&d, build and operational resources, basically a waste of human capital. over-serving products have been optimised well beyond what a user can consume. atomic cafe neon the famous neon sign at @atomicliquors, #lasvegas’ oldest bar, where s patrons used to enjoy views of nuclear tests from the roof. i had the joy of meeting the former owner, joe sobchik, on a visit in . i stopped by around am (yes, i make a habit of visiting bars early in the morning) and found the owner, joe sobchik, sipping a coffee at the bar. he was a man full of stories, i could tell, but i was foolishly unprepared. aws' andy troutman on component reusability what we do first is we build very simple foundational building block services … we will build the simplest possible service that you could think of. the next thing we do is we encourage an open marketplace within amazon so individual teams can use, optimize, and extend these basic services. we use our individual [teams] as a test lab to experiment on better ways to do things, and when we find something that seems to be working, we look for ways to [grow it and use it more] broadly. drivers and “standards” for both network and block storage, aws is doing significant work to develop and maintain drivers in a variety of guest oss. some of this work improves performance for guest oss running in any modern hardware virtualized environment, but not everything is directly portable. this discussion about adding ena support for netmap is one example. otoh, amazon seems to be sponsoring driver development (see freebsd) when they’re not doing it themselves (see linux). hardware virtualization has moved to hardware one of my takeaways from aws’ bare metal announcements at re:invent this week is that the compute, storage, and network aspects of hardware virtualization are now optimized and accelerated in hardware. aws has moved beyond the limitations that constrained vm performance, and the work they’ve done applies both to their bare metal hardware and their latest vm instance types. notes from "life of a code change to a tier service (dev )" at aws re:invent andy troutman’s talk is useful in explaining complex deployment workflows to management types. camera advice: a film camera for a novice a friend of mine sent me a question about a good film camera to get started with: my partner has been thinking for some time about her first camera and she likes the idea of film photography. her birthday is coming up and i’m thinking of buying a camera as a surprise gift to bring on an upcoming backpacking trip. it’s just a thought. we don’t buy each other a lot of stuff because we’re big on experiences, and we save our money so we can travel to see each other. dave wascha's years of product management advice in minutes dave wascha (li) speaking at mind the product in san francisco on advice he wished he had as a younger product manager: link to video. you should watch the video, but here’s the short version: listen to your customers: focus on deeply understanding your customers’ problems. don’t listen to your customers: it’s up to product managers to figure out solutions to those problems, not customers. my addition: they’d ask for faster horses. vcrs that rewind faster a story, possibly apocryphal (i.e. i can no longer find the source), tells of electronics manufacturers asking customers what features they wanted in their home video equipment. “vcrs that rewind faster,” they cried. instead they got dvds that didn’t need rewinding. i was remembering that story and went looking to source it and all i could find was my blog post from a decade ago. of course once we got dvds, we then needed to solve the frustrations of the video rental store. continuous disruption trains were once seen as icons of freedom. they freed riders from the dust and bumps of horse or stagecoach travel, and dramatically shortened travel times. but that view of trains as agents of freedom changed with the development of the automobile—and the way it shifted control of routes and schedules from the railroad to the driver. this isn’t about transportation policy , it’s about how previously novel solutions become subject to disruption once they become the baseline against which alternatives are compared. mortmar, california carniceria, liquor, grocery this was once north shore, california, but many maps now label it mortmar. at instagram. gender stereotypes, toys, and the sears catalog elizabeth sweet, writing in the new york times, way back in on her research into the role of gender stereotypes in the marketing of toys: during my research into the role of gender in sears catalog toy advertisements over the th century, i found that in , very few toys were explicitly marketed according to gender, and nearly percent showed no markings of gender whatsoever. in the s, toy ads often defied gender stereotypes by showing girls building and playing airplane captain, and boys cooking in the kitchen. lawrence lessig: republic, lost lawrence lessig in a talk at google in speaking on the topic of his book, republic, lost. his talk concludes: this nation faces critical problems requiring serious attention, but we don’t have institutions capable of giving them this attention. they are distracted, unable to focus. and who is to blame for that? who is responsible? i think it’s too easy to point to the blagojeviches and hold them responsible, to point to the looking up at muir woods end of summer at #muirwoods with a @lomography #spinner at instagram. extraterrestrial highway, nevada the extraterrestrial highway, just north of area at instagram. ranch hand at auction a ranch hand stands ready to call a bidder in the cowboy auction at the @californiamidstatefair. though they’re traditionally agricultural events, fairs were typically founded by local businesses leaders seeking to grow commerce. basically, they were the tech events of their time. at instagram. street jazz new orleans-style jazz on the embarcadero near fisherman’s wharf, shot on #kodak #ektar with a #hasselblad #hasselblad elm at instagram. mendocino sunset sunset on the mendocino coast outside at @heritagehouseresort at instagram. no more border walls, please america’s greatest legacy is found in the freedoms we uphold for all, not the prohibitions we levy on others. fences, walls, and travel bans are contrary to that legacy. #usmexicoborder #borderfence, #calexico at instagram. hearst castle tour hearst castle in mm music: cc-by-nc charmed life by adam selzer at instagram. contrails above sutro tower #parallel #contrails above #sutrotower, from #twinpeaks, #sanfrancisco at instagram. user stories are documentation while writing up the draft docs for joyent’s container name service i leaned heavily on the user stories and use-cases for the feature. it has me realizing that we should consider user stories to be the first draft of the user documentation. indeed, consider that well-written docs and user stories have similar qualities: a user, goal, and benefit, in clear language that’s accessible in small, focused chunks. the cns docs are now in our core documentation library, and i’m happy that we’ve updated the content management system to support deep linking to individual headings, like this one about adding cns service tags when creating an instance with the triton cli. everybody smiles while rolling down the hill... everybody smiles while rolling down the hill at the bring your own big wheel event! at instagram. ancient aztec chemistry a - blend of morning glory juice and latex created rubber with maximum bounciness, while a - mix of latex and morning glory made the most durable material. it seems they were making bouncy balls for fun and sport. but, to be clear about the ingredients: morning glory plants tend to grow near rubber trees, and both plants were considered sacred in several mesoamerican cultures. morning glory, for example, was also used in religious ceremonies for its hallucinogenic properties. no gas at mina, nevada mina, nevada at instagram. echoes of product management advice in declarative vs. imperative programming the following line in a post about the difference between declarative vs. imperative programming caught my attention for the way it echoes product management best practices: [i]t’s often good not to think of how you want to accomplish a result, but instead what the component should look like in it’s new state. of course it does matter how you get to where you’re going, but it’s a whole lot easier if you first focus on aligning everybody on goals and where you’re going. the hotel huntington and sf skyline the hotel huntington (now @thescarlet_sf) atop #californiastreet, #sanfrancisco music: “faster does it” by kevin macleod (cc-by) at instagram. mcway falls #mcwayfalls in #bigsur music: “tomie’s bubbles” by candlegravity (cc-by-nc-sa) at instagram. sutro tower #sutrotower, #sanfrancisco music: “feeling dark (behind the mask)” by oop d (cc-by-nc) at instagram. tree, paso robles #lonely #tree in a #field in #pasorobles #california music: “silence await” by idk (cc-by) at instagram. at the little a’le’inn, rachel... at the little a’le’inn, rachel nevada. film, light leaks, bikers, and aliens. at instagram. following a winding road #summer on a #windingroad in #cambria #california music: shady grove by shake that little foot (cc-by-nc-sa) at instagram. the top of the mark #sunset at #topofthemark, #sf video: https://www.instagram.com/p/bgrsnl hejs/ at instagram. hotel huntington sign at sunset the hotel huntington (now @thescarlet_sf) atop #californiastreet, #sanfrancisco at instagram. winding road, cambria #summer on a #windingroad in #cambriacalifornia at instagram. will luo will luo at @tempestbarsf at instagram. get list of functions in bash script…look for those in argv # get function list as array funcs=($(declare -f -p | cut -d " " -f )) # parse out functions and non-functions i= declare -a cmdargs declare -a otherargs for var in "$@"; do if [[ " ${funcs[@]} " =~ " ${var} " ]]; then cmdargs[i]=${var} else otherargs[i]=${var} fi ((i++)) done echo ${cmdarg[*]} echo ${otherargs[*]} on disfluencies your speech is packed with misunderstood, unconscious messages, by julie sedivy: since disfluencies show that a speaker is thinking carefully about what she is about to say, they provide useful information to listeners, cueing them to focus attention on upcoming content that’s likely to be meaty. […] experiments with ums or uhs spliced in or out of speech show that when words are preceded by disfluencies, listeners recognize them faster and remember them more accurately. san francisco’s mark hopkins hotel san francisco’s #markhopkins #hotel at the top of #californiastreet, on # mm #kodakfilm. #sanfrancisco #sanfranciscoca #sfca #sf #olympusstylus #kodakgold #kodakultra #kodakultragold #analog #film at instagram. compact camera recommendations a friend asked the internet: can anyone recommend a mirrorless camera? i have some travel coming up and i’m hesitant to lug my dslr around. of course i had an opinion: i go back and forth on this question myself. my current travel camera is a sony rx mark (the mark was recently released). some of my photos with that camera are on flickr. if i decide to get a replacement for my for my bigger cameras, i’ll probably go with a full frame sony a of some sort. bring your own big wheel brings smiles only a fool would try covering the #bringyourownbigwheel action on #film. i’m that fool. #sf #sanfrancisco #bigwheel #byobw #hasselblad #ektar #kodakfilm #film at instagram. zach houston’s poem store #zachhouston used to be a #mission regular, peddling his #poetry from a #poemstore made up of an old #mechanical #typewriter and carefully selected scrap papers. #sf #sanfrancisco #themission #valenciastreet at instagram. rewrite git repo urls a question in a mail list i’m on introduced me to a git feature that was very new to me: it’s possible to have git rewrite the repository urls to always use https or git+ssh, etc. this one-liner seems to force https: git config --global url.https://github.com/.insteadof git://github.com/ or you can add these to your .gitconfig: # use https instead of git and git+ssh [url "https://github.com/"] insteadof = git://github.com/ [url "https://github.com/"] insteadof = git@github. the tools on the jeremiah o’brien... the tools on the jeremiah o’brien are built to work on steam cylinders larger than oil drums. they’re mounted to the wall like trophies. on flickr: https://www.flickr.com/photos/maisonbisson/ / #bw #tools #jeremiahobrien #libertyship #sf #sanfrancisco at instagram. docker stories from new relic from new relic’s august blog post: [w]e didn’t try to create a full paas framework all at once. though this may be our eventual goal, it wouldn’t have solved the immediate deployment problem. we did not begin dockerizing our applications by starting with those that have the highest data volume. rather, we started with our simplest internal web apps, particularly stateless things that could scale horizontally. our early testing showed that high throughput apps are not a good choice for your first docker deployment, due to the docker network stack. sinistrality vs. dextrality in design photo cc-by-sa gerry dincher this post on why people focus on the right-hand side of a design is an old one, but still valuable today: these days there is a lot of talk about emotional design and how to properly create a connection between users and our products. focusing on the right-hand side of our designs can create these connections. we have the ability to influence and change a user’s belief in what is right and honest with our designs. hasselblad dating hasselblad historical and blue moon camera both offer this table to translate hasselblad serial numbers to year of manufacture: v = h = p = i = c = t = u = r = e = s = that should work for both the body and film magazines, though there are some exceptions noted in the comments at blue moon camera: how jackie chan wins tony zhou’ video is genius, as are the nine principles of action comedy he’s identified: start with a disadvantage use the environment be clear in your shots action & reaction in the same frame do as many takes as necessary let the audience feel the rhythm in editing, two good hits = one great hit pain is humanizing earn your finish read the full video description for more, and consider donating to support his work. photo hipster: playing with cameras after playing with fuji instax and polaroid (with the impossible project film) cameras, i realized i had to do something with kodak. my grandfather worked for kodak for years, and i have many memories of the stories he shared of that work. he retired in the late s, just as the final seeds of kodak’s coming downfall were being sown, but well before anybody could see them for what they were. backbone.js and wordpress the three are from , so details may have changed, but they seemed useful enough that i’ve had them open in my browser for a while: http://kadamwhite.github.io/talks/ /backbone-wordpress http://code.tutsplus.com/tutorials/using-backbone-within-the-wordpress-admin-the-back-end–wp- http://code.tutsplus.com/articles/using-backbone-within-the-wordpress-admin-the-front-end–wp- parable of the polygons is the future of journalism okay, so i’m probably both taking that too far and ignoring the fact that interactive media have been a reality for a long time. so let me say what i really mean: media organizations that aren’t planning out how to tell stories with games and simulators will miss out. here’s my example: vi hart and nicky case’s parable of the polygons shows us how bias, even small bias, can affect diversity. unit test wordpress plugins like a ninja (in progress) cc-by zach dischner unit testing a plugin can be easy, but if the plugin needs dashboard configuration or has dependencies on other plugins, it can quickly go off the tracks. and if you haven’t setup travis integration, you’re missing out. activate travis ci to start with, go sign in to travis now and activate your repos for testing. if you’re not already using github to host the plugin, please start there. unit testing wordpress plugins we’ve been unit testing some of our plugins using the old wordpress-tests framework and tips from this blog post. the good news is that the framework has since been incorporated into core wp, the bad news is that it was changed along the way, and it wasn’t exactly easy to get the test environment setup correctly for the old wordpress-tests. i’ve had a feeling there must be a better way, and today i discovered there is. deliverables, iteration, and constraints when asked to give a timeline for project delivery, my first questions, of course, are about the details of the project. then, i take a guess about the timeline and double it, and fight like hell to eliminate blockers and distractions for the team, work with them on implementation theories, ask leading questions that help balance the “optimum” solution against the timeline, and put up whatever obstacles i can to any changes to the plan. ruins of roebling’s works from flux machine: a tumbler of kevin weir’s creepy gifs. the original is from the library of congress. if the name “roebling” sounds familiar, it’s because this is the company, founded by john a. roebling, that built the brooklyn bridge and setup a good business making cables, or wire rope. the roebling brothers suspected the fire was german sabotage. given the activities of the german ambassador at the time, the claim has a whiff of plausibility. google’s link policies raise hell for simple bloggers i get a bunch of emails like this: we have recently received a notification from google stating that our website has unnatural links pointing towards it. this has affected our rankings on google and as a result, we’re trying to clear things up. our website url is www.builddirect.com. we noticed the following links are pointing to our website from your site: http://becomingdonnareed.com/ http://becomingdonnareed.com/blog/ /season- -episode- -style-note/ http://becomingdonnareed.com/blog/author/sandee/ http://becomingdonnareed.com/blog/category/style/ http://becomingdonnareed.com/blog/tag/crate-and-barrel/ http://becomingdonnareed.com/blog/tag/ikea/ http://becomingdonnareed.com/blog/tag/lumens/ http://becomingdonnareed. a/b split testing calculators mixpanel’s a/b testing calculator is a competent performer and valuable tool: thumbtack’s split testing calculator, however, is a surprise standout: that their code is in github is especially delightful. algolia search the multi-category autocomplete and autocomplete on filtering operators demos are interesting: mastery sarah lewis on mastery: mastery is in the reaching, not the arriving. it’s in constantly wanting to close that gap between where you are and where you want to be rebuild iphoto library yeah, iphoto is just about dead, and i’m probably a little crazy to still be using it at all, but i do and now i need to rebuild the library. the knowledgebase article can be summed to this: hold down the command and option keys while opening iphoto. you can’t just click the icon in the dock, you’ve got to double-click the icon in a real finder window (or some other context that doesn’t trap the keys like the dock does). x-ray scanners vs. film i’ve been enjoying my fuji instax , but i’m preparing for an upcoming trip and just remembered the challenge of flying with real film. cc-by-nc-sa vegard hagen. the flickr fuji instax room has a couple discussions on the topic, but the answers are inconclusive and unsupported by references. some people shared personal experiences suggesting there was nothing to worry about: studioesper: “never had any problems. i use to work by airports and go thru carry on xray just about everyday with a instax wide. porn consumption by geography and type this is shamefully old news, but pornhub released stats that correlate viewing preferences by geography and pulled out a quote too juicy to ignore: dixie loves dicks so much that the percentage of gay viewers for every single state in the south is higher than the average of the legal gay marriage states. i’m concerned that some of the numbers are contradicted in three different places in the same article, but it suits my worldview, so why bother questioning it? followup: triggertrap latency and fuji instax tips short answer: triggertrap app audio triggering latency is too long to capture a fast moving event. the app, the dongle, my trusty eos rebel xti, lensbaby (manual focus, soft edge details), and neewer flash worked, but too slowly. the phone was just inches from where i was throwing the dice, but the flash and camera were triggered after most of the action happened. most of the time the die flew off the table before the picture was captured. air-gap flashes for fun, and more fun this blog post by maurice ribble explains the problem with xenon flash tubes such as those typically used in photography: [x]enon flash tubes have a minimum duration of / , th of a second. that’s fast enough for most things, but not for a shooting bullet [that] travels around feet/second. in / , th of a second that bullet can travel about / rd of an inch leading to blurry photographs of bullets. what’s the minimum latency when using triggertrap audio triggering? cc-by-nc-nd by airguy the core point of triggertrap is to release the camera shutter faster and more reliably than can be done by hand, so this is a bit concerning: the explosion was so fast, that the triggertrap and camera just weren’t fast enough to capture it. so…what is the minimum latency between trigger noise and shutter signal when using the various triggertrap devices? it turns out they’ve gotten a lot of questions, and perhaps no small number of complaints about this issue with their mobile app. fuji instax tips and tricks cc-by-nc-sa by mychkine. on focusing and using the closeup attachment lens: if you want to take portraits, use [the included closeup adapter]. with the camera focus set to infinity, the point of sharp focus becomes meter. with the same [closeup] attachment the . - m focus setting gives pin sharp results at cm. (selfie range) the depth of field is quite shallow so it is easy to end up with blurred pictures if you mis judge the distance. yeah, he’s probably right apparently nate silver’s book on people being wrong is filled with errors: the text and chart are contradictory, and other errors in the comments. ncar’s computers are water cooled, not fanned with oxygen. meet the new media on the future of media, at the awl: of course a website’s fortunes can change overnight. that these fortunes are tied to the whims of a very small group of very large companies, whose interests are only somewhat aligned with those of publishers, however, is sort of new. the publishing opportunity may be bigger today than it’s ever been but the publisher’s role is less glamorous: when did the best sites on the internet, giant and small alike, become anonymous subcontractors to tech companies that operate on entirely different scales? the cameras i’ve enjoyed big huge labs reminded me that my flickr birthday is in just a few days. my first photo upload was on may , . flickr itself turned in february, but it was the big huge labs stat and the photo walks today that really got me thinking about how long it’s been. for whatever reason, that has me thinking about the cameras i’ve used over those years. ten years is long enough that i had to go looking to remember some, and long enough that i found some i’d forgotten. disclaimer in spam message you are receiving this e-mail because we just received a mass e-mail and the sender forgot to blind cc your addresses. we will only be sending this one e-mail so as to not pester you, so please contact us if you would like more information. people pay for photos like this first there was the bad engagement photos tumblr, but now it’s been one-upped by this crazy russian wedding photos livejournal. strobist david hobby on hdr i’ve been re-reading david hobby‘s lighting tutorial while at the same time exploring hdr (wikipedia’s hdr article is a good read for those unfamiliar with it). the question that eventually came to mind was how the guy that wrote the following feels about hdr? how often have you heard this, usually with a tone of superiority: “i am a purist, i only shoot available light.” (translation: i am scared shitless of flash. what makes us special? in daily kos this weekend: a common thread among young-earth creationists, gun enthusiasts, marriage exclusivists, and the %. the key point is that groups identify by what makes them “feel special.” distilled, here are the four groups: creationists: being created by god makes humans special gun enthusiasts: their role in protecting liberty makes them special marriage exclusivists: making marriage exclusive to straight people makes them special one percenters: their accumulated wealth makes them special i was interested in seeing the author’s evaluation of what may be a motivation for (some) members of the identified groups. on “do what you love” a friend forwarded miya tokumitsu’s essay “in the name of love” pointing out the steve jobs quote and summarizing that it “challenges the notion of work at what you love.” i read it with some frustration, then decided i had to ask my friend what he saw in it. i was already into my reply when i tried to look up other works by the author and discovered the piece has been positively covered by a lot of sites i respect. magic lantern for eos m the eos m is named as a “beta” supported camera, but you won’t find a download for it in the normal place. instead, you’ll have to use a “tragic lantern” build at tl.bot-fly.com. this forum thread is about the development, while this forum thread includes more how-to and documentation. canon eos m running magic lantern. from magiclantern.fm rumors subcomandante marcos, by jose villa, from wikipedia it started at the coffee shop. somebody pointed and made the claim, then everybody was laughing. “he looks just like him!” one said. “how would you know, he wore a mask!” exclaimed another. i looked him up. i could be accused of being a less interesting figure. how to identify context inside the wordpress dashboard on wp-hackers, haluk karamete asked: on admin pages, how can i detect that the current admin is dealing with a cpt? andrew nacin answered: get_current_screen()->post_type. [but] this will also specify a post type when it’s a taxonomy being edited. to filter that out, ensure that get_current_screen()->base == 'post', which is [true] for edit.php, post-new.php, and post.php (for all post types). haluk didn’t elaborate on the cause of the question, but the answer is very good advice for those seeking to conditionally enqueue js and styles only for specific post types. mysql performance tips from around the web gospel: use innodb, never myisam it seems everybody on stackexchange is singing from the same gospel: “[how can i] prevent queries from waiting for table level lock?” answer: use innodb. the major advantages of innodb over myisam. “even in a read-intesive system, just one delete or update statement will quickly nullify whatever benefits myisam has.” the main differences between innodb and myisam, including cache sizing recommendations. “how do you tune mysql for a heavy innodb workload? transcend wifi sd card hacking links http://www.fernjager.net/post- /sdcard: as a mhz linux system with mb of ram, using only ~ ma @ . v, the possibilities are endless! http://haxit.blogspot.com/ / /hacking-transcend-wifi-sd-cards.html: this post is written with the intention of exposing not only the exploits which will allow you to root (or jailbreak) the device, but also the process of discovering and exploiting bugs, some of which are a dead end, while others lead to the holy root b-) ads-b: the internet of things in the sky ads-b is a civil aircraft tracking and telemetry standard that the faa has ruled will replace transponders by . like a transponder, it’s used to identify air traffic, but with far more more information, such as altitude, heading, speed, and gps location. the protocol also supports delivery of weather, terrain, and notices to aircraft. the ads-b signals from aircraft in the sky are intended for receipt by both air traffic controllers on the ground and by other aircraft in the vicinity. need two-way encryption without mcrypt? in a typical lamp environment, but don’t have or can’t trust that mcrypt is available in php? try mysql’s aes_encrypt and aes_decrypt. go read the docs. where to buy a submarine no need to explain why, i understand: you need a submarine. and you don’t need a bathtub toy (really?), you need something that will truly wow them at the yacht club. there are a few soviet diesel subs built in the s through s that might be just the thing. photo: public domain, from wikipedia. source. the soviets built over whiskey-class subs, and quite a few of them are on the market now. manhattan project tours the manhattan project was among the us government’s’ first big secrets. it’s easy to forget that plutonium, the incredibly radioactive element at the core of the first atomic detonation, was only identified in . two years later army corps of engineers started construction of reactor b to produce it in industrial quantities. today, reactor b is a national historic landmark, and one of only a few locations of the sprawling manhattan project that the public can tour. where on earth can i get an weotype list? it’s not like these aren’t documented, but i keep forgetting where. woeid place types: $woetype = array( ' ' => 'town', ' ' => 'state-province', ' ' => 'county-parish', ' ' => 'district-ward', ' ' => 'postcode', ' ' => 'country', ' ' => 'region', ' ' => 'neighborhood-suburb', ' ' => 'colloquial', ' ' => 'continent', ' ' => 'timezone', ); they can be queried via yql: <?xml version=" . " encoding="utf- "?> <placetypes xmlns="http://where.yahooapis.com/v /schema.rng" xmlns:yahoo="http://www.yahooapis.com/v /base.rng" yahoo:start=" " yahoo:count=" " yahoo:total=" "> <placetype yahoo:uri="http://where.yahooapis.com/v /placetype/ " xml:lang="en-us"> <placetypename code=" ">historical town</placetypename> <placetypedescription>a historical populated settlement that is no longer known by its original name</placetypedescription> </placetype> </placetypes> when not to use esc_js() from the codex for esc_js: if you’re not working with inline js in html event handler attributes, a more suitable function to use is json_encode, which is built-in to php. dynamic range vs. price and brand dynamic range is what keeps skies blue while also capturing detail in the foreground. without enough dynamic range, we’re forced to choose between a blue sky and dark foreground, or properly exposed foreground and white sky. i’ve been using multiple exposure hdr techniques to increase the dynamic range i can capture, but multiple exposures don’t work well with moving subjects. a camera that can capture good dynamic range in one shot would be better than one that requires multiple shots to do the same. happy d. b. cooper day! the fbi’s wanted poster for d.b. cooper. d. b. cooper, the guy who hijacked a plane in and then — mid-flight — jumped into the darkness with a bundle of cash and disappeared, is celebrated on this day, the saturday following thanksgiving. granted, this is mostly just a thing in ariel washington, where it’s said to have started in , but the participants are pretty passionate about it. a smaller microcontroller for smaller jobs i’ve been thinking a bit about how overkill a full arduino is for shutterfingers, and feeling a bit sheepish about how lazy i am about learning to use some other microcontroller. then i found this guide talking about the attiny : if you’re just blinking a few leds, and reading a single sensor, you can get the job done smaller and cheaper using a simple ic, like the attiny . using it requires a programmer socket and actually mounting the ic to a pcb, but it seems to have enough going on to be useful: if i did it over again, i’d make shutterfingers smaller shutterfingers is my simple servo controller that presses the shutter on cameras that don’t support remote control. my first attempt was in a sweet looking, but big aluminum case and incorporates a mah battery to power the arduino, servo, and external power for the camera. well, it all works, but i’m not sure why i approached it that way. having extra power for the camera is essential for some applications, but i’m not sure why i was so anxious to marry the two projects into one. just catching on: mysql supports tables in plain csv the storage engine docs are quite clear — “the csv storage engine stores data in text files using comma-separated values format” — and yet i never realized mysql supported it. sure, the tables don’t support indexes and repairing them seems riskier than with other tables, but it still seems to offer a lot of convenience for some things. a comment in the docs suggests how easy csv exports can be: on gamification stowe boyd, remarking on the pew internet project report on gamification in which he was quoted: the need for a renewed push in the enterprise to reengage every person with their personal work, to find meaning and purpose, has never been greater. but adding badges to users’ profiles on whatever work management tool the company is on, showing that bette is a super expert customer support staffer, or whatever, is the shallowest sort of employee recognition, like giving out coffee mugs to the folks with the lowest number of sick days. shutterfingers works! i mentioned my plans to make a servo controller to mechanically press the shutter button on a camera when signaled from a motion control timelapse robot. the parts have arrived and it’s running on a breadboard. i’ve had to make a few changes to the code, including fixing a variable reference, but the biggest change was to implement the internal pull up resisters on the arduino and reverse the logic. that simplifies the wiring. i guess i missed the hand car regatta i followed the raygun gothic rocketship from its former site near the ferry building in sf to its new location in calgary, to the website of the artist collective that made it, to another of their projects: the lumbering contraption, to the internet archive cache of the website for the event at which the contraption appeared, the abandoned facebook page for the event and the april notice that, after four years, the event was well and truly over. pcb prototyping services expresspcb promises for a fixed price of $ , you will receive identical layer, . ″ x . ″ pcbs with solder mask and silkscreen layers. that seems like a good plan, but i’m also very new to this market. are there other, better options? and, as long as i’m asking, what software is available for macs to sketch out the schematics and layout pcbs? this spammy article names some free choices and led me to a mac port of kicad. simple cameras john gruber links to mike johnston’s post asking: i mean, with hundreds of cameras on the market, wouldn’t you think they could make one that was super-simple, just for that segment of the population that wants it? to this i offer the panasonic lumix lx . i’ve been pretty in love with it lately, and i think it’s the perfect answer to that question. that’s the camera that defied the megapixel race of the late s. installing and using mencoder for timelapsing i have a new computer, which has me looking up my old documentation on how i encode still photos from a timelapse series into a video file. as i often do, i’m blogging about it now to make it easier to find next time i need to remember what to install and what settings i’ve found work well. i’ve seen a number of different solutions, but i mostly use mencoder, a command-line tool. of course i want an enfojer enfojer is an enlarger that uses your smartphone as both light source and negative. it’s on indigogo now. from the faq: what lens are we using in the enfojer? it is a wide angle polycarbonate toy camera style meniscus lens. it blurs the image just right so you don’t see the pixels on your print. yeah, we tried sharper and better ones, but the results were too sterile. fujifilm x and sony nex lenses if i get a new camera system i’ll need new lenses. i’m looking carefully at the sony nex e-mount and fujifilm x-mount because they offer fairly compact cameras with large, aps-c sized sensors. on top of that, however, i usually like to shoot a very wide-angle lens. on a sony nex, my best choice might be sony’s - mm sel- . that’s mm after the . x crop factor, and that’s just fine. on the downside, it’s an $ lens, and only has an f maximum aperture. what camera systems are worth it? given that my feelings for canon’s lackluster approach to mirrorless cameras, i’m now obligated to look for a new camera system, and that has me looking at cameras i’d previously ignored. fujifilm’s x system is a recent entrant into the interchangeable lens mirrorless camera fray (note that not all the cameras in the x line sport interchangeable lenses, or similar sensor sizes or body types). the x-e received a gold rating from dpreview, and the new x-m is looking like another good camera as well. the eos m system might as well be dead amazon is now selling eos m cameras for $ with free shipping. at that price you have to think about buying it as a joke, but that’s exactly what it is. the camera is hobbled by canon to avoid cannibalizing sales of their other products. consider this: fujifilm’s x series, sony’s mirrorless nex and cameras, panasonic and olympus‘ micro four thirds mirrorless cameras, and others offer good manual controls despite their small size. shutterfingers i started work on my first arduino project today, though i have yet to get the hardware. the plan is to build a servo controller that can trigger the shutter on my panasonic lx camera that lacks any sort of remote shutter release. i started looking into this before and found cris benton struggled with the problem as well. i’m planning to go down a path he blazed some years ago: put a servo on it. building geos on centos it should be simple, but i ran into a number of errors. first i got stuck on libtool: line : g++: command not found. it turns out i needed to install g++ using: yum install gcc-c++ then i got stuck on this one: platform.h: : : error: #error "can not compile without isnan function or macro [...] "coordinate.inl: : error: ‘isnan’ was not declared in this scope the author of this page faced the problem, but the real insight came from this bug report on an unrelated project: about those battery life ratings i added battery life as a factor in my recent review of cameras, but what does the reported battery life of a camera mean? assuming the translated pdf is correct, cipa standards for camera battery life amount to something like this: take pictures continuously until the camera shuts down due to power loss. fire the flash at full power for every other photo, if the camera has a flash. lumix lx sample photos a friend was asking about the lumix lx i named in my camera roundup the other day and earlier this year. i keep the lx in the list because of my experience with it’s predecessor a couple generations earlier: the lumix lx . he asked how it performs, but i struggled at first to find photos demonstrating it. i began to wonder if my memory of the lx was a little more glowing than the reality. why in-camera gps matters i concluded my review of current camera options with the claim that i’d switch lens systems for a compact interchangeable lens camera that had built-in gps. why do i want gps? because the competition for all the cameras i listed there is my iphone, and one of the reasons i prefer my phone is because every photo i take with it is a little breadcrumb helping me track my travels with very accurate date, time, and location information. summer camera options i reviewed a lineup of cameras i’d consider to replace my aging canon rebel xti and panasonic lumix lx back in february, but i’m on a roll after collecting some film camera party packs so i decided to update this list as well. since i gathered my original list i’ve started using motion control robots and my photo habits have changed. given that, the priority of some of the options has changed a bit as well. back to the vault: old vacation pics shot on film my love letter to film cameras as a solution to smartphone addiction at parties had me looking for some old film photos. do we enjoy the idea of film more than the reality? i found a set of photos from a vacation to las vegas in april . it’s clear that whatever photographic technique i’d developed years before had gone fallow. at the time i was shooting with an olympus stylus epic, probably on kodak or speed print film. film camera party-packs in the old days, or the s at least, party hosts distributed disposable cameras. then digital cameras and smartphones after that became common. the number of photos has been growing, and in some cases so has the quality. but as the number of cameras has exploded so has the presence of cameras themselves in the photos, and as groups of people line up to be photographed, they’re often now outnumbered by photographers on the other side. detect mysql’s “too many connections” error wordpress appears to continue with execution even when mysql refuses connections/queries after init. here’s a comment in the mysql docs suggesting how to detect the condition in raw php: $link = mysql_connect("localhost", "mysql_user", "mysql_password"); if (mysql_errno() == ) { // == er_too_many_user_connections (mysqld_error.h) header("location: http://your.site.com/alternate_page.php"); exit; } just a note to myself, but i wonder if there’s opportunity here. sf gentrification debate i wade into this topic wearily, but i do love my new city, even in the moments where it drifts from critically self-aware to navel gazing. ian s. port’s july review of the media coverage of the gentrification debate included this nugget discussing ilan greenberg’s angle on the topic: [w]hat’s happening here isn’t gentrification at all, but merely middle-class residents using the word to conceal discomfort over richer people coming in and ruining their good time. data sources for geographic boundaries world.geo.json to mock something fast and loose with geo-json data for the world, this is your fix. legal status of this dataset: dubious? for a good time, drag them to http://bl.ocks.org/ and paint the globe! world-atlas [a] convenient mechanism for generating topojson files from natural earth. natural earth natural earth is a public domain map dataset available at : m, : m, and : million scales. featuring tightly integrated vector and raster data, with natural earth you can make a variety of visually pleasing, well-crafted maps with cartography or gis software. built for a purpose: geographical affordances and crime in cabinet spring , geoff manaugh investigates the relationship between geography and the crimes that geography affords. in the s, los angeles held the dubious title of “bank robbery capital of the world.” at its height, the city’s bank crime rate hit the incredible frequency of one bank robbed every forty-five minutes of every working day. [an fbi special agent once joked] the agency even developed its own typology of banks in the region, most notably the “stop and rob”: a bank, located at the bottom of both an exit ramp and an on-ramp of one of southern california’s many freeways, that could be robbed as quickly and as casually as you might pull off the highway for gas. peeking into other people’s photo rigs this all started because i went looking for a way to remote trigger a panasonic lumix lx . the internet is pretty certain that the only way to do it is mount a servo to mechanically press the shutter button. sad. but that led me into cris benton‘s world of photography from poles. yes, he mounts his camera at the end of a carp fishing pole (a noun so unknown to me i almost put it in quotes) to loft it up to ′ in the air. speeding up mysql joins on tables with text columns, maybe the thing about wordpress’ db schema is that text and varchar content is mixed in the posts table (to say nothing of the frustrations of datetime columns). that’s not such a problem for a blog with a few hundred posts, but it’s a different matter when you have a few hundred thousand posts. and it wouldn’t even be a problem then, except for this quirk in mysql: instances of blob or text columns in the result of a query that is processed using a temporary table causes the server to use a table on disk rather than in memory because the memory storage engine does not support those data types (see section . what is the difference utf _unicode_ci and utf _general_ci? from the mysql manual: for any unicode character set, operations performed using the xxx_general_ci collation are faster than those for the xxx_unicode_ci collation. for example, comparisons for the utf _general_ci collation are faster, but slightly less correct, than comparisons for utf _unicode_ci. they have a amusing “examples of the effect of collation” set on “sorting german umlauts,” but it unhelpfully uses latin _* collations. and another table that helpfully explains: a difference between the collations is that this is true for utf _general_ci: canon + ios tethering solutions there’s magic that happens inside the camera. yes, magic. most cameras expose the controls to that magic via some knobs and buttons and a small lcd screen. the knobs and other physical controls we like, but the screen pales in comparison to those on our iphones. and that’s the thing, the hundreds of apps on our iphones leaves us wondering why our dslrs aren’t an open platform, ready to be reshaped by one app after another. testing apply_filters() times testing how long it takes to assign a variable versus assigning through wordpress’ <a href="http://codex.wordpress.org/function_reference/apply_filters">apply_filters()</a>. filters are core to wordpress, but i haven’t yet looked at the total number of apply_filters() calls used throughout the code. the answer to this question is that calling a non-existing filter before assignment is about times more costly than simply assigning it. that’s nothing compared to the cost of actually doing some filtering, however. clarity from a distance the sky looks big from earth, but it’s rather different the other way around. i’m not saying it’s not quite an experience, but inspecting the metadata on this photo of new york and surroundings taken on christmas day, , during the first international space station mission surprised me. to wit: it’s only a mm lens. granted, that’s on an old kodak dcs digital camera (a nikon body with kodak imaging unit attached) with a . rd party js libraries cause downtime facebook connect went down hard tonight. huffpo reports that their site was redirecting to a facebook error page, even when people weren’t attempting to log in. yep. busted third-party javascript brings portions of the internet to its knees: huffingtonpost.com/ / / /fac… — kent brewster (@kentbrew) february , it makes me more comfortable with our decision to strip so many rd party javascripts from gigaom during our last redesign. camera frustrations and other first world problems i’m not a camera pro. i have some photos on flickr, but it’s just for fun, so i don’t really need a new camera. but i do want one. thing is, there a lot of cameras out there, but none of them has the goldilocks factor. none has the right mix of features, size, and price that makes me happy. i now have an old canon rebel xti, panasonic lumix lx , and gopro hd hero in my camera bag, but i began to feel an itch when i realized my mm f . testing file include times for a file that may or may not exist question: should you check for a file before attempting to include it, or just suppress errors? calling file_exists requires stating it twice if the file does exist, so that could take longer. answer: the file_exists pattern is more than five times faster than the @include pattern for a file that doesn’t exist, and not substantially slower when the file does exist. the test: <?php $start_time = $end_time = $i = ; $start_time = microtime( true ); for( $i = ; $i <= ; $i++) { include __dir__ . an american iphone in europe by way of update on my earlier post after researching options for at&t iphone users in europe (with an unlocked phone), i ended up not bothering with local sim cards in either the netherlands or france. a savvy user should be able to find a local pay as you go sim plan that’s less expensive than at&t’s data roaming packages, but i’m that user and know very little about the local operators (not even all their names). svn or git? @film_firl poked @wordpressvip to ask @wordpressvip @mjangda @viper bond moooove to git!!! she half-kids. no really, please? — christina warren (@film_girl) january , @nacin piled on with @viper bond @film_girl @mjangda vip aside, it’s fairly crazy that wordpress.com hasn’t migrated. svn != tenable dev environment. — andrew nacin (@nacin) january , @viper bond tried to defend the team, and added @film_girl @wordpressvip @mjangda that said transitioning is not always worth it. where did all the votes go? what happens to voting data after the election is over? what happens to all those certified results by polling place? how is it that there’s so much coverage leading up to and on the night of the election, but this guy seems to be one of the few sources of historical voting data? amusingly, i found it linked on the library of congress’ website! there’s some very old sources from e. on wp_enqueue_scripts and admin_enqueue_scripts an argument has erupted over the wordpress actions wp_enqueue_scripts and admin_enqueue_scripts vs. init. one of the points was about specificity, and how wp_enqueue_scripts and admin_enqueue_scripts can reduce ambiguity. i didn’t realize i had strong opinions on it until the issue was pressed, but it turns out i think wp_enqueue_scripts and admin_enqueue_scripts are unnecessary and unfortunate additions to the actions api. here’s what i wrote in that discussion thread: is spatula city the store that’s most specifically targeted to the sale of fine spatulas? confirming that object references in arrays are preserved while cloning the arrays a short test to confirm references are preserved in cloned arrays. // create a stdclass object (using my lazy way of coercing arrays to objects) $object = (object) array( 'thing' => 'original' ); // add that object to an array element $array = array( 'object_one' => $object ); // clone the array by assignment to a new variable $array_two = $array; // add a new copy of the original object to a new element in the new array $array_two['object_two'] = $object; // show what we have so far var_dump( $object , $array , $array_two ); the result is: ignoring noise in svn diffs svn diff -x "-bw --ignore-eol-style" is your friend when somebody decides to change the end of line style and strip all trailing whitespace from the files in your repo. is perl the best solution to write code that needs setuid? a bunch of searching the web for things related to setuid and shell scripts lead me to this answer in stack exchange: perl explicitly supports setuid scripts in a secure way. in fact, your script can run setuid even if your os ignored the setuid bit on scripts. this is because perl ships with a setuid root helper that performs the necessary checks and reinvokes the interpreter on the desired scripts with the desired privileges. there’s no ‘git cp filename’? here’s a sequence of unbelievable things: yes, despite a lifetime in subversion, i’m really this new to git! i’m going to link to livejournal in this post! git really doesn’t have an equivalent to svn cp filename! i spent a surprisingly long time reviewing the man pages and surfing the internet to confirm this, but git really assumes you’ll never want to copy a file with history. here’s that livejournal link i promised, where markpasc has similar complaints — from , no less. aww, i got thanked! i recently backed the syrp genie, one of a handful of recent motion control timelapse projects on kickstarter. it’s well past its expected ship date, but they done a good job of keeping backers updated on progress and just today they shared photos of the box that will soon be on it’s way to me. they’ve thanked backers with a card in every one of them. if you look closely, you’ll see my name straddling the “thanks” in the center. greetings library scientist the california library association is pretty much like every other regional library association i’ve seen, not least because their most visible presence is their annual conference. it may be the season, but the cla is more politically active than others i’ve known. at their core, most such associations exist to promote efficient transfer of operational knowledge from one library to another, from one generation to another. libraries today unfortunately, in less than a generation’s time, the very foundations of libraries has been rocked by technological, legal, and economic changes unlike any these organizations have seen before. our arbitrary alphabet we have been gaslighted by the alphabet and now believe the arbitrary string of letters is actually organized according to some plan. hegemonic language and arbitrary order the signs used in writing originate in arbitrary decisions, but the connection with arbitrariness is lost when convention takes over. the convention of long usage kills even the memory of the initial arbitrariness of the signs and gives them an objective and seemingly inevitable presence. strange things running on my mac my imac screen is dark and isn’t lighting up like i expect it to when i tap the keyboard. i can, however, ssh into it and see what it’s doing when not responding to me. i found googlesoftwareupdateagent running, this faq item vaguely tells me it’s part of chrome, and that if i try to uninstall it without also uninstalling chrome it will simply “be reinstalled after a few hours.” action camera market not yet saturated, according to sony i wondered if the gopro-style action camera market had already become saturated back in january, now i’ve learned that sony apparently doesn’t think so. at least one imagines that’s the conclusion they came to before deciding to join the competition with a camera of their own. they call it the action cam, and it clearly takes its design cues from contour. what does sony offer to stand apart from the established players? usb camera control problem the canon eos m doesn’t include a remote shutter release cable port, and the on-camera controls don’t expose features such as bulb-mode exposures. further, simple remote shutter release doesn’t support the sophisticated camera control necessary to do timelapses with complex exposures. what kind of complex exposures? imagine a timelapse going from day to night. during daylight the exposure might be f , / second at iso , but the night exposure might require f / second at iso . geography vs. stereotypes alphadesigner is trying to put a finger on it with his mapping stereotypes series. others, including how americans see europe and the world according to america, are not nearly as well designed. we’d be fools, however, to think we invented the idea of mapping our prejudices. this flickr set of maps from through is good evidence of that. chance vs. lasers via tweet: claw arcade games are not skill games, rather, the claw strength is randomized and is often only strong enough to successfully grab the prize in one attempt out of , or . operator manuals linked in the quora answer explain the different modes and odds. string cutting games, however, can be defeated with lasers! apigrove: api management software apigrove is an api management tool by alcatel-lucent. it proxies apis (presumably those you built and host, though the example is for twitter) , supports authenticated access, throttles to help manage demand, usage logging and reporting. more info @apigrove, hat tip. be careful what you measure seth godin on what to obsess over: what are you tracking? if you track concepts, your concepts are going to get better. if you track open rates or clickthrough, then your subject lines are going to get better. up to you. it’s long something i’ve believed: if you measure it, you will attempt to maximize it, even if the metric is something you’d rather minimize, like co emissions. preparing my iphone for europe there’s uncertain talk of a european trip coming up, so i’m making nonspecific preparations for it. one of the questions i have is how to avoid hefty roaming charges from at&t. in previous trips abroad i’d purchased overseas voice and data add-ons so i could use my iphone. that works, up to a point. on my return home from a trip to taiwan a few years ago i got a call from at&t informing me that i’d gone over my data limit and was facing a $ charge for the usage. higgs-bugson a higgs-bugson is a hypothetical error whose existence is suggested by log events and vague reports from the users that cannot be reproduced in development conditions. qa and user support teams point to the higgs-bugson as an explanation for the results they see in the field. software engineers, however, often deny the existence of the higgs-bugson and offer alternative theories that often blame the user. engineers, after all, don’t write bugs. gopro hd hero lens correction gopro’s hd hero action camera is everywhere, so perhaps we’ll all be used to the fisheye’d images it produces soon. on the other hand, there are software solutions to rectify the image to rectilinear. vimeo user peter inova has a few videos demonstrating his photoshop action sets to straighten out an hd hero’s output. a person could probably significantly improve performance by giving up on photoshop and building a video filter based on the panotools image manipulation library. making sense of at&t’s shared data plans kevin’s coverage at gigaom helped, but what i really needed was a chart that compared the different options. i couldn’t find one, so i made my own: <td valign="top"> <strong> iphones</strong> </td> <td valign="top"> <strong> iphones</strong> </td> <td valign="top"> <strong> iphones</strong> </td> shared data, unlimited minutes gb <td valign="top"> $ </td> <td valign="top"> $ </td> <td valign="top"> $ </td> gb <td valign="top"> $ </td> <td valign="top"> $ </td> <td valign="top"> $ </td> gb <td valign="top"> $ </td> <td valign="top"> $ </td> <td valign="top"> $ </td> gb <td valign="top"> $ </td> <td valign="top"> $ </td> <td valign="top"> $ </td> gb <td valign="top"> $ </td> <td valign="top"> $ </td> <td valign="top"> $ </td> gb <td valign="top"> $ </td> <td valign="top"> $ </td> <td valign="top"> $ </td> <td valign="top"> </td> <td valign="top"> </td> <td valign="top"> </td> individual data, shared minutes mb <td valign="top"> . motion control timelapse projects on kickstarter some time ago i backed the syrp genie (estimated delivery july ), but today i learned of the radian and astro. unlike the radian and astro, the genie supports linear motion, but it’s also much more expensive, bigger, and appears to have more complex controls. here are the videos for all three projects: [http://www.kickstarter.com/projects/syrp/genie-motion-control-time-lapse-device/widget/video.html] [http://www.kickstarter.com/projects/ /radian-a-motion-time-lapse-device-for-everyone/widget/video.html] [http://www.kickstarter.com/projects/ /astro-time-lapse-motion-control/widget/video.html] eduard khil, mr. trololo, dead at eduard khil is dead. the man, whose work and career had earned high praise, including the order of the red banner of labour ( ), lenin komsomol prize ( ), order of friendship of peoples ( ), meritorious artist of the rsfsr ( ), people’s artist of the rsfsr ( ), order of merit for the fatherland ( ), and international fame with his performance of trololo. the performance that made him famous: a stage performance: composited timelapse and real-time skateboarding video http://www.vimeo.com/ russel houghten‘s open horizon is part skate film, part time lapse, and mostly awesome. then somebody pointed to this jimmy plmer/z-flex video that shares a number of features with houghten’s work, but is less ambitious in scope. at least they did a behind the scenes video that shows the sweet red camera and rails. find neighbors on the same ip what other sites share the same infrastructure with your site, or any other? bing‘s ip search can answer. do a search by ip number: ip: . . . ip: . . . ip: . . . site load performance benchmarks the loop’s jim dalrymple compiled the following numbers for the time it takes various tech sites to load in a browser in late : the loop: requests; . kb; . secs daring fireball: requests; . kb; milliseconds macworld: requests; . kb; . secs ars technica: requests; . kb; . secs apple: requests; kb; . secs cnn: requests; . kb; secs bgr: requests; . mb; . secs appleinsider: requests; . is this the best imdb api? imdbapi.com css speech bubbles twitter front-end guy nicolas gallagher likes both css and speech bubbles enough to want them unadulterated by images and non-semantic markup. the lesson from his many examples is that it all comes down to an :after pseudo element that puts the little triangle in there: .speechbubble:after { content:""; position:absolute; bottom:- px; /* value = - border-top-width - border-bottom-width */ left: px; /* controls horizontal position */ border-width: px px ; /* vary these values to change the angle of the vertex */ border-style:solid; border-color:#f c transparent; /* reduce the damage in ff . semantic news markup and seo schema.org newsarticle hnews rnews (and the war between rnews and hnews) google news technical requirements on the likelihood of unicorns research by robert e. hall and susan e. woodward shows that % of venture-backed firms exit for less than $ million ( % exit for less than $ million). in a world where instagram can exit for $ billion with no revenue or monetization plan, anything less than $ million is an implosion. marathon spoiler guides marathon and marathon : durandal are available as ios apps. the classic marathon spoiler guides might be good companions. airparrot turns appletv into a secondary display from the faq on the airparrot site: what does airparrot do? airparrot lets you airplay your mac’s screen to a second or third generation appletv. what you see on your mac’s screen will appear on the appletv, wirelessly! how do i use airparrot? once you’ve opened airparrot, click on the icon in your menu bar. select the airplay device (such as your appletv) and then select which screen you want to mirror. sf police, fire, ems, and airport radio monitoring listen in with radioreference.com’s index of live police, fire, ems, and airport radio feeds in san francisco. is this the best way to copy voicemails from an iphone? instructables tells us to get the files from the iphone backup in ~/library/application support/mobilesync/backup/, but “itunes renames all your files xxxxxxx.mddata. so all you need to do is figure out the original file name extension and you will be able to view the file.” ugh, isn’t there a better way? html form elements mark pilgrim’s overview of html form elements includes the following: placeholder text autofocus fields email addresses web addresses numbers as spinboxes numbers as sliders date pickers search boxes color pickers form validation required fields further reading configuring amazon linux for web services (spring ) i’ve tested this cookbook against amazon linux, but it will probably work just as well with the current version of centos. basic installation first, get root and update the os: sudo -s yum update with that done, let’s get the basic packages and services installed: yum install mysql mysql-server mysql-devel httpd httpd-devel mod_ssl php php-devel php-mysql php-gd php-dom php-pear php-json memcached svn gcc pcre-devel make that gets us apache httpd with ssl, php with a number of modules, memcached, and a few system tools. php vs. frameworks six years ago this month the zend framework preview was released and rasmus lerdorf published a blog post titled “the no-framework php mvc framework” (italics added). r. rajesh jeba anbiah noted irony. scanwiches scanwiches: scans of sandwiches for education and delight. above is parisi bakery’s ham, swiss, tomato, lettuce, mustard, mayo, on a hero. prints were said to have been available — i’d like the dagwood, thank you — but the store seems in a sad state. pew internet project: “ % of adults own a tablet computer” we’ve heard stories about how significant the growth of apple’s ipad is, but pew internet and american life project director lee rainie speaking at the national federation of advanced information services (nfais) conference on mobile devices and the delivery of information shared a stat that made me pause: % of adults own a tablet computer – ipad to clarify, that % does not include ebook readers (they’re tracked separately). rob reid’s copyright math rob reid’s copyright math at ted : the claimed effect of entertainment piracy to us economy is larger than value of most of our agricultural output. pantone yummies by emilie griottes: open access and open data finally getting public attention complaints over the cost of academic journals have long been a trope that repeats at library conferences with no denouement, but there are new signs that might be changing. the issue is that a large portion of the research done in the us is performed by faculty paid by academic institutions and supported by public money, often grants from the nih. a significant condition of promotion in academic careers is publication of original research in trusted journals, which is entirely reasonable to most everybody involved, except for the librarians who have to pay for the journals. the microsoft store experience there’s a microsoft store right across from the apple store in the valley fair mall. cliff and i realized this after exiting the apple store there with a new keyboard and headphones. we’d never been in an ms store before, so we ambled over with our clean white apple-branded accessories in hand. the windows phone display was in the back corner, attended by a nice woman who offered to fetch a nokia lumia phone from the back for us to inspect. marta becket’s final performance tonight legend has it that marta becket rolled in to death valley junction in and has been performing at the amargosa opera house since, but tonight is her last performance. i visited in and took in the show then. it’s a certain kind of show and performer that can run years non-stop (it was in its th year when i saw it). action cameras you know about countour and gopro, but you may not have seen drift and swann. is this a market that is getting saturated, or is it about to explode? contour marketing video: gopro marketing video: drift marketing video: http://www.vimeo.com/ swann marketing video: three of the cameras compared: happy new scriblio! the most recently released, stable version of scriblio is marked . -r and was last updated in june . you can be forgiven for thinking development had ceased in the interim. today, however, i’m proud to introduce a completely new scriblio, re-written from the ground up to take advantage of the latest features of wordpress and eliminate the mistakes made in previous versions. this update allows users to search and explore wordpress sites using facets that represent the tags, categories and other aspects of the collection. how wordpress taxonomy query urls could be more awesomer (updated, see below) wordpress . introduced some awesome new taxonomy query features, and the url parsing allows some rudimentary syntax to query multiple terms and choose if the query is or’d or and’d. the url syntax is as follows: a comma (,) between terms will return posts containing either term (logical or), like this http://maisonbisson.com/post/tag/wordpress,mysql/ . a plus sign (+) between terms will return posts containing all terms (logical and), like this http://maisonbisson. ge public relations gets smart to the cool video thing the video from general electric is cool, and shot at least in part with cameras mounted on rc helicopters, but strangely missing is any mention their manufacture of nuclear generation equipment such as the fukushima plants that melted down earlier this year. “hot sweet wings” and other wonders composed with the help of songify cliff introduced me to the wonder of the songify app. here are some tips to making the best of it: longer text makes for better songs. repetition makes for better songs, don’t be ashamed of repeating yourself. speak in a monotone voice, let the app handle the tune. speak nonsense. no sense in trying to make sense, it doesn’t make for a better song. if you insist on trying to make sense, then just pick a single sentence and repeat it several times with slight variations. wikileaks embassy cables first wikileaks published the collateral murder video, then a massive-but-redacted dump of diplomatic cables, then people figured out how to get the unredacted content. though this information was already public, the aclu pursued a foia request on these very cables, the result was a heavily redacted record of the cables, and a clear picture of the government’s ongoing touchiness about torture, rendition, guantánamo, and targeted killings by drones. an on the media segment (mp download) explains further. parallel-flickr backs up your flickr library parallel-flickr: a tool for backing up your flickr photos and generating a database backed website that honours the viewing permissions you’ve chosen on flickr. more details from the website: it downloads and stores your original photos and their “ x” versions. currently photos are stored locally but there’s a plan to add support for s . for each photo it downloads and stores the contents of the flickr.photos.getinfo api method as a json file. predator drones used in domestic police action the la times on december reported that predator drones such as those now being used by the air force and cia were used to support police in their investigation of cattle rustling. theft of livestock has long been a serious matter, but regulations and procedures typically make it difficult to sell stolen cattle. according to fred frederikson of the north dakota stockmen’s association, “all horses, mules and cattle leaving [north dakota] must be brand inspected. the war on cameras wnyc’s on the media did a nice piece on it back in september (mp download): judging from the arrests and harassment, photographers are part of a terrorist plot. or something. the copblock (tagline: “badges don’t grant extra rights”) map of actions taken against photographers is littered with activity. alterego: democratizing two-factor security alterego promises two-factor authentication security without the silly key-fob. neat. electric chariot sure, this electric chariot combines all the inconvenience of a scooter with some of the frustrations of an actual car, but it looks cool. sort of. though it’s made by a medical equipment manufacturer, at least it conforms to the rule of auto shows and objectifies the women demoing it as much as the vehicle itself. correction: steadicam smoothee for gopro hd hero in my earlier post on steadicams for gopro hd hero cameras i incorrectly stated that the steadicam smoothee is exclusively for iphones and ipod touches. they seem to have mounts for gopro hero and flip mino cameras as well, it’s just impossible to find that info on their website and most retailers don’t carry the other mounts. if you don’t mind the color, you can pick up a third-party mount for under $ from shapeways. web strategy discussion starter what follows is the text of a document i prepared to start and shape discussion about the future of the university website at my former place of work. the pdf version is what i actually presented, though in both instances i’ve redacted three types of information: the name of the institution (many already know, but that’s no reason to allow it to appear in search results), pay rates for proposed employees, and identification of proposed service providers. which steady cam is best for a gopro hd hero ? i have a new gopro hd hero , one of the best new video cameras available (if what you like in a video camera is a compact, wide-angle, and waterproof), and i’m looking for a way to steady it for handheld shots. the steadicam smoothee is built for iphones. their demo video and this comparison of the iphone s with and without the smoothee suggest it can work wonders, but it appears to be iphone-only [correction: it’s officially compatible with the gopro]. aoc ″ usb-connected flat panel aoc’s new ″ usb-connected monitor looks like an interesting toy. it draws its power and signal from the usb. mixed information suggests that four or eight can be connected to a single computer. at about $ , this could be a cheap way to build a large display wall. what content should a university website include? i no longer have a dog in this race, but in cleaning up my hard drive of old files i’ve run across a few items of note. for example, the above illustration i once used to describe the different content, audiences, and uses of a university website. current students, prospective students, their family, faculty, employees, and their family all use and expect to get answers from the website. websites for large organizations fail their users when they only share the details that they once exposed in view books and catalogs. what went wrong if i’m lucky, the only reason i get a phone call before am is because somebody on the east coast forgot about the timezones between them and me. the alternative is almost always bad news. today i wasn’t lucky, and neither were a huge number of readers and users at gigaom who received multiple copies of our daily newsletter. for a news and research organization that values — loves — its users as much as we do at gigaom, this was all hell breaking loose. comcast’s folly [harry shearer] , the bassist for spinal tap, voice talent for many characters in the simpsons, and host of le show has no difficulty criticizing the unnecessary complexities of modern media technology, but not until his august episode (subscribe to the podcast) has he admitted to the frustrations of modern cable. “it’s now easier to watch tv on your computer than on your tv,” says shearer. perhaps that’s why comcast, the leading cable operator in the us, lost , tv subscribers last quarter, and the company has been seeing its subscriber base shrink for a while (though they’re showing growth in internet subscribers). the end of paper domtar, “the largest integrated manufacturer and marketer of uncoated freesheet paper in north america and the second largest in the world,” launched a campaign to promote paper consumption. this much is old news, as the campaign is about a year old already. among the messaging goals, according to the agency that designed it: it’s easier to learn on paper, because reading on paper is up to percent faster than reading online. search the sears and roebuck catalog you’d think the sears archives would offer an online search of their historical catalogs, but the best you’ll find is a list of libraries holding the microfilms. ancestry.com offers an online search, but only to paying members. i’m looking into this because i was looking for historical trends in consumer products and thought the catalog would be a good source. it might be, if only i was ambitious enough to go to my downtown library. ed rondthaler’s spelling reform flip chart http://www.vimeo.com/ ed makes a good argument for spelling reform, but he demonstrates an outstanding flair for presentation, even at the age of . sara cannon on responsive web design at wcsf sara cannon‘s talk on responsive web design (resizing the page to suit different client devices) was spot on. her slides are below, but she also recommends this a list apart article on the matter, as well as less framework and css grid (especially as alternatives to .gs). responsive web design – wordcamp san francisco view more presentations from sara cannon estelle weyl on css at wcsf i’ve long been a fan of css , but estelle weyl‘s wordcamp sf talk on it charged me up again. her slides are not to be missed. an interesting insight into mobile safari on ios a post in a y combinator discussion thread: mobile safari parses websites as a big canvas and then pretends the screen is a window through which you’re looking at the canvas. what you think of as scrolling, the browser thinks of as moving the canvas around (or the window depending on point of view). because of that, no scroll events ever get fired. even :fixed doesn’t behave as expected. applescript: get every movie in itunes applescript can be frustrating, but it’s an easy way to get info out of itunes. the following is a fragment of a script i’m working on, this part simply gets a record list of every video in the current itunes window that is a “movie” (the alternatives include music videos and tv shows, among others). credit goes to some examples i found in doug’s applescripts for itunes. boo, however, to a few scripts that are saved as “run only” and can’t be inspected, even for security. civic comparators it’s from early , but cameron marlow’s comparison of sf to nyc neighborhoods and jason kottke’s comparison of the physical geography are amusing to me as a new san franciscan. on the other hand, is it a sign of civic insecurity to make such comparisons? doublehappy game creator doublehappy, by instinct, the same folks who make the getshopped ecommerce plugin for wordpress, is an interesting game creation tool. all the game elements are stored in wordpress using custom post types and other advanced features, but it was their demo of the html editor that most amazed me. the games still play in adobe flash, but surely they’re working on rendering that to html as well. using keynote as a motion graphics tool bill keaggy just posted on the xplane blog about using apple’s keynote presentation software to make motion graphics and movies. we’ve found that in some cases, a keynote-authored video is what you might call the “good enough” solution. […] keynote lets you create and edit presentations, make things move, is ridiculously easy to learn and exports to quicktime. he offers his tips on how to make the best of it, as well as these videos made using keynote: notes to self: twitter’s website rocks on mobile devices twitter’s mobile site rocks on my iphone. especially worth noting: they’ve figured out how to pin their header to the top while scrolling the content in the middle. they’re also using pushstate() and other cool tricks to make the experience feel very native, but the scroll behavior is rare among web apps on ios. kent brewster makes a point about how difficult it is in his mistakes i made building netflix for the iphone talk from sxsw. wordpress nocache_headers() vs. nginx typically, you can call wordpress’ nocache_headers() function when you don’t want content to be cached. typically, but when you’re serving from behind nginx as a reverse proxy, consideration must be paid. it’s a year old now, so i shouldn’t have been surprised by it, but this thread on the nginx forums explains that cache-control: private headers are meaningless when nginx is being used as a reverse proxy: nginx completely ignores the ‘private’ keyword and will cache your document regardless. phpquery i have matthew batchelder to thank for introducing me to phpquery. i haven’t used it yet, but someday i’ll have need to select text elements from html using the php pear module. from the description “server-side, chainable, css selector driven document object model (dom) api based on jquery javascript library.” i get email: food tech society’s food ingredient and food additive forum the july food ingredient and functional additive forum looks to have a great lineup of talks, including nano food, interesting ingredients in milk and dairy products, ingredients in functional food and drink, sea food & frozen industry, and minutes (the longest of any of the talks) set aside just for soy sauce. incoming support request you haven’t fixed the bing search page on cafe world. it comes up when i click on an oven, when i click on a mission and then everything is ruined. fronterville: i haven’t been able to play fronterville for four days. i can send gifts, but don’t know if anyone receives them but they must because i get gifts. but i have a spouse and it is stuck. it won’t custom or random or play or anything and it freezes the whole page so i can’t do a thing and there is a white avatar that says spouse? smiley’s bar, bolinas, ca captain, ship, crew, twelve points, and a shot of whisky at smiley's i heard a story that the “bolinas border patrol” removes all the signs pointing to town, so cliffy and i had to go check it out. border patrol or not, there are no signs, but smiley’s bar is my kind of place. given the story about the signs, i worried they’d be leery of outsiders, but it turned out to be the sort of place that welcomed you in and offered you a glass. social compass it looks gorgeous, but the points and bearings brian solis lays out in his social compass seem so obvious to me that i almost dismissed it as meaningless. then i remembered there really are people who don’t know the message they’re trying to send will be filtered through people and technologies they can’t control and depend on adoption and repetition by agents working in their own interests. anyway, there are more posters in his store. radiation is all around us the environmental protection agency on radiation and cigarette smoke: studies show filters on ordinary commercial cigarette remove only a modest amount of radioactivity from the smoke inhaled into the lungs of smokers. link. photo by lanier . the story of nukey poo the video of nuclear boy and his stinky poo that’s supposed to explain japan’s nuclear crisis isn’t the first time anybody has mixed poo and nuclear reactors. a reactor at antarctica’s mcmurdo station that operated through the s was nicknamed “nukey poo” because of its poor performance and reliability (though some reports simply point to “frequent radioactive leaks”). first, here’s the japanese video: the original nukey poo was oficially named pm-a . nostalgic joy: apple emulators you can emulate an apple ][ or apple iigs in your browser with a plugin and , disk images, including oregon trail. don’t want to run an apple //e in your browser? download [virtual ] for the job (you’ll need disk images and a rom file). sweet can answer your apple ][gs emulation fix, and there’s a surprisingly large collection of sort-of-recent software available, including castle wolfenstein d, an html editor, and aim client. what time is it? the claim that changing the clocks saves energy is unsupportable by facts. some say it’s more likely to spur consumption and benefit commercial interests, but i’m curious why the teabaggersparty people haven’t risen up against this alarming government intrusion into our private lives. wijax widget lazy loader idea: a simple way to improve load-time performance by lazy loading some of the content on the page. answer: wijax. the more content in the initial download of the page, the longer readers have to wait to see it. some content is critical to each page load, but why make people wait for every last piece of the page before they can start reading the post they came to see? wijax allows you to defer loading widgets on the page so that they arrive after the main content. net render your ie compatibility tests maisonbisson in ie geotek‘s netrenderer makes it possible for me to see how badly old versions of ie are mangling my web pages without actually having to run the malware on a box of my own. unfortunately, the ie rendered returns errors and hasn’t worked in a while. maisonbisson in ie ebook user’s bill of rights it’s easy to see the ebook user’s bill of rights as a sign of the growing rift between libraries and content producers. easy if you’re me, anyway. it connects very conveniently with richard stallman’s open letter to the boston public library decrying what he summarizes as their complicity with drm and abdication of their responsibilities as public institutions. all those things are easy, what’s hard is recognizing that the depth of change the publishing industry is facing. van ness station escalator ambient video flickr video more mesmerizing than a fireplace video? saving backup space with time machine and iphoto three things that, when mixed, can consume a surprising amount of disk space: backup automatically with time machine use iphoto and take a lot of photos sync photos to one or more ios devices like iphones and ipads i do all three, and on top of that i have three current computers backing up to a gb time capsule. all of this combined was forcing time machine to expire old backups faster than i wanted as it churned through the disk space. wordpress comments_template() and wp_list_comments() performance this thread on memory usage while executing wordpress’s comments_template() raised my awareness of performance issues related to displaying comments on posts in wordpress. the first thing to know is that all the comments on a given post are loaded into memory, even if the comments are paged and only a subset will be displayed. then comments_template() calls update_comment_cache(), which has the effect of doubling that memory usage. finally, wp_list_comments() and the walker_comment class can take a surprisingly long time to iterate through a long list of comments. gigaom mobile site launched this week we launched a new mobile theme at gigaom.com. it was out for just a day or two before dennis bournique surprised us with a review on wapreview.com. i have no way of knowing if i would have linked to the review if it wasn’t positive, but i would likely have found a way to link to this advice to other developers regarding url consistency: a url should lead to essentially the same content (reformatted in necessary) regardless of which browser is used. helvetic neue on the web css tricks tips “better helvetica.” guillermo esteves explains that specifying font names in css is really about specifying font families: if you want to use a specific font face, you have to use font-family along with the font-weight property, calling both the postscript and screen names of that face for backwards compatibility which, for a person trying to use helvetica neue light means the following: font-family: "helveticaneue-light", "helvetica neue light", "helvetica neue", sans-serif; font-weight: ; steve cochrane, meanwhile, explores the use of helvetica neue light and ultra light. call it rolling shutter or focal plane shutter, it looks weird…cool i’ve been both frustrated by and in love with focal plane shutter distortion (wikipedia calls it rolling shutter) for a while, now i’ve discovered there’s a group for it. one of the photos i pointed to in my earlier post was of a low-flying helicopter (bottom), a couple other photographers have captured the effect the distortion has on propellers: about those unencumbered video formats the free software foundation tells us the h. avchd video encoding standard violates the very tenets of freedom, they claim competitors such as vp /webm and ogg theora are both unencumbered and technically equal to h. . what they really mean is that software patents are evil. now the mpeg la, the body that administers the h. patents and a number of others has announced it’s forming a patent pool that covers vp , proving that saying something is free doesn’t make it so. iphone camera details i have to look this stuff up every time i play with hugin, the open source panorama stitcher. thankfully i can find it at falk lumo.com: pixel pitch: . µm sensor size: . x . mm^ , . mm diagonal aspect ratio: . : focal length and aperture: . mm f/ . lens mm equivalent crop factor: . equivalent mm focal length and aperture: mm f/ the comments there are top notch, but what’s not mentioned is how the video mode substantially narrows the field of view. wordpress mu/ms empty header and broken image bug fixed i just switched to a new server and found myself struggling with empty http headers and broken or partial images. the problem is the memcache extension for php and wordpress mu/wordpress multisite’s need to reinstantiate the wp-cache after determining the correct blog for a given request. versions of the memcache extension prior to . go wrong somehow and it shows up when you try to do an http head request on a page (the result is empty) or enable x-sendfile support for wp mu/ms’ file handling (all the files and images in the media library will break). configuring amazon linux for web services updated: an updated installation cookbook is available. amazon has introduced their own distribution of linux with tweaks to optimize it for their elastic compute cloud platform. like centos, it appears to be based on red hat enterprise linux, though unlike the current versions of rhel and centos, the packaged applications are up to date with current expectations. that’s refreshing news for those comfortable with rhel, but uncomfortable its ancient packages. mysql . world’s largest canned food structure some records in the guinness book reflect outstanding accomplishments in hotly contested fields. others reflect the imagination it now takes to create a new class of records. food industry thailand‘s , food cans fall into the second category. don’t get me wrong, though, i’m not suggesting anybody’s imagining new fields, just that they’re imagining themselves pursuing crazy records. examples of things i think we should have records for, but i’m too lazy to look up: happy holidays from maisonbisson! another cheesy holiday card from maisonbisson and, for those who like cheese as much as us, from left to right: cotswold double gloucester with onion & chive mannoni pecorino barbagio point reyes toma we picked them mostly for color and texture, but they all tasted plenty good. i especially liked the cotswold. a holiday gift, thanks to some genius and hardworking djs, is in the nest. facebook iphone app is happy to suck in your contacts i discovered a sync button in the facebook app for iphone today: then i read the privacy notice: clearing the browser cache on ipad apple’s knowledge base article on it could be as simple as the following screenshot: instead, the docs say something like: go to settings, click the safari tab, click the big clear cache button, duh. so now you know: world’s heaviest snow plow this probably looks like a snow blower, but the railroads call it a snow plow. a rotary snow plow, yes, but still a snow plow. a ton, foot long snow plow. caveman explains: the union pacific railroad designed and built this monster in the omaha shop. this rotary snowplow is the heaviest snowplow ever built. this baby boasts a gm/emd -cylinder, , horsepower, turbocharged diesel engine that drives an electric generator which provides the power to turn those massive -foot rotary blades at rpm. where are san francisco’s love padlocks? i discovered it in the flickr blog and followed it up with considerable googling, but i can’t find any love padlocks in sf, much less a popular location for them. the wikipedia article lists two dozen notable locations in europe and asia, but not one in the americas. i searched flickr’s san francisco map and found two almost promising photos: an unrelated collection in the mission that was removed by municipal workers in , and this one in my backyard that i plan to confirm shortly. failed hard drive noises there’s nothing amusing about this list of failed hard drive noises if you’re looking through it for a sound matching what drive on your desk is making (which i am), but i’m sure there’s some good material for the click-hop crowd. photos by jon ross and james harvey, used under cc license. better xml/json display in safari i’m one of the few people who loves safari, but i was happy to admit that it didn’t display xml or json very well. marc liyanage’s xml view plugin fixes that. improving will norris’ open graph plugin will norris put together a nice wordpress plugin to place open graph metadata on the page. today i patched it to address a few bugs i and others have found. the patch switches functions that depended on globalizing $post to use $wp_query->queried_object and similar. opengraph_default_url() is changed to try get_permalink() only when is_singlular() is true. otherwise it uses the blog’s base url. this isn’t perfect, but it’s better than having the front page and all tag/category/archive pages report their og:url as being the permalink for the first post on the page. things learned about the gap inc. corporate archives if a customer saw it, or if it was shared with employees, i want some version of it in our archive. –rochelle mccune, gap corporate archivist rochelle took a few of us on a tour of the gap inc archives, a rather different archive than i’m familiar with. things learned about natural language processing at thatcamp bay area the first session i joined at thatcamp was aditi muralidharan‘s text mining boot camp, and the topic seemed to set my agenda for the rest of the event (though i wish aditi had also hosted her proposed data visualization session). aditi’s blog: mininghumanities.com. if i understood correctly, much of aditi’s presentation and experience is based on the stanford parser. unfortunately, the project seems wrapped in some licensing difficulty: it’s gpl, but they claim a license is required for commercial use. becoming donna reed sandee has just launched her new site, becoming donna reed: armed with a notepad and pen, my trusty macbook, and the desire to be the best domestic goddess i can be, i will watch the show from the beginning and find the lesson in each episode. consider this your cliff’s notes on household harmony. she’ll still be updating the feathered nest with food recipes and insights on home decor while she divines the lessons of donna reed. what the critics are missing about the apple tv it’s not just the critics, nobody seems to get the story on apple’s new tv-connected device right. darrell etherington at the apple blog says it’s a non starter for him, and ars technica’s john siracusa describes it as just the most recent entry in a product line that has been “a persistent loser” for the company. even john gruber is damning it with faint praise. they’re all wrong. of course the problem didn’t start there. dancing dog i’ve got a dozen top priorities this morning, but this dancing merengue dog just delayed them all. twitter is like a conversation in a bar mathew ingram on twitter, esquire magazine, and bars: it’s called social media because it’s social. in other words, it’s a conversation; and yes, sometimes it’s like a conversation in a bar. speed wordpress multisite with x-sendfile for apache like wordpress mu before, multisite implementations of wordpress . use a script to handle image and other attachment downloads. that script checks permissions and maps the request path to the files path on disk, then reads the file out to the web server, which sends it to the browser. that approach has some inefficiencies, and for me it introduces some problems. the process would often give up before completing the file transfer, resulting in broken images and truncated mp s among other problems. post loop by category alex bluesummers asked on a wordpress list: how do i order posts in the loop by whether or not it is in a category, then by date? suppose i have posts, of which are in the category “sports” and are in the category “blog news”. both “sports” and “blog news” posts are mixed together chronologically. “sports” and “blog news” posts share other categories and tags. i want both types of posts to be present in the loop regardless of whether it’s the front page or category archive view, but ordered by “sports” and “blog news” and then by date. migrating from wordpress mu to wordpress . multi site i’ve been running a few instances of wordpress mu for a while now, so i was more than a little anxious about the merge of the mu functionality into the core of wordpress. it’s a good thing, but sometimes such dramatic changes pose rocky challenges. not so in this case. pete mall blogged about it in may, and i’m happy to say that i followed those instructions (summary: upgrade, it will work) to upgrade both this site and scriblio. donut tour : the video please enable javascript and flash to view this viddler video. we planned the donut tour. we did the donut tour. we ate donuts. we made five stops on the tour, but this video only covers four of them. we were too stuffed to say anything about japonais, even though the donuts there were delicious. here’s the full lineup: donna’s donuts (yelp!) ziggy’s donuts (yelp!) kane’s donuts (yelp!) sun guang bakery (yelp! how to: plan a donut tour since , the first friday of june has been hailed throughout the us as national donut day. it was founded in recognition of the great comfort donuts provide to those who eat them, and to honor those who serve them. museum of family camping closed memorial day weekend is universally recognized as the start of summer. tradition allows that we can start wearing white, gather family and friends for barbecue, and, for those so inclined, go camping. for the past many years it’s also been the start of the museum of family camping’s season. the interior displays at the museum of family camping celebrated many generations of camping history. my docent made much of the dingle stick (the vertical stick that holds the cooking tin); good manners demanded they be left at the camp site for the next camper. sandee’s homemade wrapping paper sandee’s been getting into disposable art. first it was her holiday dames on the chalkboard in our kitchen, and more recently she’s been crafting one of a kind wrapping paper. it gets torn up and discarded in just a fraction of the time it takes her to sketch and shade it, but act of creation is what she enjoys. i guess that’s why her favorite artistic endeavor is baking. step by step: turn on the iphone/ipad’s web debugging console you can’t view a web page’s source, and you can’t command+f to search for text on the page, but you sure can get a debugging console to see the errors on the page. here’s how: find and open the settings app [<img src="http://farm .static.flickr.com/ / _ f_m.jpg" alt="start in the settings app" width=" " height=" " />][ ] select safari [<img src="farm -static-flickr-com- _ b af b _m.jpg" alt="safari in the settings app" width=" " height=" " />][ ] scroll down to find the developer option at the bottom [<img src="farm -static-flickr-com- _ fb . ipad + velcro = < http://www.vimeo.com/ huffington post introduces badges and social rewards how do you make news fun? or, how do you make moderating often fractious comments on news stories fun? you follow foursquare’s example and introduce badges: the moderator badge allows you to more actively participate in this process. if you are a level moderator (earned by flagging at least comments that we deleted, with a high ratio of good flags to mistaken ones), your flags now carry five times the weight of a standard flag. mick jagger on the music business mick jagger to bbc: [p]eople only made money out of records for a very, very small time […] if you look at the history of recorded music from to now, there was a year period where artists did very well, but the rest of the time they didn’t. via. remixed: my photo in truthout.org i was happy to see one of my photos used as source material for this illustration in truthout.org’s seven year reality check on the iraq war. will mobile flash be relevant when it finally works? john gruber linked to the sizzle in jeff croft’s post: in the [flashcamp seattle] opening keynote, ryan stewart, a flash platform evangelist at adobe, demoed flash player . running on his nexus one phone. […] here’s what happened: on his mac, ryan pulled up a site called eco zoo. it is, seemingly, a pretty intense example of flash development — full of d rendering, rich interactions, and cute little characters. listening is just the start jeff howe writes: idea jams “allow people to discover the fringe question (or idea, or solution), then tweak it, discuss it and bring the community’s attention to it.” “idea management is really a three-part process,” says bob pearson, who as dell’s former chief of communities and conversation rode heard on ideastorm. “the first is listening. that’s obvious.” the second part, pearson says, was integration, “actually disseminating the best ideas throughout our organization. pearls of wisdom in mail list threads david cloutman on code lib: don’t forget to look at trends outside of “libraryland”. a lot of professional library discussion takes place in an echo chamber, and bad ideas often get repeated and gain credibility as a result. librarians usually overstate the uniqueness of their organizations and professions. when the question, “what are other libraries doing?” arises in addressing a technical problem, don’t be afraid to generalize the question to other types of organizations. respond to your next subpoena like a pro thanks to kathleen seidel, a fellow new hampshire resident and blogger at <neurodiversity.com>, i now have what appears to be a good example of a motion to quash a subpoena (even cooler, she filed it pro se). i’ve also learned that nh is among the states that allows lawyers to issue subpoena in civil cases without prior approval of a judge. take a look and prepare yourself for some law talking. steve jobs on apple vs. adobe and iphone vs. flash steve jobs’ thoughts on flash minces no words in its conclusion: besides the fact that flash is closed and proprietary, has major technical drawbacks, and doesn’t support touch based devices, there is an even more important reason we do not allow flash on iphones, ipods and ipads. we have discussed the downsides of using flash to play video and interactive content from websites, but adobe also wants developers to adopt flash to create apps that run on our mobile devices. blogging in academia a comment in the university of lincoln’s audio production course blog demonstrates the value of public blogging in academia: i am looking forward to beginning this course in september and have been finding these blogs very useful in providing a guide as to what sort of things to expect during my first year. keep up the good work! thanks to joss winn for the tip. ssd mysql performance the above graph and this mysql performance blog story are from last year, but i believe are still relevant and instructive now. sure, the fusionio is faster, but how the hell can you beat a single ssd in terms of price/performance? raid : . transactions per minute per dollar ssd: transactions per minute per dollar fusionio: . transactions per minute per dollar improving p — order posts by last comment date i’m a big fan of the p theme for wordpress. it makes it dead easy anybody familiar with wordpress to host a discussion site and improve collaboration across time and distance. that said, one feature i’d like to see is the ability to order the posts by the last comment date, rather than post date. when we started using p to power a workgroup discussion last year, i wrote a bit of code to sort the posts that way, here’s how: irony: nh liquor commissioner suspected of dui in it was the deputy chief of liquor enforcement. last summer it was the wolfboro police commissioner who was arrested importing pounds of marijuana from canada. this week it’s a liquor commissioner who was stopped on suspicion of dui. i’m a carnie huckster, you know it and i know it, but that’s ok the title is a quote from seth stevenson slate.com piece on pitchman vince offer, where he explains that vince’s “smooth-talking condescension” is the most appropriate sales tactic in today’s cynical world. “jaded consumers expect to get snowed and almost distrust the very pretense of trustworthiness.” the rap chop remix of vince’s slap chop actually ran on tv. three sweet globe images [][ ] hey, it’s [earth day][ ]! [ ]: http://www.flickr.com/photos/kumasawa/ / ““wind andamento” by karen ami (cool globes) by kumasawa, on flickr” [ ]: http://en.wikipedia.org/wiki/earth_day auctions and negotiations: starting price matters via mind hacks: auctions with a low starting price may result in higher final sale prices than those with a high starting price. but negotiations with a high starting price often result in higher final sale prices. cleaning up category relationships in a wordpress scriblio site a few lines of sql i used to clean up a scriblio site. it’s probably useless to anybody but me. i’m not suggesting anybody else use this code, as it will result in changed or deleted data. update the post author for catalog records (identified because they have a specific post meta entry): update wp_ _postmeta join wp_ _posts on wp_ _posts.id = wp_ _postmeta.post_id set post_author = where meta_key = 'scrib_meditor_content' get the categories attached to every catalog record (except the “catalog” category): loading: global warming sure i’m a fan of marilyn monroe, but stéphane massa-bidal’s activist illustration is even hotter. he’s online at rétrofuturs.com. la times on ipad vs kindle the kindle feels like an e-reading device, whereas an ipad feels like reading. from latimes.com via joseph monninger. a few lines of sql: cloning blogs in mu the following sql is what i used to clone the content from one blog in mu to another for testing. it’s probably useless to anybody but me. anybody who can’t figure out from the code that wp_ _posts is the source table and wp_ _posts is the destination probably shouldn’t try to use the code, as data will be lost. clone the content from one mu blog into another: truncate table wp_ _posts; insert into wp_ _posts select * from wp_ _posts; truncate table wp_ _postmeta; insert into wp_ _postmeta select * from wp_ _postmeta; truncate table wp_ _terms; insert into wp_ _terms select * from wp_ _terms; truncate table wp_ _term_taxonomy; insert into wp_ _term_taxonomy select * from wp_ _term_taxonomy; truncate table wp_ _term_relationships; insert into wp_ _term_relationships select * from wp_ _term_relationships; truncate table wp_ _bsuite _search; insert into wp_ _bsuite _search select * from wp_ _bsuite _search; truncate table wp_ _scrib_harvest; clone a few options: solving problems in secret matt blaze computer and information science at university of pennsylvania and blogs about security at exhaustive search. his recent post on mistakes in spying techniques, protocols, and hardware caught my interest: indeed, the recent history of electronic surveillance is a veritable catalog of cautionary tales of technological errors, risks and unintended consequences. sometime mishaps lead to well-publicized violations of the privacy of innocent people. there was, for example, the nsa’s disclosure earlier this year that it had been accidently “over-collecting” the communications of innocent americans. the reward for re-discovering archive collections documentarians spend most of their time digging up materials that few people know exist. they frequent basements and dark storage rooms, endure conversations with crazy collectors, and typically develop vitamin-d deficiency and light sensitivity in search of what they need. their reward for finding the material? a bill from the original creators (the ones who lost and forgot about the work in the first place) for the privilege of using it. iphone use heavy at am, bumps at lunch, peaks at pm via localytics: iphone users generate % more traffic on the weekend than the average weekday. saturday traffic ramps quickly from a morning low at : am to over % of peak usage by : am—and stays near the peak for the rest of the afternoon and evening. by comparison, weekday app usage is more concentrated in the evening with a slow ramp during the working day and a peak at : pm est, when east coast users are at home and west coast users are commuting home. is the filesystem finally dead? from rob foster/nimble design: by releasing the iphone os, apple is putting a bullet in the head of a long standing convention that most folks could do without. he’s talking about the filesystem. user-accessible filesystems, anyway. this isn’t news, i don’t think the newton even had a hidden filesystem, but it hasn’t gotten old yet. my question: when will i finally get a system that cleverly mixes cloud and local storage to give me seamless access to all my photos, videos, music, and email…ever? why php’s regex is slow, and what you can do about it (if you happen to be a committer on the php project) regular expression matching can be simple and fast, by russ cox: perl [and php and others] could not now remove backreference support, of course, but they could employ much faster algorithms when presented with regular expressions that don’t have backreferences. how much faster? about a million times (no, i do not exaggerate). i use a lot of regular expressions, and relatively few of them use backreferences. it’d be worth optimizing. edison phonograph eula think end user license agreements (eulas) are recent inventions? thomas edison used them on his phonograph cylinder at the start of the s. the eula didn’t protect edison from innovations elsewhere; discs quickly beat out cylinders once the patents expired. photo from fouro. college students use, love, are aware of the limitations of wikipedia how often do college students use wikipedia? how today’s college students use wikipedia for course-related research: overall, college students use wikipedia. but, they do so knowing its limitation. they use wikipedia just as most of us do — because it is a quick way to get started and it has some, but not deep, credibility. % of respondents use wikipedia frequently or always, typically at near the beginning at the start of research ( %). drm evils: now comic fodder brad colbow does some good looking design and an occasional comic. he isn’t the first to address drm woes in comic form, but his comic is one more public cry for rationality. and continuing that cry is this from an unnamed source, originally published at geekologie.com. scott smitelli on hacking youtube’s content id drm system scott smitelli uploaded a total of test videos and received content id emails in the name of science: testing youtube’s content id system. he reversed the audio, shifted the pitch, altered the time (without changing pitch), resampled (pitch and time), added noise, messed with the volume, chunked it up into pieces, and fiddled with the stereo fields. in the end, he found both amusing and frustrating results. he did his tests about a year ago. connect-a-desk looks ridiculous (though i may secretly want one) i was about to tell sandee how foolish these people look with their laptops stuck to their torsos, but she hit me with “that looks like something you’d use.” ouch. worse, i’m not sure she’s wrong. double ouch. maybe the company could send me one. then i could have these conflicted feelings for real. social media usage stats retrevo claims to help electronics shoppers decide what to buy, when to buy, and where to buy it,” so their recent survey on social media addition is probably more significant as link bait than as serious research. despite my concerns about confirmation bias, i’m as amused as anybody by the numbers. % of adult respondents say they check or update twitter or facebook before getting out of bed in the morning, a number that rises to % for iphone users of all ages. addressing hateful and libelous internet speech in the post juicy campus era juicy campus is gone, but other sites have taken its place as a hub for anonymous slander around college campuses. intentional or not, the conversation at these sites tends toward abusive, with successive commenters attempting to one-up each other with each insult. students targeted by the abuse and defamation have little easy recourse. some sites allow users to mark comments as offensive, but require membership to do so, and the anonymous nature of the posts limits the real world social group’s opportunity to moderate itself and its members. html media – project hosting on google code i was wondering when somebody was going to do what html media does: html video tags make embedding videos into documents as easy as embedding an image. all it takes is a single tag. unfortunately, not all browsers natively support html video tags. html media is a javascript library that enables tags for clunky browsers. url path bug in wordpress.com video server you’ve got to both respect automattic for releasing their internal code as open source while also giving them a break for not assuring that it works for anybody else. one of their projects, the wordpress.com video server is a sophisticated wordpress plugin that handles video transcoding and offers a bit of a youtube in a box solution for wordpress. the bug i found is that the code assumes wpmu is running in subdomain mode, rather than subdirectory mode. rock out with a cardboard record player http://www.vimeo.com/ the physical, analog nature of vinyl has long appealed to the diy crowd. this cardboard record player capitalizes on that to create a direct mail marketing campaign that people appear to actually enjoy receiving. from the description at agency news: grey vancouver created a portable record player from corrugated cardboard that folds into an envelope. the record can be spun with a pencil and the vibrations go through the needle and produce a recording of a children’s story called “a town that found its sound. the cost of ie’s non-compliance google this month dropped internet explorer support in google apps and youtube, and others are lining up at idroppedie .com. still, even newer versions of ie suffer from poor standards support, and there are doubts about the just announced ie . to put this in perspective, billforbill.com is adding up the costs of all the workarounds that web developers have to go through to make it buggy browser work. after just a few days and only submissions the total is over $ million. wp memcache object cache breaks http head requests i just posted about the following confounding problem to the wp-hackers list: when running wordpress mu (tested in . x and . x) with the memcached object cache active, it refuses to respond to http head requests. the result of this is that head requests to check the mimetype of a linked file (as for setting the enclosure) or size (as the video framework plugin does) fail. curl -i http://url.path returns either an empty result, or (if fronted with varnish) a error. wordpress bug in setup_postdata() wordpress is built around the loop, and all the cool kids are using multiple loops on the same page to show the main post and feature other posts. the problem is: wordpress doesn’t properly reset the $pages global for each post. if the post in main loop (or default query) is paged, then all the other posts will show the same paged content as in the main post. i started a ticket and submitted a patch, but in the meantime you might have to unset( $globals['pages'] ) in your custom loops just before calling the_post(). web vs. native apps one lesson here is that a simple but well-done web app […] can be vastly superior to a full-fledged but terrible iphone application. usability nightmare: the my.sxsw iphone app. consumer society and citizen networks logo consumer society and citizen networks “aims at promoting access of citizens to information on product safety, consumer rights protection, and to results of independent testing, as well as promoting wide public discussion of challenges facing the consumer society in ukraine.” their logo, however, is pure genius: some sketches from logolog showing how it came together: christian madrasas from the march newsletter of the north texas skeptics: in the madrasa, the religious school, i watched and listened as the instructor related his view of the world to the students and the others present. politics, personal relationships, nations, and the physical world were interpreted in the light of the speaker’s religious teachings. hinduism and buddhism were lumped together with that quaintly american religion called new age. pagan symbols invoke demons to do dirty work for cultists, and evolution is the root of much of this evil, the students were told. auto-tune put to better use: news auto-tune has been prettying up vocal tracks for more than a decade now, but applying it to news is simply brilliant. the gregory brothers‘ autotunethenews.com is worth a look. nh’s proud political system a nh house judiciary committee hearing recently made new hampshire famous in boingboing and the huffington post. watch the hearing where the speaker describes sex acts, and take special note of the amazing poker face of the others during the talk. i’ll stop the world and melt with you flickr video watching valentine’s rose fade the georgia o’keefe view, above, or the still life view, below: this isn’t so much about valentine’s day as it is about finally getting setup to do time lapse video like this. more to come at maisonbisson.com/timelapse. valentine’s rose (o’keefe view) flickr video valentine’s rose flickr video what the critics are missing about apple’s ipad it’s doubtful that anybody reading this blog missed the news that apple finally took the wraps off their much rumored tablet: the ipad. trouble is, a bunch of folks seem to be upset about the features and specs, or something that made the buzz machine go meh. it’s just a bigger iphone, complain the privileged tech pundits. they apparently missed the recent pew internet project report on internet usage by demographic. blogging by email wordpress has some simple built-in support for posting by email, but that didn’t stop a couple people from developing plugins that might do better. postie and postmaster both claim to support attached photos (though neither appears to use wp’s built-in media management). but if your goal is to post photos, you might consider posting through flickr. organizational vanity, google alerts, and social engineering as more and more organizations become aware of the need to track their online reputation, more people in those organizations are following google alerts for their organization’s name. that creates a perfect opportunity for scammers to play on that organizational vanity to infect computers used by officers of the organization with malware that can reveal the inner workings of that organization. i’m not exactly sure what clicking the button above does. apple’s netbook a post on thomas fitzgerald.net serves to remind us that apple released their first netbook in : the apple emate : …next time you see people ranting about an apple netbook, remember that apple had something similar long before anyone even uttered the phrase “netbook.” the device ran netwon os with a - hour battery life (yes, - hours). i’ve written more than a few posts eulogizing the emate’s tablet-shaped sibling: newton message pad . coda feature wishlist i’d long been a user of barebones’ bbedit, a product that’s served me well for a number of years. but upgrading from version . to is a paid deal, and after spending days with the demo of bbedit , i decided i wanted to look around a little bit. my friend matt switched from bbedit to panic’s coda some time ago, and i liked the demo of that well enough that i bought a license. put an ssd in your expresscard slot? i spied the wintec filemate gb ultra expresscard and began to wonder how it works as a boot drive for mac os x in a late macbook pro (the model just before apple replaced the expresscard slot with an sd slot). but i didn’t have to wonder too much, as a post to this macobserver forum thread offers enough details to make a geek salivate: the computer now boots primarily from the ssd card and will start up the computer in less than / the time of the internal hd […] i have all the applications and system files on the ssd card, the user files/record on the internal hd. do e-books have a future? david weinberger kicked off the latest installment in the ongoing debate about the future of electronic books versus paper books in his will books survive? a scorecard… post. he’s got some good points, but like many of the smart folks i admire, he approaches this question assuming that books, in any form, are important. ursula k. le guin’s excellent essay on “the alleged decline of reading” is especially informative on this point: books don’t matter to most americans, and they haven’t for some time. even if they don’t click ethan zuckerman’s recent post, what if they stop clicking? points out the difficulty of building a business on ad revenue. he points to statistics that show fewer readers are clicking banner and arguments from the web advertising industry about how un-clicked ads still build brand awareness. it’s not really central to zuckerman’s point, but i didn’t sense that he was aware that google has picked up the same argument. i commented on the post that google has started reporting the numbers of people who are presented (but don’t click) ads, then later visit the advertisers that are paying for, um, clicks. my wordcamp nyc talks authentication hacks my first talk was on user authentication with mu in existing ecosystems, all about integrating wp with ldap/ad/cas and other directory authentication schemes, as well as the hacks i did to make that integration bi-directional and deliver new user features. my slides are online (.mov / .pdf), and you can read earlier blog post summing up the project. plugins mentioned wpcas (long description) alternate contact info wordpress ticket framework wpsms (long description) scriblio i was most excited, however, to talk about scriblio, a plugin that turns wordpress into a library catalog with faceted searching and browsing. spell checking matt demanded accent-aware spell checking for the wordpress spell checking plugin his company acquired earlier this year. and just a little more than a month later, after the deadline delivered. now beyoncé, café, coöperate, and even my resumé look prettier. separately, wordnik offers a new take on online dictionaries, and they just launched an api. backblaze storage pod backblaze is a cloud backup service that needs cheap storage. lots of it. they say a petabyte worth of raw drives runs under $ , , but buying that much storage in products from major vendors easily costs over $ , , . so they built their own. the result is a u rack-mounted linux-based server that contains terabytes at a material cost of $ , , the bulk of which goes to purchase the drives themselves. drobo: sweet storage, one big flaw i’ve been a fan of drobo since i got mine over a year ago. the little(-ish, and sweet looking, for stack of disks) device packs as many as four drives and automatically manages them to ensure the reliability of your data and easy expandability of the storage. however, thomas tomchak just pointed out one major flaw: if you overflow your drobo with data, the entire device may give up and you’ll lose everything. the bugs that haunt me a few years ago i found an article pointing out how spammers had figured out how to abuse some code i wrote back in or so. i’d put it on the list to fix and even started a blog post so that i could take my lumps publicly. now i’ve rediscovered that draft post…and that i never fixed the bad code it had fingered. worse, i’m no longer in a position to change the code. ssh tunneling examples most of my work is available publicly, but some development is hosted on a private svn that’s hidden behind a firewall. unfortunately, my primary development server is on the wrong side of that particular firewall, so i use the following command to bridge the gap: ssh -r :svn_host: username@dev_server.com that creates a reverse tunnel through my laptop to the svn server and allows me to checkout code using the following: yelp: a poster child for semantic markup search engine land.com: yelp…is…essentially a poster-child for semantic markup. this spring, google’s introduction of rich snippets has allowed yelp’s listings in the serps to stand out more, attracting consumers to click more due to the “bling” decorating the listings in the form of the star ratings. there are now some very good reasons why sites with ratings and reviews should be adopting microformats, and it’s not that hard to do! iphone’s anti-customer config file in march of this year apple applied for a patent on technology that enables or disables features of a phone via a config file. the tech is already in use: it’s the carrier profiles we’ve been downloading recently. on the one hand this is just an extension of the parental controls that apple has included in mac os x since the early days, but it also implies some rather anti-consumer thinking at the company. evil evil klaomta.com a quick google search of klaomta.com reveals more than a few people wondering why it’s iframed on their websites. the answer is that the site has been compromised. unfortunately for the fellow who asked me the question at wordcamp, solving the problem can be a bit of a chore. keeping your wordpress installation up to date is important, as there are some known security flaws in older versions, but most of the attacks that crackers use are targeted elsewhere. the wordpress way plugin development will norris‘ talk at wordcamp pdx introduces wordpress coding standards, common functions, and constants to would be plugin developers (and smacks those who’ve already done it wrong). also notable: functions, classes, variables, and constants in the wordpress trunk. custom installations just as wordpress has a number of hooks and filters that plugins can use to modify and extend behavior, it also has a cool way to customize the installation process. hacking wordpress login and password reset processes for my university environment any university worth the title is likely to have a very mixed identity environment. at plymouth state university we’ve been pursuing a strategy of unifying identity and offering single sign-on to web services, but an inventory last year still revealed a great number of systems not integrated with either our single sign-on (authn) or authorization systems (authz, see difference). and in addition to the many application/system specific stores of identity information (even for those systems integrated into our single sign-on environment), we also use both ldap and ad (which we try to synchronize at the application level). worst of all, the entire environment is provisioned solely from our mis database, which is good if you want to make sure that students and faculty get user accounts, but bad if you want to provision an account for somebody who doesn’t fit into one of those roles. the one way relationship between our user accounts and the mis database also makes it difficult to engage with new users online. if you can’t get an account until you become a student, how do you allow potential students to apply online if all your systems are integrated with single sign-on? and if you can’t authenticate the online identity of your users, how do you set initial passwords into your system? or allow them to reset a forgotten password online? internet companies never struggled with this issue, as their customers could only approach them online, but most universities built systems around paper applications and have fond (and relatively recent) memories of offering their students their first internet experience. it’s still not unusual for universities to offer their students their campus computing account with a default password based on supposedly secret data shared between the user and the school. but your ssn, birth date, and mother’s name are no longer secret. a proposed change in ferpa policy (see the the top of page in the nprm) would have barred the use of “a common form user name (e.g., last name and first name initial) with date of birth or ssn, or a portion of the ssn, as an initial password to be changed upon first use of the system” in systems that store academic data. the final rule excluded that provision, much to the relief of those schools with more lobbying clout than brains. pigeon beats adsl: slow networks or massive storage capacity? it was a tech story so apparently humorous that the popular media felt compelled to cover it: carrier pigeons delivered gbs of data faster than an adsl line. the bbc story’s subtitle read “broadband promised to unite the world with super-fast data delivery – but in south africa it seems the web is still no faster than a humble pigeon,” and that’s how most stories played it. unfortunately, they all got it wrong. moving data by homing pigeon requires some planning, and pigeons. source. the race was run by the unlimited group, but the clearest telling of it comes from wikipedia: inspired by rfc {.external.mw-magiclink-rfc}, on september the marketing team of the unlimited, a regional company in south africa, decided to host a tongue-in-cheek “pigeon race” between their pet pigeon “winston” and local telecom company telkom sa. the race is to send gigabytes of data from howick to hillcrest, approximately km apart. the pigeon carrying a microsd{.mw-redirect} card (an avian variant of a sneakernet), versus a telkom adsl{.mw-redirect} line. winston beat the data transfer over telkom’s adsl line, with a total time of two hours, six minutes and seconds from uploading data on the microsd card to completion of download from card. at the time of winston’s victory, the gb adsl transfer was just under % complete. jsnes: javascript nintendo emulator ben fisherman’s jsnes runs entirely in the browser using nothing more intrusive than javascript. it apparently manages real-time performance within chrome, but it works (if not playably) on an iphone. i wish the screen was resizable and that it supported iphone compatible controls, but both of those assume that browser performance will improve enough to make it playable. interestingly, though not surprisingly, the safari js engine is limited to consuming a single cpu (which it quickly does while playing jsnes). itunes : closer to an api? will norris has discovered that itunes ’s interactions with the store are more web-happy. i’ve been asking where the itunes store api was for some time, now i think i’ve got what i need to build one. wordpress hacks: nested paths for wpmu blogs situation: you’ve got wordpress multi-user setup to host one or more domains in sub-directory mode (as in site.org/blogname), but you want a deeper directory structure than wpmu allows…something like the following examples, perhaps: site.org/blogname site.org/departments/blogname site.org/departments/blogname site.org/services/blogname the association between blog ids and sub-directory paths is determined in wpmu-settings.php, but the code there knows nothing about nested paths. so a person planning to use wordpress mu as a cms must either flatten his/her information architecture, or do some hacking. am i supposed to feel bad for at&t now? with at&t facing lawsuits for not delivering mms features at the iphone gs launch, they kind of had to do something. i’m not sure if i’d be satisfied by this video if i were among the plaintiffs, but i think it does a good enough job. the stat about % annual increases in mobile data use is pretty powerful. i’d heard it a dozen times before*, but because i wasn’t in austin for sxsw iphone meltdown, i don’t have quite the same appreciation as some do. at&t added capacity then, and they seem to have been scrambling elsewhere too. iphone users are said to be six times as likely as anybody else to watch video on their phones, and if wifi aggregator jiwire’s report says anything about cell data, the iphone has certainly changed the game. jiwire’s mobile audience insights report shows that over % of the devices on their network are either iphones (about % of the total) or ipod touches! and all the way back in in britain, iphone users were times as likely as other phone users to send or receive more than mb a month. it will be interesting to see what happens to other carriers as they get devices that encourage use as the iphone has. *actually, i hadn’t heard the % stat specifically, just inspecific reports of increased usage. now i want to watch (or re-watch) all these okay, i don’t want to watch all the movies depicted in this year overview of film special effects, but i did just add a few to my netflix queue. wordpress hacks: serving multiple domains situation: using wordpress mu (possibly including buddypress) on multiple domains or sub-domains of a large organization with lots of users. wordpress mu is a solid cms to support a large organization. each individual blog has its own place in the organization’s url scheme (www.site.org/blogname), and each blog can have its own administrators and other users. groups of blogs in wpmu make up a “site” and one or more sites can be hosted with a single implementation. (i’m capitalizing site for the same reason wordpress docs capitalize page) each site has a defined set of administrators and options controlling various features. you might, for instance, lock down the plugins on your blogs.site.org, while keeping it open on your www.site.org. or maybe you’d like to let your helpdesk staff create new blogs at blogs.site.org, but not at www.site.org. that’s what wpmu’s notion of site can help you control. online advertising metrics i don’t know if it’s just the mother’s day effect, but the top online retailers for may were dominated by flower shops. the top shop is converting almost % of their visitors to buyers, though the average is just over %. tim, meanwhile, claims he’s lowered his bounce rate to just %. not my chair, not my problem liam lynch explains the origin of the video, but what was dan deacon thinking as he [recorded the audio][ ]? of all the [free mp downloads][ ] he offers, [two friends][ ] from the acorn master album may be the most, um, listenable. thanks to [daily songsmith corey b (corey blanchette)][ ] for the tip. [ ]: www-dandeacon-com- drinking out of cups.mp [ ]: http://www.dandeacon.com/mp / [ ]: www-dandeacon-com- _two_friends.mp [ ]: http://coreyb .com/ who gets to control the future of libraries? the following was my email response to a thread on the web lib mail list: okay, it must be said: you’re all wrong . i can understand that news of a librarian being fired/furloughed will raise our defenses, but that’s no excuse for giving up the considered and critical thinking that this occasion demands. consider this: the principle’s blog reveals a reasonable person actively trying to improve academic performance despite crushing economic conditions. martin belam’s advice to hackers at the guardian’s july hack day an amusing hacks-conference lightning talk-turned-blog post on web development: “graceful hacks” – ux, ia and interaction design tips for hack days. martin belam‘s talk at the guardian’s july hack day must have been both funny and useful: funny: “however, i am given to understand that this is now deprecated and has gone out of fashion.” useful: “the yahoo! design pattern library is your friend.” hnews might not be so bad the ap’s diagram of their protect, point, pay “news drm” scheme looked like a joke, then i saw the parody. despite all the smoke and hype, ed felton explains that it’s underwhelming, at most. still, hnews might be an interesting format for some blogs to adopt. most of what the ap is rattling their saber about is in the rights (containing ccrel declarations). felton thinks the dependence on ccrel may extend derivative usage rights, rather than limit them. get your beer pong skills on do facebook ads work? all facebook is happy to share the ten laws of facebook advertising, but will those rules lead to better results than the . % ctr bob gilbreath got a year ago? newspaper business: news was a loss leader howard weaver wants newspapers to play offense against google and others, but chris tolles, ceo of news aggregator topix.com says he’s been trying weaver’s plan for a while, and there’s no bucket of gold to be found in it. the problem, it would appear, is that newspapers don’t sell news. they sell advertising space and pair it with news as a loss leader to keep the eyeballs. and while that worked in print, it doesn’t work on the web. google recommends microformats and rdfa google’s own webmasters help site recommends microformats and rdfa structured data to improve indexing and usefulness of the data. review metadata appears to have full support, while people, product, and business data are in beta. do air taxis actually work? i just thought to follow up on this story about dayjet, a high-flying air taxi service that planned to operate tiny, three-passenger eclipse jets. the story doesn’t deviate from economic trends: dayjet ceased operations in september , and the aircraft manufacturer entered chapter in february . the air taxi association says their operators save big money over scheduled airline service, but finding the price of that service can be hard. mozilla labs’ ubiquity http://www.vimeo.com/ mozilla labs’ ubiquity has a lot of promise: ubiquity is an experiment into connecting the web with language in an attempt to find new user interfaces that make it possible for everyone to do common web tasks more quickly and easily. it’s a firefox extension, so it works on macs, windows, and linux. with only a couple keystrokes, it lets you use language to instruct your browser. you can translate to and from most languages, add maps to your email, edit any page, twitter, check your calendar, search, email your friends, and much more. tomas mankovsky’s sorry i’m late http://www.vimeo.com/ i’m simply in love with this video. watch through the credits to see a bit of how it’s made. go blog, small orgs (or large) philip greenspun suggests small organizations use a blog for their website (ironically, not blogged): the small business web circa in , a small organization that wanted a web site would hire a “web designer” skilled in the exotic art of “html programming” to produce a static web site, i.e., a cluster of linked pages with a distinctive design and color scheme, giving information about the company or non-profit org. get the zimbra isync connector it can be difficult to get the zimbra isync connector, as the company doesn’t offer a simple download from their site. fortunately, the license allows us to freely redistribute their software. download the zimbra isync connector here. what is david mcnicol’s url cache plugin? the description to david mcnicol’s url cache plugin raises more questions than it answers: given a url, the url_cache() function will attempt to download the file it represents and return a url pointing to this locally cached version. where did he plan to use it? does he envision the cache as an archive, or for performance? why hasn’t it been updated since ? it caught my interest because i’ve long been interested in a solution to link rot in my blog. book search results vs. users bret victor offers the above design suggestions (from ) to amazon in the book search results display (he’s comparing to this). i didn’t discover them at the time, but many of them are still relevant now. bret notes that amazon’s display doesn’t do a good job of answering the questions a person has when searching for books: “what is the book about?” and “is it any good?” unfortunately, these questions are completely unaddressed by the information provided. too bad the hanzo archives wordpress plugin is caput the hanzo archives wordpress plugin is something i’d be very excited to use. ironically, it’s disappeared from the web (though the blog post hasn’t): we’ve released a wordpress plugin which automatically archives anything you link to in your blog posts; it also adds a ‘perma-permalink’ for the archived version adjacent to each original link. an amazon web services case study put me on to hanzo a while ago, and in may i actually spoke with mark middleton (the markm who posted the entry above). customizable post listings lorelle is a big fan of scott reilly’s customizable post listings: display recent posts, recently commented posts, recently modified posts, random posts, and other post, page, or draft listings using the post information of your choosing in an easily customizable manner. you can narrow post searches by specifying categories and/or authors, among other things. using vlc as a live video stream transcoder for axis camera and flv [i]n theory, i should be able to issue one command to vlc and have it receive the mpeg -es stream from the camera, transcode it to h. , and stream it to the wowza, which would handle the rest. via john beales. leaked video of bumblebee’s breakdance moves http://www.vimeo.com/ well, not ‘leaked,’ but just in time for the new transformers movie, patrick boivin has posted this video of bumblebee breakdancing. video or audio comments in wordpress with riffly in line with yesterday’s discovery of the viddler wp plugin, riffly webcam video comments also supports video or audio comments within wordpress: riffly is a free service that easily plugs into your site allowing visitors to create video and audio comments. the service is advertising supported. we cover all the costs for bandwidth, servers, and maintenance. optionally, we also offer premium riffly accounts that provide you with additional benefits, such as advertising removal, control panel access, analytics, and much more. video comments with viddler wordpress plugin the viddler wordpress plugin promises to “enrich your site’s commenting experience by enabling video comments….” users can record direcly from a web cam or choose a video they’ve previously uploaded to viddler.com. viddler evangelist colin devroe has it on his site, where i can see it requires would-be commenters have a viddler account. that last bit is too bad. i like viddler, but i can’t force my readers to like it and get accounts as a prerequisite to commenting. wolfram|alpha’s missing feature: libraries john timmer brings up my two biggest complaints about wolfram|alpha. the first is that it’s even harder to identify the source of information than it is in wikipedia, the other is what happens when searches fail: a bad web search typically brings up results that help you refine your search terms; a bad alpha search returns nothing, and it’s not clear that there’s an easy way to fix that. systems wrangling session at wordcamp developer day what is the current status of web servers…is apache .x “fast enough?” automattic uses lightspeed (for php), nginx (for static content), and apache (for media uploads). for wordpress-generated content, all server options are approximately the same speed. what about apc? automattic uses beta versions of apc, and provides a - x performance increase. it’s tied closely to the php version, so automattic recently switched from php to php . databases? andy peatling on buddypress why buddypress? “build passionate users around a specific niche.” do you have to become a social network? “no, look at gigaom pro,” a recently launched subscription research site based on buddypress. but, yo do get “byotos: bring your own terms of service.” that is, you get to control content and interactions. and your service won’t be subject to the whims of a larger network like facebook (or vagaries of their service — think ma. wordpress . script handling jquery . . is in wordpress . , but the most exciting changes are in the automatic concatenation and compression of scripts via the script loader. andrew ozz says “this feature can easily be extended to include scripts added by plugins and to use server side caching, however that would require some changes to the server settings (.htaccess on apache).” i have yet to figure out how to extend that feature to scripts in my plugins, but i’m working on it. google’s matt cutts on building better sites with wordpress % of wordpress blogs he sees are spam. but for those who aren’t spammers and want to do better in google…. “wordpress automatically solves a ton of seo issues…wordpress takes care of - % of seo.” still, he recommends a few extra plugins: akismet — reduce spam comments cookies for comments — reduce spam comments feedburner feedsmith wp super cache — improve performance “we crawl roughly in order of pagerank…higher ranked sites get crawled faster and deeper. understanding, leveraging google image search above is [peter linsley][ ] speaking about google image search at [smx west][ ] in february, . meanwhile, [stefan juhl suggests some javascript][ ] to break your site out of the image search result pages: many google image search users are quickly clicking on to the direct image url and thereby not seeing the page with the image. also, it seems that many of the users don’t hesitate to click back to the image serps when they don’t see the image “above the fold” – probably because of google image search framing the page with the picture and thus making it almost too easy to do so. on the one hand he wants to catapult chicken droppings, on the other hand he did catapult his wife; repeatedly the homeland security press is just getting wind of joe weston-webb’s attempts to deter vandals with nonlethal weapons, but the story became all the rage in britain when it broke last year. the stories hit all the timely bits: joe got burgled, so he announced plans to install a catapult. a what? a catapult. why? to launch chicken droppings at miscreants. unfortunately, the local constabulary warned him off, and the catapult wasn’t ready when burglars returned. zen and the art of motorcycle maintenance is available all over the web robert m. pirsig‘s zen and the art of motorcycle maintenance at amazon, a used book store, or your parent’s book shelf. still, it’s available on the web as pdf, at least two text files — one, two — and even as a podcast (subscribe via itunes). lots of people have re-traced the journey described in the book, at least one person has posted a travelogue about it to the web. henry gurr has posted pirsig’s own photos, and christoph bartneck pointed out many locations in google maps: is mysql . ready? mysql . hasn’t gotten a lot of love, but it does introduce support for pluggable storage engines. and that’s required to use sphinxse. sphinx is a fast full text search engine. it doesn’t need to run as a mysql storage engine to work, but doing that allows joining against other mysql tables. so while i’m watching the future of mysql alternatives, i’m also watching . bug fixes and playing with the coolstack-packaged . extreme sheep herding iphone g camera hacks and deets those unwilling to open of their iphone to adjust the camera focus might take a look at griffin’s clarifi, a case with a built-in close-up lens that can slide in our out of place as needed. flickr user meine ideenecke, meanwhile, has figured out the iphone camera specifications. he says it’s about mm ( mm equivalent), though this source says it’s mm. will tuneup fix my collection of podcast music downloads? now that i’ve discovered it, i’m tempted to try tuneup on my collection of mp s downloaded as podcasts (and without good id tags) from places like the kcrw’s today’s top tune. the story is that the itunes plugin automatically identifies your tracks, can fix the tags, and add album art. google street view camera sightings what happens when one of google’s street view camera vehicles encounters a low bridge or a muddy australian road? comparing panorama stitching tools the above are the result of panolab, hugin, calico, and a single shot with a very wide angle lens (canon’s - mm, effectively mm on my rebel xti). the first three originated on my iphone and the panolab shot was stitched and originally uploaded to flickr on my iphone (though i have since done some color enhancement and reuploaded the photo from my macbook pro). hugin is gpl, the other solutions are less free (in both senses). the difference between mysql’s utf _unicode_ci and. utf _general_ci collations mysql answer: utf _unicode_ci vs. utf _general_ci. collation controls sorting behavior. unicode rationalizes the character set, but doesn’t, on it’s own, rationalize sorting behavior for all the various languages it supports. utf _general_ci (ci = case insensitive) is apparently a bit faster, but sloppier, and only appropriate for english language data sets. the many uses of a pockettorch doesn’t everybody need a pockettorch? it’s a “safe, practical tool,” they say. more amusingly, the list of suggested uses includes: melting your cache of gold, scaring grandma, lighting illegal fireworks, dental/lab work, and making friends jealous. fun threads for librarians who doesn’t want to be an anarchist librarian? or a bibliophian? photoshop retouching magic vs. disasters compare the retouching portfolio here against the regular posts at photoshop disasters. lessons learned: why it’s better not to use parentheses when they’re optional there it is in the php manual for return(): note: since return() is a language construct and not a function, the parentheses surrounding its arguments are not required. it is common to leave them out, and you actually should do so as php has less work to do in this case. i knew the parentheses were optional, but i’ve been merrily using them all along. and i probably would have continued doing so until i saw the second note attached to the docs: mysql correlated subqueries correlated subqueries are said to be “inefficient and likely to be slow,” but that doesn’t mean i’m not glad to have learned of them. what is an archive in the digital age? jessamyn pointed out the dust up over the dissapearing of paperofrecord.com, a historical newspaper archive. most annoying song ever? is this the most annoying song ever? independent + catchy and pop gone wrong. how much do you want that job? one of the many odd questions this prank job application asks is: what are you willing to wear at work? (check all that apply) paper hat tie hairnet spandex singing omelet costume sweet vw bus scooter sidecar i spied this drool-worthy scooter and sidecar combo on scooter sidecars. wordpress action ticketing api this plugin is the next step after my proposal for a common invite api. here’s how i described it when requesting hosting at the plugin directory: a common framework for registering tickets that will be acted upon later. use it to manage challenge/response interactions to confirm email addresses, phone numbers, im screen names, twitter accounts, etc. build an invite system around it, or use it as the foundation of a short url system. you think you’re paying too much for mobile data? a caller to clark howard’s cnn show complains of being billed $ , by his cell phone provider for data usage. and oklahoman billie parks has filed suit over a $ , bill. saving objects in wordpress’ user meta there’s a hole in the wall at about head level next to my desk. i’ve spent most of the day trying to track down a bug with some code i’ve been working on to add fields to a user’s profile in wordpress. the problem is that upon trying to save the profile i’d get an error like the following: catchable fatal error: object of class stdclass could not be converted to string in /wp-includes/wp-db. expandrive ftp/sftp/amazon s client expandrive makes ftp, sftp, and amazon s connectivity dead easy. expandrive acts just like a usb drive plugged into your mac. open, edit, and save files to remote computers from within your favorite programs—even when they are on a server half a world away. expandrive enhances every single application on your computer by transparently connecting it to remote data. php magic constants: __line__, __file__, __dir__, __function__, __class__, __method__, and __namespace__ i’ve been using __file__ for years, but i never thought to look for its siblings. echo ' line:'. __line__ .' file:'. __file__ .' directory:'. __dir__ .' function:'. __function__ .' class:'. __class__ .' method:'. __method__ .' namespace:'. __namespace__; i feel as though i should have noticed these earlier; they’re clearly referenced in the docs for debug_backtrace(), after all. down the drain: flowers in the in-sink-erator flickr video i can’t explain my fascination with putting flowers into the in-sink-erator, but the sink does smell like flowers afterwards. music is evil by beads. trash fiction book covers a while ago i discovered a great collection of scanned book covers from s-ish pulp fiction in flickr. i had gone looking for things to post on our clipboard wall, but these are too fun to walk away from — especially now that sandee’s put cats up. marc acito on strunk and white’s elements of style when it comes to “shall” and “will,” strunk and white gives the following example: “a swimmer in distress cries, ‘i shall drown; no one will save me!’ ” but a suicide says, “i will drown; no one shall save me!” and i say, “you two (pedantic) know-it-alls deserve to drown.” i mean, what about “help!” via who needs a manual to write real good?. yahoo! bids adieu to yahoo! has divested itself of blo.gs and is shuttering geocities. would this have happened in a good economy? no. did it need to happen anyway? yes. yes. yes. and for the love of god, yes. tips to publishers from google news it turns out that there are a lot of differences between google’s regular web crawler and the google news crawler. and though very few of us will find our content included in google news, it still seems like a good idea to make our content conform to their technical requirements. here are a few of them: in order for our crawler to correctly gather your content, each article needs to link to a page dedicated solely to that article. correction: i do still need the wufoo forms wordpress embed shortcode a few weeks ago i said i no longer needed the wufoo embedding code that i’d put into bsuite. i was wrong. so i’ve taken another look, fixed the code from my old post, and coded it up into a stand-alone plugin. i’ve added installation and usage instructions to the bottom of the original post. what’s the best panorama stitching app for iphone? i spent some time looking for panorama-related apps for the iphone and came up with the following: panorama by airshed panoramas by helix interactive tripstitch by byteslice software pano by debacle software panoramascope by phil endicott panolab and panolab pro by originate lab i’ve actually played with panolab a bit (landscape, portrait) after seeing p ps harlow using it. fixing batcache to send the correct content-type header i’m a fan of batcache, the memcached-based wordpress full-page cache solution, but i’ve discovered that it ignores the content-type header set when the page is initially generated and re-sends all content with content-type: text/html. i posted a note about this at the wordpress support forums, but then i realized what the problem was: apache_response_headers() doesn’t return the content type, but headers_list() does. the solution is to replace apache_response_headers() with headers_list() in the code, though headers_list() is php + only, so it might be a while before we see a change like this committed. facebook’s favorite metadata [facebook’s guide to sharing][ ] details some meta tags to make that sharing work better: in order to make sure that the preview is always correctly populated, you should add the tags shown below to your html. an example news story could have the following: > > as shown, title contains the preview title, description contains the preview summary and image_src contains the preview image. please make sure that none of the content fields contain any html markup because it will be stripped out. google labs: similar images and news timeline new releases from google labs: similar images and news timeline. i count it as a failure for google that the news timeline doesn’t show future events. three or more ways to record or intercept voip calls voip now offers a few tips, hackszine discusses voipong, and mac voip mentions cain & abel and describes arp poisoning to make a man-in-the-middle intercept. jeeves is back! does your organization need its own avatar/personality? if you remember ask.com, you probably remember jeeves. now he’s back on the uk site. it turns out that people liked the old chap, and in this age of social media, it’s probably prudent to have a corporate avatar (it looks a lot better on facebook, anyway). there’s more about the resurrection at search engine land. flight level , pvd kent wien‘s photo of providence, rhode island is better than average for the camera out the window genre. william shatner’s rocketman still makes me laugh elton john and bernie taupin wrote it, but william shatner did it best. watch the video now and download the mp for future enjoyment. thanks to vasken for pointing out the video. do we need a wordpress common invite or challenge-response api? the buddypress forums have a number of threads about handling invitations (two worth looking at: one, two), but no real solution has emerged. at the same time, there’s also a need for some means of confirming other actions such as password resets, email changes (both of those are already handled by wpmu, i know), cell phone numbers to receive sms messages, and other actions that need to be confirmed later. fixing user meta to accept repeating fields — just in time for the wordpress has-patch marathon there’s a wordpress has-patch marathon going on now and i’m hoping one of my recent patches gets some attention. i’m hoping to fix the user meta functions to allow them to accept multiple values per key, per user. it’s listed there among the other has-patch tickets in trac, and there’s been some discussion in wp-hackers. why not take a look? wifi is critical to academia, the wifi alliance says a study sponsored by the wifi alliance reveals the following: wifi and college choice % of college students say wi-fi access is as essential to education as classrooms and computers % say they wouldn’t go to a college that doesn’t have free wi-fi % say that without wi-fi access, college would be a lot harder % agree that widely available wi-fi on campus is an indication that a school cares about its students wifi and where they use it % have connected from coffee shops and restaurants % from parks % from in their cars wifi in the classroom % have checked facebook™ or myspace™ and sent or received e-mail while using their laptop in class % have sent instant messages to a friend during class % used wi-fi to get a head start on an assignment before a class was finished wifi and linkbaiting statistics if forced to choose, % would give up beer before giving up wi-fi survey methodology: “in conjunction with the wi-fi alliance, wakefield research surveyed u. globesurfer x- wireless broadband router option globesurfer x• router: “a new product that transforms any usb wireless modem into an instant internet-connected wifi network capable of supporting multiple users.” too bad i can’t figure out where to buy it. also too bad that i can’t simply do this with a jail-broked iphone. i mean, doesn’t an iphone have everything it needs built-in: a cell-phone modem, wifi hardware, and enough unixy goodness to support nat and routing? bumptop: taking the desktop metaphor deeper bumptop: a fun, intuitive d desktop that keeps you organized and makes you more productive. like a real desk, but better. your desktop doesn’t have to be a boring graveyard for lost and forgotten files anymore! transform it with bumptop. create the desktop that suits your needs and style. recently reviewed in arstechnica. extracting/decompressing .rar files on mac os x mac os x doesn’t ship with unrar, the common linux utility, but you can easily get it bundled in unrarx, a convenient mac os x utility. dig around and you’ll find it in unrarx.app/contents/resources. not sure that rev=“canonical” is really the solution anything that can help stop this kind of madness is worth a good long look (yes, i don’t like the diggbar any more than john gruber, despite digg’s assurances it’s safe), so i’ve had rev=“canonical” on my mind (yes, that’s rev, not rel). chris shiflett thinks it will save the internet, but matt cutts suggests what i’ve always thought: why not resolve short urls to their long form and store/display them that way? cas is a standard protocol, not a standard application i’m not really part of the jasig cas community (learn more), but i do maintain the wpcas wordpress cas client and i’ve started development of a cas server component for wordpress. that project is on hold because one of the products that i’d expected to integrate with it doesn’t use standard cas and the vendor of that app has chosen to modify the jasig cas server to support their apps. weird screw drive russian truck can we stop complaining about taxes already? andrew tobias asks if we can finally put the tax argument to bed: is the reason you’re not investing in stocks these days (a) the prospect of having to pay % capital gains tax? or (b) the fear of further losses? (well, or – c – that you don’t have any money?) is the reason you don’t start a new business that (a) if it made you a lot of money you’d have to pay a lot of taxes? sniff sniff — network sniffing in mac os x adam had to remind me of this: sudo tcpdump -i en -s port of course tcpdump can only tell us what other machines the computer is talking to, not what the conversation is. that requires a sniffer like wireshark. iphone earbud + business card hacks: speakers and cord winder two interesting submissions to the core business card hacks challenge: earbud speakers and a cord winder. you’re nobody unless you’re fake — on twitter here’s a simple way to tell whether the star you’re following is the real thing. are the alleged celebrity’s tweets funny and entertaining, with a palpable sense of self-awareness and wit? full on fake then, and by default, well worth following. oh, and twitter, if you’re still confused, the fake celebs are the ones who cannot afford a publicist to announce that the @fakeaccount everyone’s following isn’t really them. damn firewalls…but which firewall? for some reason two cdns, bitgravity and castfire, are being blocked on campus. you might think firewall, but the problem even seems to appear outside the firewall. international pillow fight day world pillow fight day in boston last saturday was not only a lot of spring fever fun, it also resulted in a marriage proposal. banditos misteriosos estimates there were over , pillow fighters, apparently making it one of the largest fights that day. view the above panorama large to see the crowd. detroit police shut down the fight there, confiscating pillows and demanding permits, though the calgary fight went without incident, despite concerns about permits. adventure cameras: olympus vs. panasonic i’ve been keeping my eye on the olympus stylus tough- . it’s reportedly durable and waterproof to feet. but i’ve just discovered the panasonic lumix dmc-ts , also supposedly tough and waterproof (though only to feet). the panasonic, however, can shoot hd video and has a higher maximum iso. the panasonic also does some funky facial recognition (which favors recognized faces when focusing), but the olympus can stitch multiple-shot panoramas in the camera and has “tap control” that allows, well, it appears to allow you to control the camera’s settings by tapping the sides rather than fiddling with buttons. things learned from the durex sexual wellbeing survey yes, they did a survey, and the results show the french have plenty of sex, but are among the least satisfied for all that activity. russians ( %), brazilians ( %), and greeks ( %) appear to be the most likely to get it at least once a week, while in japan it appears both infrequent and unsatisfying. new zealand distinguished itself for being the only country where women averaged more partners than men. we were warned about this… years ago fortune magazine, march , : like alligators in a swamp, financial derivatives lurk in the global economy. deriving their value from the worth of some underlying asset, like currencies or equities, these potentially lucrative contracts are measured in trillions of dollars. but they also lie in convoluted layers in a tightly wound market of global interconnections. and that gives them the capacity to bring on a worldwide financial quake. new plymouth state university mascot matt worked this up for our university portal today. plymouth has long been the panthers, but a little change does the university good. panthers may have paws, but platypi have venom. crime vs. highways. or, internet security is a social (not technical) problem stefan savage, speaking in a segment on march ’s on the media, asked: the question i like to ask people is, what are you going to do to the highway system to reduce crime. and when you put it that way, it sounds absolutely ridiculous, because while criminals do use the highway, no rational person is suggesting that if only we could change the transportation architecture that crime would go away. mm f/ . the canon mm f . is the stuff of legend. sure it wasn’t particularly sharp, and depth of field was so short that you’re unlikely to get an entire face in focus, but the notion of a lens that bright is more than a little attractive (even if you’re unlikely to have enough light to focus at all if you’re in a situation where you need the f . maximum aperture). php icalendar php icalendar can parse and render ical formatted files. apple’s developer docs, amusingly enough, offer a few more hints along those lines. wufoo forms wordpress embed shortcode i tossed this together a while ago, and it even made it in to bsuite for a time, but i don’t have a need for it anymore, and i’m cleaning house. function shortcode_wufoo( $arg ){ // [wufoo id=z x m domain=place.wufoo.com] $arg = shortcode_atts( array( 'id' => false, 'domain' => false, 'height' => , ), $arg ); if( !$arg['id'] || !$arg['domain'] ) return( false ); return( str_replace( array( '%%id%%','%%domain%%','%%height%%' ), array( $arg['id'], $arg['domain'], $arg['height'] ), '<iframe height="%%height%%" allowtransparency="true" frameborder=" " scrolling="no" style="width: %; border:none" src="https://%%domain%%/embed/%%id%%/"><a href="http://%%domain%%/forms/%%id%%/">fill out my wufoo form! jellyfish at the monterey bay aquarium flickr video the year war a commentary by doug bandow of the future of freedom foundation points out how much we love war, well at least politicians love war: war has become a centerpiece of american politics. the war on terrorism is the focus of u.s. foreign policy. a real war is being fought in iraq. jimmy carter proclaimed the “moral equivalent of war” over energy. some analysts are advocating a war on obesity. the economist on open source from the economist in : open-source business: open, but not as usual. happy st. patrick’s day the entire kitchen is sandee’s playground, and that includes the chalkboard. i’m not sure what holiday she’ll decide to honor next. she’s been busy elsewhere at home too. mysql slow query log analysis peter at mysql performance blog pointed out this sweet perl script to analyze mysql’s slow query logs. (this is supposedly a php port.) the script does a good job of aggregating similar queries (those that only differ in their query values) and displaying overall stats for them. the following two queries are showing up a lot in my wpmu installation because i also have it set to log queries that don’t use indexes. slideshare wordpress embed shortcode i’m cleaning house in [bsuite][ ], and i’ve decided that this shortcode function for embedding slideshare items in wordpress needs to go. rather than totally toss it away, however, i’m posting it here in case somebody else finds it useful. ``` function shortcode_slideshare( $arg ){ // [slideshare id= &doc=misty-holland- - &w= ] $arg = shortcode_atts( array( 'id' => false, ), $arg ); if( ! $arg['id'] ) return( false ); return( str_replace( '%%id%%', $arg['id'], ' ' )); } add_shortcode('slideshare', array(&$this, 'shortcode_slideshare')); i missed the nightclub and bar show the international nightclub and bar show ran in las vegas last week, bringing a bunch of nightclub, bar, tavern, pub, restaurant, and hotel professionals to the city, including my friends at biba. dave must be faking his shock at the free shots, music, and dancing girls filling the hall “all at noon on a tuesday!” i’m not at all involved in the business, but i think i need to go next year. volkswagen ad claimed too violent for british tv first it was , then over complaints about the matrix-style (that means fake looking) kung foo action in volkswagen’s new ad. dual-wan or multi-wan load balancing routers bonding and . ad/ . ax link aggregation it’s not, but dual- or mutil-wan load balancing seems like a good way to improve overall bandwidth and reliability. the cisco/linksys rv (just under $ ) can group up to seven different wan connections, but the customer reviews are only so-so. for a little more i can get a peplink balance that can handle three wan connections and seems built for speed. there are other products, i know, but not a lot of information about any of them. yeah, i’m that guy i’m flying virgin america from bos to sfo, and apparently all their planes on that route offer in-flight internet via gogo. $ . buys mbps down and kbps up (at least early on when nobody else seemed to be using it). i can get my iphone online for only bucks, but as far as i can tell, i’d have to buy two plans if i wanted to use both on this flight. fly safe, fly without id this is an old one, but because i’m in the air again today it’s worth digging up this up. defense tech long ago pointed out the identity project‘s position on showing id for air travel: if a year-old college student can get a fake id to drink, why couldn’t a bad person get one, too? and no matter how sophisticated the security embedded into the id, wouldn’t a well-financed terrorist be able to falsify that, too? mmm… bacon who doesn’t like bacon, or little piglets? or kittens? juice your opac richard wallace’s juice project (javascript user interface componentised extensions) is a “simple componentised framework constructed in javascript to enable the sharing of ajax stye extensions to a web interface.” wordpress or scriblio users might do well to think about it as a way to put widgets on systems that don’t support widgets, though as richard points out, “the framework is applicable to any environment which, via identifiers contained within a html page, needs to link to or embed external resources. way cooler than a catalog i got a little excited when shirley lincicum wrote to the ngc lib mail list: [o]ne of the most frustrating things for me about next generation catalog systems as they currently exist is that they seem wholly focused on the user interface and can, in fact, actually hold libraries back from designing or implementing improved “back end” systems because of the dependencies introduced by the new “discovery layer” applications. i was excited because almost two years ago i wrote something like this: usability vs. open source this article comparing the usability of joomla vs. wordpress has already been linked by everybody’s uncle, but it’s still worth a look. i find it amusing, however, that none of the comments so far on that blog post mention the commitment that the core wordpress team appears to have on making blogging fun. if you start with the goal of making something fun, then add sophistication to make it flexible without being complex, you’ll get a very different result than you would if you started with different goals. tattoo: pantone seanbonner‘s photo of esther’s new tattoo makes me want one. tgfkae’s new tattoo by seanbonner on flickr scriblio theater flickr video flickr video i should have done screencasts like the above long ago. it’s not that they’re great, but they are a wonderful excuse to use the canned lounge music i’ve got. those videos are now on the front page of the official scriblio site, and i did five more to demo the installation and configuration. big thanks go to collingswood nj public library director brett bonfield who let me use his library like this. pedal powered hovercraft i love the engineering of the lift fan on this pedal powered hovercraft. it needs a little more lift to make really work, but wow. scriblio . released my slides for my presentation yesterday at code lib are available both as a . mb quicktime and a . mb pdf, while the gist of talk went something like this: scriblio is an open source wordpress plugin that adds the ability to search, browse, and create structured data to the the popular blog/content management platform. and wordpress adds great ease of use, permalinks, comments/trackbacks/pingbacks, and other social and web-centric features to that structured data. is internet linking legal? you’d think the top search results on the matter would be newer than , but that’s where you’ll find this nyt article and publaw item story, both from precambrian times. worse, both of those articles suggest that my links to them may not be entirely kosher. the problem is probably that us courts have not spoken clearly on such a case. a danish court in did, but i think that no case in the us has gone far enough to actually set a precedent. don’t be stupid, magenta is a color anybody who claims magenta isn’t a color is stupid, lying, or link-baiting. take it from a color-blind person: all colors are a matter of perception, and claiming magenta isn’t a color because it doesn’t fit neatly in the linear spectrum of visible electromagnetic radiation is like saying this isn’t music because the vibrations that tickle our ear aren’t the result of a monotone sinusoidal wave. we have no equivalent of polyphony for light, but just as it took a whole orchestra to make jaws scary, the colors we perceive are most commonly a mixture of different frequencies of light. make yours a modbook i really don’t know what i’d do with a tablet, but it’s still plenty interesting to see this modbook come together. on the other hand, if there’s anything to the earlier rumors of an apple tablet, i hope it leads to some sort of large-screen iphone-like device. pedal powered big wheel fun this big wheel was purported to be the work of cyclecide, a sf-based bike art collective. the big wheel is cool no matter who built it, and cyclecide’s pedal powered contraptions look awesome: the pedal powered roller coaster looks tame by comparison. turning a podcast track into a music track in itunes i subscribe to a few song of the day podcasts, which makes it easy to get the tracks, but difficult to enjoy them as music in itunes. but podcast tracks can’t be simply moved over to the music section of your library, it takes a little finagling. there’s a lot of advice out there suggesting you use one of the menu commands to convert the track to mp or aac, but i prefer not to re-encode my music, and that’s a big hammer for a small problem. , (max), (avg) mysql queries per second the above graph is far from typical, but i love that the box (the top one in this picture) can do the job when it needs to. this activity is a result of bulk record imports, web activity results in relatively little database traffic due to my use of memcached and batcache. the world’s greenest roller coaster this pedal-powered roller coaster is washuzan highland park‘s skycycle in okayama prefecture, japan it appears that the only co emissions are the huffing and puffing of riders peddling to the top. the park does have three traditionally powered steel coasters (the ultra, star jet, and chupy). how to ruin valentine’s day, and a basketball game valentine’s day will never be the same for this dude. aparently, however, marriage proposal rejections at basketball games are common, though this lol cats proposal worked out well. matching multi-line regex in bbedit i love bbedit on my mac, but i was left scratching my head again today when i was trying to remember how to make its regex engine match a pattern across multiple lines. my hope was to extract a list of initial articles from a page that had html like this: wordcamp higher ed, northeast it’s not wordcamp paris (running on february), but wordcamp edu northeast is today. i’m there to meet up with fellow wordpressies and talk about extending wordpress with holladay penick and dave lester. squeezing the three of us into a single time slot requires quite a bit of cutting, especially if we hope to have time to answer questions, so i’ll be focusing on scriblio. that means i won’t be talking about how we’re going to use buddypress or replace significant portions of our university portal with it. why are these people so happy? the soothing ambient sounds and smiling faces might be enough to have you keep this site open all winter long, but then you’d have to explain it. the world record headspin master is times cooler than me darien’s new materials handling is cool, but not world record headspin cool. actually, that’s probably a false comparison, enjoy them both. woot! wordpress mu . out sure, matt says it’s thank a plugin developer day, but let’s hear it for the developers who just tagged wordpress mu . ! not long ago there were still files to merge, now it’s done and ready for the next version. new hampshire: live free or die by firing squad nh state representative delmar burridge recently introduced hb proscribing death by firing squad: when the penalty of death is imposed, the punishment for a defendant convicted under rsa : , i(g) shall be execution by firing squad. burridge would likely describe himself as “principled,” like when he reported one of his constituents to the cops because of his advocacy for marijuana decriminalization. the photo above is a still from a ridley report interview with him. not happy it’s called gigapan, a robotic panorama-maker. david bergman used one to take the picture above (though his view was much larger) (you can buy your own for about $ if you get in on the beta). the point, however, is that if you zoom in real close, you can see w’s pursed lipped scowl. sitting in sin thomas von staffeldt’s remix of arne jacobsen‘s “chair no. ”. above are gluttony, pride, and lust. they’re all on auction, but does that suggest avarice? through the viewfinder original_ann‘s hacked-together rig for shooting though the viewfinder of her kodak starflex has me wanting one. she has a beautiful set and points to the through the viewfinder group for more. the real intronetz argument “what happens when a group that commands respect meets an audience that doesn’t give it readily?” pete cashmore on the vatican launching youtube channel. oh noes! my table is gone! # mysqlcheck -p -a --auto-repair --optimize wp_ _options info : found block with too small length at ; skipped info : wrong block with wrong total length starting at info : found block with too small length at ; skipped warning : number of rows changed from to status : ok cleaning up the mess after a hardware failure can suck. this mysqlcheck output is from the wp_options table for this blog. a cocktail i can believe in sandee’s toasting tomorrow’s inauguration with a special “fresh start cocktail.” i’m not usually one for overwrought imagery, but the delicate fruit flavor is quite refreshing change from the dark and stormy winter we’ve been suffering. and no, i really don’t know if i’m talking about the feet of snow that’s fallen these past couple months of those eight years we’ve suffered. everybody’s underwear i was using the dirty laundry metaphor in a previous post and wanted to extend it a bit by saying something like: for the generation of children who’s parents have already posted their silliest and most embarrassing baby pictures to facebook and elsewhere, being caught in your underwear is both expected and forgivable. being evil, on the other hand… except i couldn’t find a link to support my claim. gaming help: bond : quantum of solace walkthrough shadowzack knows his games a lot better than i do. even though he says it’s “crap”, i’m enjoying playing bond : quantum of solace on my wii. i only play about one game a year, so i’m not ashamed to go looking for a bit of help in shadowzack’s walkthroughs: chapter chapter chapter chapter chapter chapter chapter chapter chapter chapter hardmuth’s diy ring flash is quite a hack this light-piped ring flash should do the trick. it’s gotta be cheaper than canon’s offering (though cheap ring lights can be had for under $ ), and it seems to work more than well enough. no such thing as bad publicity finding a blog post about a condom and a cheeseburger made a friend ask if student blogs should be moved off-domain. my flippant answer was “there’s no such thing as bad publicity.” his retort was simple and quick: “tell that to the catholic church.” it stung. he had me, i was sure. it’s hard for many americans not to think of sex abuse when catholic church comes to mind, but there are probably two lessons from that: gaming: pac-txt richard moore’s pac-txt is even more brilliant than his paper pong (which, ironically, you can play online). here’s a transcript of my best pac-txt game to date: pac-txt! -------- you awaken in a large complex, slightly disoriented. glowing dots hover mouth level near you in every direction. off in the distance you hear the faint howling of what you can only imagine must be some sort of ghost or several ghosts. patrick mcgoohan dead at patrick mcgoohan, creator of the prisoner, has died. looking back at mac hardware performance i recently replaced the mac mini i use to host my web development with a powermac g . (story: the mini was mine, a personal purchase i made to support my work on scriblio and other wordpress-related projects, but recent changes in our network and firewall policy made the machine inaccessible from off-campus without using the vpn. having a personal machine sit at my desk at work isn’t as useful if i can’t use it conveniently and for para-work activities, so i wanted to take the mini home. firefox improved rdf browsing lbjay uses both the tabulator and semantic radar firefox plugins to do magic with rdf in his browser. play flv in quicktime player using perian perian: “the swiss-army knife of quicktime components” file formats: avi, divx, flv, mkv, gvi, vp , and vfw video types: ms-mpeg v & v , divx, ivx, h. , sorenson h. , flv/sorenson spark, fsv , vp , h i, vp , huffyuv, ffvhuff, mpeg & mpeg video, fraps, snow, nuppelvideo, techsmith screen capture, dosbox capture the lgpl–licensed quicktime plugin installs easily on mac os x . and does what it promises. flv videos (such as those you’d sneakily download from youtube) open just like any other quicktime vid, and you can easily export them to other types. corey blanchette’s song project the photos meme was quite popular last year (despite the day leap year). i might have joined, but it’s unlikely i would have finished. instead, i’ve been pushing my my brother-in-law corey blanchette, nicknamed coreyb or coreyb , to do songs in . he launched on january first and since then has done songs about elves, the serotonin in saratoga, albert ayler, and a bunch of others. if i ever find myself in prague… ilya schurov thinks this is the time capsule from from isaak asimov‘s the end of eternity. it’s really the elevator and stair (or ramp)-way in prague‘s old town hall. a clock and great views of the square are at the top. thinking of interesting elevators to be found in europe: the paternoster. diy fisheye lens for aiptek go-hd camera the aiptek go-hd isn’t such a bad camera for the money. it does p video and megapixel photos, but the lens doesn’t go very wide. but a post in the flickr blog pointed to a solution: use a door peephole as a fisheye lens. it works, but holding the peephole in front of the camera can get tiresome. here’s how i solved it: a rubber stopper easily holds the peephole, while a . some predictions come true way back in dave winer made a bet: in a google search of five keywords or phrases representing the top five news stories of , weblogs will rank higher than the new york times’ web site. it’s important to remember that in people still wrote “weblogs” in quotes, as though they weren’t sure how to use the word. winer won his bet in . anybody want to make a bet about ? safe livestock transportation recommendations you might not have cared to know the recommended trucking practices for pigs or other livestock, but colorado state university professor temple grandin is happy to explain all of that and more. she’s got videos too. perhaps you know somebody who made a new year’s resolution to improve the way they truck their livestock? you didn’t know they were fighting: the karen national liberation army in myanmar this news story from alerted me to a war i didn’t know anybody was fighting: the liberation of karen state from myanmar. the knla (karen national liberation army) and knu (karen national union) have been fighting for independence since the british left burma (myanmar) in . what do you get a -year old rebel movement for its birthday? here are their demands: for us, surrender is out of the question. super cheap aiptek go-hd video camera a while ago now i bought a aiptek go-hd p from amazon for cheap. the fotoramp review was helpful; links to actual raw video convinced me; but this video review was absolutely no help at all. you can’t track the brand on flickr; but a search reveals a few photos (one, two), a video, and even some photo panoramas (one, two) assembled from the video. you can see my own test videos here (note the link to the raw video in the description of each). new year’s hangover remedies i find a few sausage, egg, and cheese breakfast sandwiches and chocolate milk do the trick, but i’d eat those every day if i could. i’m always dubious of claims to national consensus, but this is especially ridiculous. is our national hangover cure really tomato juice and eggs? i thought it was hair of the dog, or beer and eggs. friends of mine have been so concerned by the challenge that they’ve developed biba, an electrolyte rich mixer that’s supposed to reduce the risk of hangover from the start (join their facebook group to learn more). will time warner cable customers be able to watch nickelodeon in the morning (or visit nick.com)? this dispute is going on now, tonight. there are obviously at least two sides to this story (viacom &time warner cable). you’d think a media giant like viacom would know how to handle this one, but it seems that all they’ve got is that splash screen in front of a bunch of their websites and this uninspiring ad. time warner cable, which you might think is just a bunch of network plumbers, seems a little more connected. wired but disconnected duckett‘s wired but disconnected on ccmixter is actually ironic: the whole song is the result of an online collaboration. listen lensbaby baby i have an old lensbaby . (looks like this) that does a great job of making casual snapshots look like real portraits. but i also find it really difficult to get focus on my subject. blame my bad eyes, my insistence on using it wide open with it’s shallowest depth of field, and simply sloppiness, but i can’t do it. this new lensbaby composer with a sort of normal focus ring (rather than flexible bellows), might work a little better. tankmen tankmen is funny, no doubt, but i wonder what it means when we’re deeply embroiled in two of the longest running armed conflicts of us history that we find it so easy to make comedy about war. happy holidays! jappy jaladays begets a number of other punny greetings: merry mojitos! merry margaritas! tijuana celebrate? hope you do. party tortilla tired! don’t let the season tequila. salsa nice having you in our lives. let’s go singing christmas cuervos! youtomb tracks takedowns on youtube youtomb continually monitors the most popular videos on youtube for copyright-related takedowns. any information available in the metadata is retained, including who issued the complaint and how long the video was up before takedown. the goal of the project is to identify how youtube recognizes potential copyright violations as well as to aggregate mistakes made by the algorithm. hacking cellphones for public health using only an led, plastic light filter and some wires, scientists at ucla have modded a cellphone into a portable blood tester capable of detecting hiv, malaria and other illnesses. via wired. lcsh linked data lcsh.info is gone, but there’s a lot to learn from this paper. i wish i’d seen that earlier. everybody’s spoon is too big best of craigslist: manly bike for sale from the best-of-craigslist: manly bike for sale: what kind of bike? i don’t know, i’m not a bike scientist. what i am though is a manly guy looking to sell his bike. this bike is made out of metal and kick ass spokes. the back reflector was taken off, but if you think that deters me from riding at night, you’re way wrong. i practiced ninja training in japan’s mount fuji for years and the first rule they teach about ninja biking is that back reflectors let the enemy know where you are. plugin options pages in wordpress . wordpress . requires that plugins explicitly white list their options using a couple new functions. wordpress mu has required this security measure for a while, and it’s nice to see an evolved form of it brought to the core code. [migrating plugins and themes to . ][ ] article in the codex offers some guidance, but here’s how it works: first, register each option for your plugin during the admin_init action: ``` function myplugin_admin_init(){ register_setting( 'my-options-group', 'my-option-name- ', 'absint' ); register_setting( 'my-options-group', 'my-option-name- ', 'wp_filter_nohtml_kses' ); } add_action( 'admin_init', 'myplugin_admin_init' ); ``` in the example above, the value for my-option-name- will be filtered by absint before being saved to the options table. quizzes are good link bait via information nation: how long could you survive chained to a bunk bed with a velociraptor? and how many five year olds could you take in a fight?. the social beaver: s campus life at mit really, it’s titled “the social beaver,” though i can’t imagine campus life ever looking like that. aside: mit’s techtv is powered by viddler’s white-label solutions. woodman institute, dover, nh the woodman institute museum in dover nh is famous for having a four-legged chicken, but that’s only a small example of the weirdness you’ll find inside. a big collection of snakes and bugs and bears in top hats along with other examples of taxidermy fills the first two floors. the top floor is dedicated to war and includes the obligatory rusty cannon ball that killed and maimed. what could have been: lee mercer’s presidential campaign former presidential candidate lee mercer shares your concern for circumstances and issues. he wants to crack down on treason and recognizes democratic concerns about expansion of executive power. mysql . released, community takes stock mysql . is out as a ga release, but with crashing bugs that should give likely users pause. perhaps worse, the problems are blamed on essential breakdowns in the project management: “we have changed the release model so that instead of focusing on quality and features our release is now defined by timeliness and features. quality is not regarded to be that important.” still, people are finding inspiration in ourdelta and drizzle. simile timeline for, um, timelines timeline is a simile project that uses exhibit json (which you can create with babel). longwell rdf browser longwell mixes the flexibility of the rdf data model with the effectiveness of the faceted browsing ui paradigm and enables you to visualize and browse any arbitrarely complex rdf dataset, allowing you to build a user-friendly web site out of your data within minutes and without requiring any code at all. demos another approach to web forms just saw a cool demo of xforms and orbeon forms. wordpress for zach’s web programming class zach is apparently too lazy to prep his own lectures for the last few days of his intro to web programming class. after bringing his students from zero to database-backed web-apps, he asked matt do javascript and me to introduce wordpress as an application platform. the wordpress api makes it easy to write plugins that modify wordpress’ behavior with filters and action hooks. additionally, shortcodes allow you to put small bbcode-like tokens in your wordpress posts and pages that are replaced with by functionality defined in your plugins. real data architecture: stockholm data cave need a retro-looking bomb shelter for your server, or are you a big fan of the cheyenne mountain scenes in wargames? the bahnhof pionen white mountains hosting facility is a cave below stockholm. you’d expect the sysadmin blogs to call it fit for a james bond villain, but even the architecture blogs are a gaga. trendhunter compares it to the rfm fm radio headquarters (poland) and john lautner‘s chemosphere house (los angeles). lens lust digital photography review’s look of sigma’s mm f/ . has me drooling. i have an el cheapo mm f/ . and am looking to upgrade. at $ , canon’s mm f/ . is just way too expensive, but their mm f/ . just didn’t seem to be enough of a upgrade to be worth the price. sigma’s new lens, seems to do it. i stumbled into that lens, however, as i was looking up canon’s ef mm f/ . derailed eu-jin ooi‘s picture of rail trucks piled up after a derailment isn’t nearly as scary as this derailment found at dee’s inbox: can anybody name that incident? (the top one is bnsf, barstow ca, april . what’s the bottom one?) piano man light-paint piano player from ryan cashman on vimeo. mobile safari advanced features if you’re already building web apps, you might wonder why you should bother to build an iphone native app. the short answer is that you might not need to, but you should still optimize the app for iphones. native-looking chrome set these in the head: ``` // set a custom icon for when a user bookmarks the app to the home screen // hide the browser chrome //set the phone status bar style; can be grey, black, or black translucent </td> </tr> </table> </div> caveats: * only works for web pages that have been saved to the home screen and opened from there. iphone dev camp nyc i’m at apple’s iphone tech talk in new york today. info is flowing like water through a firehose, so i’m not going to attempt live blogging, but here are their suggested ingredients for a successful iphone app: delightful innovative designed integrated optimized connected localized the picture is of the main theater for the event. it’s by far the most beautiful space i’ve ever been in for a tech conference. peephole diy fisheye lens flickr blog i discovered the peephole fish eye group. the idea is simple: us a $ door peephole to give your camera a fisheye lens. here are the instructions: hold peephole against rim of camera lens. set camera to “macro”. (the image is actually displayed on the inside face of the convex lens of the peephole. the camera must focus on the foreground image rather than the background image.) zoom in to the point that the viewable “circle” is framed almost evenly. i am talking to you after stuffing yourself with too much thanksgiving dinner and the tryptothan kicks in, there’s some time when all conversations seem to work like this one from martin wilson. a dc story one sunny day in january, an old man approached the white house from across pennsylvania avenue, where he’d been sitting on a park bench. he spoke to the u.s. marine standing guard and said, “i would like to go in and meet with president bush.” the marine looked at the man and said, “sir, mr. bush is no longer president and no longer resides here.” the old man said “okay”, and walked away. after the thanksgiving feast: answer who owns the fish you can only eat so much, and though we’ll likely stretch those limits tomorrow, at some point we all have to take a break. the good folks at coudal partners have the perfect solution: a simple test (available as a convenient pdf) that einstein says only a handful of people can actually figure out. the premise is simple: somebody in the neighborhood keeps a fish, but who? read the clues, work it out, and send your answer to the coudal folks. if you’re right they might have a prize for you. you can leave your answer in the comments here too, but all i’ll have for you is left over turkey. amazon’s content delivery network launches in beta amazon calls it cloudfront, and it costs $ . – $ . per gb at the lowest usage tiers. it seems that you simply put your files in an s container, make an api call to share them, then let your users enjoy the lower-latency, higher performance service. their domestic locations include sites in virginia, texas, california, florida, new jersey, washington, and missouri. internationally, they’ve got amsterdam, dublin, frankfurt, london, hong kong, and tokyo covered. web search re-imagined: searchme iphone app re-imagined a bit, anyway. why browse a vertical list of results when you can flip through them like pages in a book (or album covers in itunes). searchme on the iphone and ipod touch does just that. as you type your search term, icons representing rough categories appear, allowing you to target your search and helping people who’re searching for information about pythons the snake avoid results about the programming language. video drm hammering legal consumers nobody but the studios seem happy about apple’s implementation of hdcp on its recent laptops. the situation leaves people who legally purchased movies unable to play them on external displays (yeah, that means you can’t watch movies on the video projector you borrowed from the office). a related story may reveal the extent of the problem. the mpaa is petitioning the fcc to allow it to use “selective output control” to block playback of video content in a manner similar to hdcp. sco vs. novell lawsuit over, linux safe according to groklaw, the long running battle between sco and novell may finally be over. the judge ruled that sco, the company that claimed linux infringed on it’s ip and sued everybody in sight, never did own any rights to unix in the first place, and has ordered the company to pay millions. novell and others are unlikely to ever see much of that, though, as sco is in bankruptcy. toshiba takes bullet time up a notch supposedly this is more real than it looks. see how it was made. the uss albacore, portsmouth nh the albacore is a post world war ii experimental submarine now on display in portsmouth nh. seeing the sub on land, some height above sea level, is a bit surprising, and it’s clear that moving it there was no small task. five dollars will get you inside the sub’s tight and awkward quarters, where you’ll see the frankensteinian bathroom (and that’s for officers) and details such as lithium hydroxide canisters and signal ejector instructions that stand as reminders of the dangers of submarining. nest: the softer side of maisonbisson sandee’s not such a fan of the new theme here at maisonbisson. without really telling me that i should have discussed the new decor with her before making any big decisions, she does say she feels it doesn’t suit her style. there are lots of ways to resolve the, um, difference of opinion, but we decided that just as sandee gets most of the authority regarding the kitchen and i get the office, we can find a way to share the website. lincoln obama paste up mashup enrguerrero‘s photo of a lincoln/obama paste up mashup on the corner of larkin and myrtle streets in san francisco. fiddling with open source software for libraries theme i generally liked commentpress, but when the institute for the future of the book website went down recently, it started throwing errors in the dashboard. so i decided to re-do the open source software for libraries website using derek powazek’s depo masthead. i think it’s a beautifully readable theme, and i only had to make a few modifications. i’ve ostensibly lost commentpress’ paragraph-level commenting features, but i discovered those may have been broken all along (that was what started me thinking about replacing the theme). obama’s use of complete sentences stirs controversy from the borowitz report: in the first two weeks since the election, president-elect barack obama has broken with a tradition established over the past eight years through his controversial use of complete sentences, political observers say. “every time obama opens his mouth, his subjects and verbs are in agreement,” says mr. logsdon. “if he keeps it up, he is running the risk of sounding like an elitist.” more… mcgill university powered by wordpress well, not the entire university, i guess, but a number of online publications use it. the newspaper is featured above, their cio has a blog, and they’ve started a pilot with wpmu to offer blogging to everybody in the university. abandoned cars, yes, but abandoned jumbo jets? residents of mumbai (bombay) were wondering who was responsible for removing an abandoned in their chembur neighborhood. then, as quickly and mysteriously as it appeared, it vanished. the times of india says the plane arrived by truck, but the driver took a wrong turn and couldn’t maneuver the foot long hulk out. wingless planes and beached whales aren’t so dissimilar. the oregon highway department knows how to take care of the latter (though, it turns out that whales are known to spontaneously self destruct). tricky uses of bsuite after writing the project page for wpsms i didn’t have much more to say in a blog post announcing it. the cool thing about writing pages in wordpress is that i can create a taxonomy like /projects/wpsms/ to place them in. the downside is that new pages never appear in the rss feed. so i need both the page and a blog post to announce it. i could have simply copied the content from the wpsms page into a blog post, but that creates confusion and splits the audience between the two pages. wordpress uses: oobject oobject‘s galleries of abandoned pools, subway architecture, and revolting gold gadgets, among others, are all built in wordpress. using wordpress with external smtp server i really don’t like having sendmail running on a webserver, but some features of wordpress just don’t work if it can’t send email (user registration, for example). still, wordpress offers support to send email through external smtp servers instead if a local mailer. in <a href="http://trac.wordpress.org/browser/tags/ . . /wp-includes/pluggable.php">/wp-includes/pluggable.php</a> around line , change ``` $phpmailer-ismail(); ``` to ``` $phpmailer-issmtp(); ``` then, in <a title="/tags/ . a day in the life… dgenerate nation – skate with me from dgenetics on vimeo. whisky and gin dispenser gaellery‘s hotel room whisky and gin dispenser. push in the drawer, pull out, and find a tiny bottle of booze. just like those movies you claim you didn’t watch, it’s automatically charged to your bill. uploading .docx files in wordpress it may be a sign that none of the core wordpress developers much likes or uses microsoft office, but the core code hasn’t been updated to recognize the office file extensions like .docx, .pptx, or .xlsx. it’s no criticism, wouldn’t have discovered it if a user hadn’t complained, and i stewed a bit before deciding it was a bug. it’s now ticket # in the wordpress.org trac. it only affects my mu users now, though, and the same patch works there. world usability day today the usability professionals’ association says “a cell phone should be as easy to access as a doorknob.” and since they’ve been organizing world usability day to help make that happen. locally the upa boston chapter is holding events at the boston museum of science (in cambridge, actually) that explore the clues we use to understand how to operate doors and the frustrations of setting an alarm clock. this year’s theme is transportation, and they have an online transportation survey that helps us see our “transportation footprint and learn how small travel changes can make a big impact on all our lives. google brings video to gtalk, but why no ichat/skype interoperability? google yesterday introduced video chat to the web-based version of it’s google talk app (think gmail), but doesn’t appear to interoperate with any of the many existing video chat apps, ichat and skype tops among them. getting a teflon fix teflon might be just what i need to get my walking desk treadmill back in working order. but where to get it? turns out that dupont sells in both teflon spray and squeeze bottle. found via. the animated llama you didn’t know you needed click for more. i dare you. wordpress education mail list wp-edu, the wordpress for education mail list has launched. join up, catch up on the archives, and set it up at your school. new plugin: wpsms supports sending sms messages [include post_id=” ″ field=”post_content”] poke a muffin click for more. i dare you. a bullet dodged we all knew the sordid details of palin’s candidacy would emerge, but who figured they pour out so soon or on fox news? via borkweb.com declaration of metadata independance declaration of metadata independance: we hold these truths to be self-evident, that metadata is essential to all users, and that the creation of metadata endows certain inalienable rights, that among these are the right to collect, the right to share and the pursuit of happiness through the reuse of the metadata… (read more) via. svn repository hooks rock i stumbled on them by accident, but once i discovered subversion supports action hooks that can fire before or after a transaction, i knew exactly what to do with them. presidents change…presidential limousines change presidential limos are armored, yes, but gregg merksamer reveals that george w. bush’s limos sport five-inch thick glass, more than twice as thick as in clinton’s limo. merksamer should know, he wrote the book on so-called “professional cars”. he says half an inch is enough to stop a . magnum at point blank range, and bmw’s x “security” model features only a little more than that. so what’s it mean when a person needs ten times that amount? mccain staffers: more whisky. stat! john mccain’s election team apparently told staff at the phoenix biltmore to have extra whisky on hand for their election party tonight. they’re not just planning to drown their sorrows: republicans and republican-leaning independents drink more whisky than the national average. sweet photo by bearfaced, though i almost used this picture of barrels (or this one). techno viking rocks more than other vikings (and vikings generally rock) the technoviking will have you scratching your head for the first seconds, then rofling for a while. not enough yet? watch him dance to “it’s a piece of cake to bake a pretty cake.” this one claims to be the original, and though the sound is bad the video quality is much better than the others. thing is, now that you’ve watched it a couple times, did he stop a pickpocket or admonish a groper at the beginning? wikipedia api? i’ve wanted a wikipedia api for a while. now i might’ve stumbled into one: commons.wikimedia.org/w/api.php. it doesn’t do exactly what i want, but it might yet be useful. engrave your tech the image on this moleskine notebook was custom laser engraved by engraveyourbook.com, a part of engraveyourtech.com, where they recently announced they were suspending moleskine engraving due to atmospheric health concerns. you can’t get a notebook, but you can ogle the fancy, laser engraved macbooks creative commons licenses not compatible with gpl? gpl and cc are incompatible? fsf says so, and the debian free software guidelines agree. i’m as opposed to ruinous compromises as the next guy, and i feel the gpl fever, but i just want to use mark james‘ excellent silk icons in my gpl’d wordpress plugin. csshttprequest: cross domain javascript solution who’d a thunk it: csshttprequest is a way of doing cross-domain ajax by using css’ @import method to fetch the data. super mario quilt keith lewis bakes, paints, makes robots with machine guns, and has stitched not one but two mario quilts (closeup, from back). they apparently make good gifts, who wouldn’t want one? diagramed: things said during sex view it large, for all the details. via anonymous. asian robot olympics news of brickcon the web and the flickr earlier this month, but mse ’s photos of robot competition have my attention now. but what am i looking at? what was the competition? steve souders website performance o’reilly webcast i’ve linked to steve sauders‘ webcasts on website performance optimization before. here’s another. turns out that he’s co-chairing the o’reilly velocity conference in june. apache virtual hosting black magic i’ve configured apache for virtual hosting on more sites than i can count, but i’ve always just kind of stumbled through until now. what’s changed? the apache . documentation is worlds better than the old . docs (even though the old docs rank highest in google). so here they are: name-based virtual hosts, plus virtual host configuration examples (including an example mixed name and ip virtual hosting, which is what i needed), and some tips on dynamically configured mass virtual hosting. sarah palin is a vampire i think this election has designers more involved than most. (via dottiebobottie.) determining paths and urls in wordpress . + wp . allows sites to move the wp-content directory around, so plugin developers like me can’t depend on them being in a predictable location. we can look to the wp_content_dir and wp_plugin_dir constants for answers, but a better solution is likely to use the x_url() functions. the most useful of those is likely to be plugins_url(). even better, you can give these functions a relative path and they’ll return a fully qualified url to the item. xfruits: “compose your information system” is xfruits a worthy replacement for yahoo! pipes? wordpress bug: duplicate post_meta entries i just submitted a trac ticket about this: the update_post_meta() and delete_post_meta() functions don’t know how to deal with post revision ids. add_post_meta() does, it uses the following block of code to make sure the passed $post_id is a real post, not a revision: ``` if ( $the_post = wp_is_post_revision($post_id) ) $post_id = $the_post; ``` this is important because the global $post_id when a post is being saved is for the revision, not the real post. are you ready for the digital tv conversion? this psa should help you understand the upcoming switch to digital television. (via) comfort, thy name is sumo i sink into a strange, giant blue marshmallow and sigh contentedly. i balked at this new furniture. i balk at anything that i don’t actually pick out. i didn’t pick this out, casey acquired it on his own. our home is small and i am very picky about what goes into it. this was a beanbag. a beanbag? i can’t think of a more immature piece of furniture. libraries vs. it departments the chronicle‘s tech therapy podcast last week featured libraries vs. it departments. (via.) xkcd against drm i think richard m. stallman would agree with xkcd: drm is evil. it’s bad for both customers and content creators — even hilary rosen and steve jobs have their doubts about it. got wood? you can get a carved wood replica macintosh or faux-wood vinyl wrap for your mac mini, but asus is demoing a series of bamboo-covered computers and fujitsu is showing their cedar concept. and then miniot has a series of wooden cases for your iphone and ipod touch. olde skool ipod cases contexture design‘s ipod classic and nano cases made of reclaimed rpm vinyl or audio cassettes are just fine. too bad they’re all sold out. edward tufte on the iphone’s ui design edward “to clarify add detail” tufte, who criticizes the powerpointing of america, earlier this year posted a video on the iphone’s ui design. he loves the photo viewer (except the grid-lines between images are too big), he loves the web browser (except the navigation bar takes up too much space), he calls the weather app an elegant way to demo your iphone to friends (but says it’s devoid of information), and calls the stock market app cartoonish. how wikipedia works when phoebe ayers isn’t hanging out at roflcon she’s probably doing something related to wikipedia, so i’m looking forward to reading how wikipedia works: and how you can be a part of it. extra points: phoebe and her co-authors somehow convinced their publisher to release the entire work under the gfdl, the same license wikipedia uses. you could read the entire thing online for free, but that’s the easy part. beat it: instant rimshot scott carver has his hand in a number of projects — the penny jam is especially outstanding — but his instant rimshot is one of those silly infectious sites that’s you can’t help but share. another reason i’m glad i left verizon i received the following message from clickatell, the sms gateway provider i use to programmatically send text messages to cell phones: please be advised that us carrier verizon wireless has announced that they will be charging an additional c per sms for all application originated mobile terminated messaging beginning november , . this increase will apply to standard rate and premium programs only through the verizon wireless network. transaction fees will not apply to free- -end-user, mobile giving or non-profit organizational programs, according to verizon. wordpress event calendaring plugins i actually use event calendar, which has been abandoned for some time. looking at the alternatives listed in the plugin directory, calendar, events calendar, and gigs calendar add full calendar management features to wordpress. while ics calendar, ical events, and upcoming events, simply offer the ability to display calendar data from elsewhere. what i liked about the old event calendar plugin is how events were posts. creating an event started with creating a new post. converting mysql character sets this gentoo wiki page suggests dumping the table and using iconv to convert the characters, then insert the dump into a new table with the new charset. alex king solved a different problem: his apps were talking utf , but his tables were latin . his solution was to dump the tables, change the charset info in the dump file, then re-insert the contents. tracking aircraft movements from justin: real-time flight tracking. you can even overlay it on google earth. none of them as pretty as aaron koblin’s flightplan, though. acronym overload: iis + isapi + cas i’m working to integrate an application on a remote-hosted iis server into our cas environment. casisapi (svn trunk or svn tags/production) may do the trick, though phil sladen struggled with it (in ). there’s reason to doubt it. not only is the sparse information all old, i first learned about it from a page full of broken links and the apparent author recommends against it. there’s a little more information here for those who can read danish. sarah palin’s debate strategy flowchart via jon link: sarah palin’s debate strategy flowchart. eh. at least she had a strategy. what’s mccain’s plan going to be for tonight? autoerotica, detailed photos of the silver suv apparently backed out into the street so fast that it struck and flipped the blue car, then mounted it. nobody appears to have been seriously hurt, so we all have a guilt-free pass to mock the, um, compromising situation. found in paula wirth‘s photo stream. demetri martin flips his chart you’ll find more than a few of demetri martin‘s (his site) videos on the web (one, two, quotes). though i think he’s particularly good at powerpoint comedy and this flipchart thing, you’d think he doesn’t like to do interviews. solaris’ cachefs could be the space ship i’ve been looking for joerg moellenkamp‘s post explaining cachefs has me excited: long ago, admins didn’t want to manage dozens of operating system installations. instead of this they wanted to store all this data on a central fileserver (you know, the network is the computer). thus netbooting solaris and sunos was invented. but there was a problem: all the users started to work at o’clock. they switched on their workstations and the load on the fileserver and the network got higher and higher. this stone laid by l.g. bogus physically located in katoomba; found in seb chan‘s photo stream. do wordpress pages better with bsuite wordpress‘ pages feature makes the popular blogging platform a sophisticated cms. bsuite adds a few features to make it even better. write excerpts, tag, and categorize your pages wordpress excerpts are an underused but powerful feature that allow you to explain to your readers why they should read the page you wrote. tagging and categorization of pages help improve the findability of those pages, especially in search engines. what is social media? social media in plain english and rss in plain english, among others from common craft among the best explanations you’ll find. knowledge, distilled and sketched on index cards maslow without the pyramid, found at jessica hagy’s “indexed”. she posts new explanations of the world daily. more available in her book. website performance vs. crawl rate simple fact of the google economy: people can’t find stuff if it’s not indexed in major search engines. a slow site might not seem as bad as blocking the crawlers that search engines use to index your content, but it does seriously affect the depth and frequency of crawling they do. the above is google’s report of their crawling activity on a site i’ve been trying to optimize server performance on. beginner’s guide to dataportability, the video dataportability – connect, control, share, remix from smashcut on vimeo. from dataportability.org: the dataportability project is a group created to promote the idea that individuals have control over their data by determing how they can use it and who can use it. this includes access to data that is under the control of another entity. you should be able to decide what you do with that data and how it gets used by others open source solutions are preferred to closed source proprietary solutions bottom-up distributed solutions are preferred to top down centralized solutions my devcamp lightning talk hi, i’m casey. i developed scriblio, which is really just a faceted search and browse plugin for wordpress that allows you to use it as a library catalog or digital library system (or both). i’m not the only one to misuse wordpress that way. viddler is a cool youtube competitor built atop wordpress that allows you to tag and comment inside the timeline. staypress is a property management and booking system also built atop wordpress. scaling php this two year old post about rasmus lerdorf’s php scaling tips (slides) is interesting in the context of what we’ve learned since then. apc now seems common, and it’s supposedly built-in to php . still, i’d be interested in seeing an update. are mysql prepared statements still slow? and that’s where rasmus’ latest presentation comes in. we don’t learn anything about mysql prepared statements, but we do learn how to find choke points in our applications using callgrind and other tools. scared of the dark? who knew an ad that targeted our fear of the dark could work so well or playfully? then again, what would this ad feature if it played here in the us? do you still use your walking desk? michael pratt asked me recently: do you still use your treadmill desk? do you continue to find it beneficial? i love the idea of these things, but worry a little that i might tire of it in practice, or that it might be difficult to work at it for long periods. it may seem a perfect opportunity to revisit my old walking desk blog post, but that just raises the guilt level i feel every time i see the thing unused. sweet business cards this handful of business cards is good for a little design inspiration. and here’s more if you need an extra shot. thanks to frank for the tip. amazon to offer content delivery services via an email from the amazon web services group today: …we are excited to share some early details with you about a new offering we have under development here at aws — a content delivery service. this new service will provide you a high performance method of distributing content to end users, giving your customers low latency and high data transfer rates when they access your objects. the initial release will help developers and businesses who need to deliver popular, publicly readable content over http connections. the url is the citation from jessamyn: “don’t toss up a bunch of bibliographic citations when a decent url will do. you’re online, act like you’re online.” yet another encryption crack those kwazy kids will quack anything now. stream ciphers may never have been expected to be that secure, but adi shamir’s cube attack breaks them like so many, um, bits of data. michael pick screencast master professional screencast producer michael pick has joined automattic and shuttered smashcut, his production company. it’s not all bad, though. he’s been busy making instructional videos for wordpress.com (many of which are useful for wordpress.org users), explaining things like how to manage tags or use the press this! feature, and answering the question “what should i do first?” what does this suggest about the pro screencasting marketplace? pick says “this is a huge underdeveloped niche, [with fewer] screencasters with chops than there are jobs. google minus google from the register: inspired by a recent new york times piece that questioned whether the mountain view search monopoly is morphing into a media company — which it is — finnish blogger timo paloheimo promptly unveiled google minus google. key in the word “youtube,” and the first result is wikipedia. open source citation extractors for non-structured data hmm-citation-extractor, parscit and freecite (not to be confused with freecite, the f/oss endnote-like app). freecite is available as a service and a download. still, wouldn’t a simple url be easier than all these unstructured citation formats? installing php apc on rhel/centos yum up some packages: ``` yum install php-pear php-devel httpd-devel </td> </tr> </table> </div> . install apc using pear (the pear installer is smarter than the pecl installer): when the installer asks about apxs, say ‘no’. </p> <div class="wp_syntax"> <table> <tr> <td class="code"> ``` pear install pecl/apc </td> </tr> </table> </div> tell php to load apc: ``` echo extension=apc. some might suggest banning sticky notes from the office eepybird’s sticky note experiment from eepybird on vimeo. i have some experience with post-it notes in the office, and though that achieved international recognition, it doesn’t quite compare to what we see in this video. our , post-it notes just don’t compare the , we see slinking across the screen now. web form validation with jquery josh bush’s masked input plugin and paulo p. marinas’ alphanumeric are both jquery plugins to prevent input of invalid data in web forms. greensql | open source database security greensql promises to protect sql databases against sql injections. greensql works as a reverse proxy and has built in support for mysql. the logic is based on evaluation of sql commands using a risk scoring matrix as well as blocking known db administrative commands (drop, create, etc). css transformations in safari/webkit (and chrome too?) the cool browsers support radius corners, but safari supports css transformations that allow developers to scale, skew, and rotate objects on the page like we’re used to doing in postscript. and better than that, we can animate those transformations over time — all without any javascript. fire up safari or chrome and mouse over the examples here. the screencast at the top is from the menu on that page. there are, obviously, better uses for these transforms, but it’s easy to see it at work there. browser-based json editors jsonlint, a json validator, was the tool i needed a while ago to be able to play with json as format for exchanging data in some apis i was working on a while ago. and now i like json well enough that i’m thinking of using it as an internal data format in one of my applications, especially because it’s relatively easy to work with in javascript. or, at least that’s the promise. nfl powered by wordpress wordpress.com vip hosts some high-traffic sites, including gizmodo’s live coverage of the iphone g introduction. now that the nfl has selected the service for their blogging we’ll get a chance to see how they handle the superbowl rush. michael stephens teaching on wordpress mu michael stephens is now using wordpress mu to host his classes online, and that opening page is really sweet. it’s hardly the first time somebody’s used a blog to host course content, but i like where he’s going with it. we’re significantly expanding our use of wordpress at plymouth, and using it to replace webct/blackboard is definitely an option. the biggest difference may be that course content in blogs is public, by default, but content in blackboard is shared only with the members of the course. google’s own satellite it’s not truly “google’s own,” but the internet giant will get exclusive use of the images for mapping purposes, according to reuters: geoeye inc said it successfully launched into space on saturday its new geoeye- satellite, which will provide the u.s. government, google earth users and others the highest-resolution commercial color satellite imagery on the market. of course, google doesn’t need a satellite to watch us all very closely. thesis and f — two sweet commercial wordpress themes good work deserves compensation, but commercial themes are still unusual in the world of wordpress. the new themes directory has well over free themes listed, and the old directory had thousands of them. still, i like thesis and f . actually, i like a bunch of themes from graph paper press (get them all for $ !). and, as we see wordpress adding so many options that require theme support, the promise of free lifetime upgrades for thesis is also appealing. installing memcached on centos/rhel using info from centos forums, sunny walia and ryan boren, here’s how i got memcached running on my dotster vps: install libevent: ``` wget http://www.monkey.org/~provos/libevent- . e.tar.gz tar zxvf libevent- . e.tar.gz cd libevent- . e ./configure make make install ``` install memcached ``` wget http://danga.com: /memcached/dist/memcached- . . .tar.gz tar zxvf memcached- . . .tar.gz cd memcached- . . ./configure make make install ``` we will start the server to use megs of ram (-m ), listen on ip . want: canon’s eos d news of canon’s new eos d with iso sensitivity as high as , has my mouth watering. i used to push my black and white film so much that development times were as long as minutes (i bought super cheap asa and pushed it to ) just so i could get decent natural light. i leave my canon digital rebel set for and usually only remember to knock it back when i go outside and find i can’t shoot wide open. axiotron modbook: cool, but bad timing? the axiotron modbook is cool, i gotta admit, but with so many rumors of a macbook touch due this fall, i suspect that potential buyers might be holding their breath. but, on the other hand, those people have been waiting for a mac tablet since jobs killed the newton, and rumors of a tablet are hardly unusual — see , , , , , , . still, the whispers of an over-grown iphone device are getting a lot of echos lately. jon stewart vs. gop/sarah palin media machine dragonflyer x uav remote control helicopter is sneaky, awesome i so want one of these sweet draganflyer x helicopters. the two pound powerhouse can carry up to one pound of camera equipment, carrying it smooth enough to get decent video and stills. more videos are at the dragonfly website, including one which supposedly demonstrates that it’s quiet enough for wildlife photo work (scroll down and look for “hawk”). who knows how much it costs, but i requested a quote. automated website screen captures on os x i’m not sure exactly what i’ll do with it, but thanks to this tip about webkit png, i now know how to get screen captures of websites. maybe useful for archiving. who knows. wordpress cas integration plugin cas — central authentication service — has no logo, but it’s still cool. heterogeneous environments like mine offer hundreds of different online services or applications that each need to authenticate the user. instead of throwing our passwords around like confetti, cas allows those applications to identify their users based on session information managed by the cas service. it also obviates the need for users to offer their credentials to potentially untrusted systems — think externally hosted systems. bush trying to figure out how to invite volleyball team to white house sure, volleyball is the new gymnastics, so much so that the white house posted a picture of bush with olympians misty may-treanor and kerri walsh in their “news & policy” section. chalk it up to august being a slow news month. still, i can just imagine the old man telling laura “i think you should invite those volleyball girls to the house sometime.” and laura, i hope, responds: “you can watch them shake it on tv if you need another look. joshua longo’s longoland is full of fuzzy, but not cuddly animals brooklynite joshua longo‘s crazy animals are showing at the shelburne museum in vermont through october th. sweet for me: i’ll be in town this weekend. i’m hoping to check it out. are rock operas too weird for remixing? i love remixes, mashups, and covers. i love it when bad songs get good covers, i love it more when it’s a bad cover. i’m a fan of coverville and i get excited every time i find yet another version of smells like teen spirit (hey, this is just a sampling: lullaby version, patti smith, the bad plus, another jazz version, and another jazz version, a string version, no, two string versions, a tango, a damn chant version, some lounge thing, and one for the opium lounge). but i think i have yet to hear a decent cover or remix of a track from a rock opera. take one night in bangkok: sexing it up doesn’t help. you just can’t out rock a rock opera. (really, look for yourself.) it might help that chess featured a character loosely based on eccentric chess master bobby fischer, but rock operas just might be too weird for remixing. though…i’d like to be surprised. perhaps a folk version? can design save democracy? from the new york times: how design can save democracy …recently, the brennan center for justice at new york university school of law issued a report outlining the importance of well-designed, easy to understand ballots. duh. and, i guess we’re giving up on electronic voting. . million self-hosted wordpress sites and counting the huge problem with open source software is that there are no sales numbers to show how many people are using it. we know that wordpress.com hosts over three million blogs. we know edublogs powers nearly , . but how many sites are hosted using the original, downloadable, self-installed and managed version of wordpress? now, the automatic update notification system in wordpress gives answers to that question and others. most hugely: over . sweet drobo home raid i’m not sure who robin harris is, but he’s mighty sure home raid won’t fly. he’s just so certain that consumers are stupider than him and that vendors’ imaginations are as limited as his. and if harris was right, we’d probably still be using microprocessors and getting by on less than a megabyte of ram, because “nobody needs more than k.” too bad then that data robotics‘s drobo seems to do everything harris says home raid can’t. olpc origins: us and taiwan’s hardware lovechild olpc origins: us and taiwan’s hardware lovechild a deeper than expected history of the olpc’s development. part two of a three part series. ssd for my bacbook pro? sure, we can get a macbook air with gb solid state disk (ssd), but what about upgrading a macbook pro? ryan block put one in his mbp and got a second startup. ridata released a gb . “ sata ssd in january that looks compatible with my macbook pro. newegg has it for under $ . for comparison, however, a gb . ” spinning platter sata drive can be had for under $ . more web performance tips from steve souders hearing steve souders at wordcamp last week got me thinking about website performance, so i went looking for more. the slides from his wordcamp talk are online, but he gave a similar talk at google i/o which got videotaped and posted richer detail than his slides alone will ever reveal. also on his blog: use the google ajax libraries api when you don’t have a cdn, and a post that asks why make users wait to download all your javascript before they see the page if you’re only going to use % of it at first? commentpress comments the rights to my library technology report on open-source software for libraries have reverted back to me, so i’m posting the text online under a cc-by-sa license. more importantly, i’m using it as an opportunity to play with how longer-than-blog texts can be represented online. the institute for the future of the book has spent some time thinking about that very question, and their answer is commentpress, a theme for wordpress that enables commenting on each paragraph of a text and organizes posts into a book-like table of contents with the first (and oldest) posts on top. mysql performance monitoring tips from the mysql newsletter google turned this up, but i have no idea how old it is: how to monitor mysql’s performance. the war on photography amanda mooney posted a note about being told she needed corporate permission to take a picture in a store. mooney’s interest was in telling others how much she likes the products and the brand — exactly the sort of word of mouth advertising most brands are anxious for, but imagine some more pedestrian uses: what about the customer who wants a friend’s opinion about a new skirt? can that customer snap a cell phone pic to send? global voices on wordpress i hadn’t heard of global voices online, a community generated global group news blog, until jeremy clarke spoke of it at wordcamp. and i didn’t think the site, with it’s do-good premise, worked until i actually explored it for a while. but, well, it’s a bit fascinating. global voices grew out of a one-day conference in december at harvard law school which brought together bloggers from around the world to discuss ways in which the new medium could foment global dialogue at the grassroots level. quercus php to java compiler vs. wordpress emil ong is the chief evangelist and a lead developer for caucho technology, the developers of the quercus php to java compiler. the idea, i guess, is to write in php, deploy in java, which some people say is better supported by the “enterprise.” ong claims % performance improvement over apache + mod_php + apc. that sounds great, i suppose, but it’s less than what chris lea suggests is possible if you simply replace apache with nginx. chris lea on nginx and wordpress “apache is like microsoft word, it has a million options but you only need six. nginx does those six things, and it does five of them times faster than apache.” —chris lea. why? no forking. no loading of unnecessary components. fast cgi. and to prove it’s not as complex as you might think, he’s installing it live. the session has eight minutes left, can he do it? yes, he did. mark jaquith on wordpress security for plugin developers i’ve been pretty aware of the risks of sql injection and am militant about keeping my database interactions clean. mark jaquith today reminded me about the need to make sure my browser output is filtered through clean_url(), sanitize_url(), and attribute_escape(). furthermore, we all need to remember current_user_can(), check_admin_referer(), and nonces. steve souders on website performance steve souders: % of the problem is server performance, % of problem is browser activity after the main html is downloaded. he wrote the book and developed yslow, so he should know. javascripts are downloaded serially and block other activity. most javascript functions aren’t used at onload. we could split the js and only load essential functions up front, and load all the rest later. how much might that help? he says % to %. will norris on oauth and diso will norris talking about things oauth, openid, and diso at wordcamp. demonstrates/fakes an oauth authentication and authorization process with wordpress for iphone app. does this matter? oauth support is slated for wp . , and people are finally getting smart about linking all this stuff without throwing passwords around “like confetti.” aaron brazell on blog search and findability aaron brazell at wordcamp is talking about search and finability “not seo.” riffing on ambient findability, he asks: can people find your blog? can people find their way around your blog? can people find your content and services despite your blog? remember: your blog serves as a nexus for information about you. you serve as the nexus for trust and relevance. going further? make your social content outside your blog searchable, findable via your blog. johnny cash’ hurt not every song johnny cash has covered turned to gold (see personal jesus), but hurt is magic. copying mysql usernames and database priveleges now that i’m the nominal mysql dba for psu, it became my job to jimmy up the mysql user privileges so that the new web server could connect. i’m not sure if this is the fastest, most efficient way to do it, but it worked quickly enough: ``` create table mysql.user_copy select * from mysql.user; delete from mysql.user_copy where host not like 'old_host_name'; update mysql.user_copy set host = 'new_host_name'; insert into mysql. wordpress performance tips elliott c. back points to his use of object caching, wp-cache, and mysql query caching among the reasons why his site “is so much faster that yours.” the iphone apps i’ve kept catherine asked me what iphone apps i recommend, so i went looking. exposure, wordpress, and google mobile app are on the first page of my home screen. mocha vnc and band are buried a little deeper, but deserve mention. i’m surprised to say that loopt and whrrl disappointed me. ipint was good for one laugh, but it appears to be gone from the store already. morocco, a decent copy of othello/reversi is the only the game that’s still on my phone. lyceum vs. wordpress mu the news about buddypress has fully shifted my attention from single-blog wordpress installs to multi-user, multi-blog installs. wordpress mu is my platform of choice, but i was quite fond of lyceum when i first learned of it a while ago. the big perceived advantage of lyceum is that it uses a unified table structure for all blogs, rather than creating a new set of tables for each blog as wpmu does. most expensive iphone app yet? armin heinrich‘s $ i am rich iphone app is no longer available on apple’s app store. perhaps they felt too ridiculed by the register to keep it listed? heinrich says seven people bought it, two by mistake. so, now what’s the most expensive app? oauth and wordpress i just realized oauth support is slated for inclusion in wordpress . . it’s not in trunk yet, but that’s no reason not to get up to speed. scott gilbertson says oauth and openid are foundations to the open social web, giving apps like wordpress a “secure, centralized means of identifying yourself and a way to control who knows what about you.” chris messina, who says we currently treat user credentials “like confetti,” is more than a little excited and is building a series of wordpress plugins to take advantage of these formats. is my php script running out of memory? i’ve got a php script that sometimes just dies with no errors to the browser and no messages in the error log. i’ve seen this in the past with scripts that consumed too much memory (yeah, it should have issued an error, but it didn’t, and increasing the memory limit fixed it), but now the memory limit is set pretty high and i’m not sure i want to increase it further. macintosh antivirus software setting aside questions about the usefulness of antivirus software for macs, it appears virusbarrier (commercial) and clamxav (open source) are the best options. there are others, of course. added: avast offers a free version for macos x as well. drill and burn republicans john mccain thinks fuel efficiency is for sissies. i guess he figures our oil supply is infinite, or that fossile fuel consumption has no effect on climate change. he probably also thinks the holocaust was a hoax — somebody should ask him. for now let’s call him a “drill and burn republican.” low-tech hdr: black card mask i’ve been following Ásmundur’s use of multi-exposure hdr for a while, but today i discovered max chu’s use of an older, more crafty technique: black card mask. the photo below show’s Ásmundur’s multiple photo technique, but that above is chu’s. how he do it? apparently it’s about the same as dodging a photo in the dark room: simply block the light with a card or your hand. extra: paul butzi’s thoughts on dodging and burning in the digital age. diy fig rig mike figgis‘ fig rig works equally well for guys in sneakers and guys in suits, but they’re not free, which is why you have to love keith lewis’ diy version. pvc is sexy! displays: go long, go wide if you want more monitors than you’ve got dvi or vga ports, your options include adding a video card, using a usb-based display, or this matrox hack: a small box plugs into your computer’s monitor port, and two or three monitors plug into the box, no software drivers or additional hardware required. if you want to send a video signal further than your monitor’s cable, your options include getting a longer cable (works up to about ′) or get a different cable. everybody’s smarter in glasses eyeglasses certainly add something. at least that’s the suggestion of these ads. and, thinking of comparisons: hitler vs. chaplin. found via mirage.studio. , where they think le corbusier‘s glasses are where it’s at. i’m voting republican no, i’m not likely to vote for any republican candidates, but this is funny. from the producers: i’m voting republican is a satirical look at the likely outcome of another four years of republican government. the not-so-subtle message behind the film is the importance of a united bloc of citizens willing to take the time and effort to vote democrat in order to improve america’s domestic and foreign policy. podcamp boston is this weekend hey, podcamp boston is this weekend. i can’t go, but sean m. brown will be and he’s looking for librarians to join him. web application design book recommendation i’ve learned to ignore contests on the web. banner ads that promise prizes if i click the right pixel are the least offensive, but the contests that have me creating content (and then force me to give up my copyright to it) for another person’s gain infuriate me. so when i saw author and experience architect robert hoekman jr‘s post offering a deal, i quickly skipped to the next entry in my reader. wordpress . notes wordpress . is out. it’s cool. take a look: i’m most excited about automatic tracking of changes to posts and pages, but i’ll also probably come to like the “press this” feature: if you click “press this” from a youtube page it’ll magically extract the video embed code, and if you do it from a flickr page it’ll make it easy for you to put the image in your post. web development languages david cloutman pointed to craiglist’s job ads as an indicator of programming language popularity. here’s the hit counts for “web design jobs” and “internet engineering jobs” in the bay area: <td> php </td> <td> java </td> <td> ruby </td> <td> python </td> <td> perl </td> internet engineering jobs <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> web design jobs <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> <td> </td> cloutman has a few ideas for what the numbers mean, but i’m just entertained by the data. wordpress . plugin and wp-config.php path changes ozh’s tutorial explains the details, but the short story is that we’ll soon get wp_content_url and wp_content_dir constants. and this is more than just convenience, . allows site admins to put those directories anywhere they want, so the constants will be the only reliable way of finding that info. truth have you ever argued with a member of the flat earth society? it’s futile, because fundamentally they don’t car if something is true or false. to them, the measure of truth is how important it makes them feel. if telling the truth makes them feel important, then it’s true. if telling the truth makes them feel ashamed and small, then it’s false. –from louis theroux‘s the call of the weird site back online, further downtime expected this site and a number of other projects are hosted on a mac mini that normally sits on my desk. thing is…my desk moved. and, unfortunately, i didn’t confirm the firewall rules for the network in my new office before bringing the machine over. thankfully chris was happy to put the mini on a different vlan, and that solved everything (my other machines remain on the new “secure” network…ugh). in the no too distant future, however, i’ll be moving the site again. video game controller family tree sock master did some outstanding work tracing the lineage of video game controllers from to now without missing any of the weirdness in between. search trends vs community standards via motherjones: pensacola residents clinton raymond mccowen and kevin patrick stevens, producers of a very nsfw website last week faced a judge in an obscenity and racketeering trial for their work. the interesting thing? the defense planned to use google search trends to demonstrate community standards. “time and time again you’ll have jurors sitting on a jury panel who will condemn material that they routinely consume in private,” said the defense. censorship, unpublishing, and new media the actual reasons may never be discovered, but boing boing, the perennially top ten ranked blog, has “unpublished (nsfw)” stories by, about, or mentioning author and sex columnist violet blue (nsfw). much has already been said about the orwellianism of “unpublishing” and how it conflicts with the ethics of the web, as well as the incongruence between these actions and boing boing’s position on web censorship, media manipulation, and revisionism. new theme for the past year or so i’ve been wanting to design a non-bloggy theme for this site — a beautiful theme with a magazine-like front page showing the most recent post in a handful of categories. but i’m further from it now than last year, so it’s time to move on. which isn’t to say that i settled for my new theme. it’s based on neo-sapien by small potato. i made it a bit wider, the header a bit shorter, and the image is random-ish (random, but cached). wordpress survey tools lorelle and samir both point to a number of plugins to do surveys within wordpress, but neither of them say any of them are that good. and samir is pretty disapointed: “at the end of it all, i never did find my ideal online survey tool.” survey fly is the best recommendation from both of lorelle and samir, but it isn’t wp . compatible and was las updated in summer . it’s also limited to tracking only one survey at a time. optimizing inserts/updates on mysql tables when doing a bulk insert/update/change to a mysql table you can temporarily disable index updates like this: ``` alter table $tbl_name disable keys ``` …do stuff… ``` alter table $tbl_name enable keys ``` from the docs: alter table ... disable keys tells mysql to stop updating non-unique indexes. alter table ... enable keys then should be used to re-create missing indexes. truemors powered by wordpress in the “they did this with wordpress” category (though from about a year ago, sorry) comes truemors, a digg, del.icio.us, reddit clone from guy kawasaki. calling it a clone might be a backhanded non-compliment, but the truth is that it does a credible job in this increasingly crowded space*. and it’s built on wordpress. the relevant plugins are wp-postratings and share this. electric pulp did the design, and the whole thing apparently went live quickly on a tiny budget. kitty porn newton isn’t really a kitten, but he is cute. anyway, i got a new video camera and all i’ve done with it so far is shoot closeups of a cat. is that why i got it? at least it’s not as bad as this. music is jungle struttin’, by the lions. programming vs. today’s computer architecture poul-henning kamp, the guy behind the varnish reverse proxy, talks about programming: it used to be that you had the primary store, and it was anything from acoustic delaylines filled with mercury via small magnetic dougnuts via transistor flip-flops to dynamic ram. and then there were the secondary store, paper tape, magnetic tape, disk drives the size of houses, then the size of washing machines and these days so small that girls get disappointed if think they got hold of something else than the mp player you had in your pocket. mysql bug? after an upgrade to mysql . . b on rhel i started seeing curious results in a fairly common query. here’s a simplified version: ``` select id, post_date_gmt from wp_posts group by id order by post_date_gmt desc limit ``` what i expected was to get a handful of post id numbers sorted in descending order by the post_date_gmt. instead, i got a list of post ids sorted in ascending order by the id number. huh. i wonder what he thinks about the iphone g? david lynch doesn’t like the iphone. at all. at least not for watching movies. maybe the guy doesn’t take the subway much. abandoned malls what is it about abandonment that’s so compelling? from chernobyl and pripyat to mental hospitals to lost theme parks from korea to california, we can’t help but stare at darkly vacant buildings. now add malls to the list. and put south china mall, in dongguan at the top of it. unlike most every other expanse of empty hallways we can name, this one’ been empty since it opened in . .shp to mysql gis data seems to come in .shp (shape?) files, but it’s not like mysql knows what to do with those. this mysql forum post points to a php tool and windows executable that promise to convert the .shp data into something more useful to mysql. superfluo explains a little more, and there’s lots of .shp data to be had here. dear steve i’m really glad to see the news about the iphone g. i’m interested in how the new mobile me service takes a small step toward cloud-based storage services that i’ve wanted for a while. and the news that max os x . “snow leopard” will focus on speed and stability, rather than features is good, especially considering the following. you see, i’m a fan of apple products. not because i like the brand, but because the products work for me. could buddypress go the distance? facebook and myspace are trying to turn themselves into application platforms (how else will they monetize their audience?). google is pushing opensocial to compete with it. but no matter what features they offer their users, they user still orbits the site. scot hacker talks of buddypress changing the game, turning “social networks” from destination websites, to features you’ll find on every website. and the “social network” is the internet, with all those sites sharing information meaningfully. detecting broken images in javascript we’ve become accustomed to link rot and broken images in nearly all corners of the web, but is there a way to keep things a bit cleaner? k.t. lam of hong kong university of science and technology came up with this sweet trick using jquery and readystate to find and replace broken images: ``` jquery('span#gbs_'+info.bib_key).parents('ul').find('img.bookjacket[@readystate*="uninitialized"]').replacewith('<img src="'+info.thumbnail_url+'" alt="'+strtitle+'" height=" " width=" " /'); ``` and it works really well, but only in ie. find stuff by minimum bounding rectangle mysql offers envelope() to find the minimum bounding rectangle of a geometric object. the result is a polygon with four segments, defined by five points. it took me a while to make sense of it, partially because the only documentation that i’ve run across so far for polygon() syntax is in the envelope() function mentioned above. i also had to draw a picture to think it through. they write this: polygon(( minx miny, maxx miny, maxx maxy, minx maxy, minx miny )), i think this (in pseudocode-ish form): polygon(( $point_a, $point_b, $point_c, $point_d, $point_a )), with the $point_s corresponding to the diagram. working with spatial data in mysql it’s mysql spatial data week here, though i am spreading out the posts to, um, ease the pain (or boredom). anyway, here are some commands/functions i don’t want to forget about later: start with an existing table called geometry, add a spatial column and index it: ``` alter table geometry add coord point not null; create spatial index coord on geometry (coord); ``` insert some data; think in terms of point(x y) or point(lat lon): bsuite beta i announced the bsuite public beta not long ago, now i’ve just posted a new version to svn that addresses some of the bugs and fleshes out some of the features. i have yet to update the bsuite page, but here’s a preview of what’s new or changed: additional stats reports wp . -style tag input tools on the page edit screen* wp . -style category selector on the page edit screen* wp . calculating distance between points in mysql mysql has some powerful, and perhaps underused spatial extensions, but the most interesting functions are still unimplemented: “note: currently, mysql does not implement these functions…” among those as-yet unimplemented functions is distance(). alternatives can be found here and here, though neither is clean or simple. i wonder if a simple mbrcontains() is good enough, though… anticipating steve jobs’ wwdc keynote will it be a thinner or fatter iphone? will it record live video? will it have a metal cutting laser? to heck with the iphone rumors. we know the story, all we’re waiting on are the details. i’m more interested in what we don’t know. what aren’t we expecting? will there be “one more thing”? (thanks to roblef for the sweet photo.) mysql documentation found in the mysql . reference manual: related(g ,g ,pattern_matrix) returns or to indicate whether the spatial relationship specified by pattern_matrix exists between g and g . returns – if the arguments are null. the pattern matrix is a string. its specification will be noted here if this function is implemented. (emphasis mine.) converting a wp.org site to wpmu i have a lot of wordpress sites i manage and i’ve been thinking about converting them to wordpress mu sites to consolidate management. today i attempted the first one, about.scriblio.net. there’s no proper way of doing it that i found, but here’s what i did: create a new site in mu create the users in the correct order (user id numbers must match) replace the posts, postmeta, comments, terms, term_taxonomy, and term_relationship tables with those from the original blog copy the contents of wp-content/uploads to wp-content/files update the posts table with the new path (both for regular content and attachments, see below) hope it all worked somebody is likely to say “just export the content in wordpress xml format and import it in the new blog,” but that person doesn’t use permalinks based on post_id. bsuite public beta i’ve had a lot of features on the table for bsuite for a while, but this recently discovered comment from john pratt (whose smorgasboard.net is a lot of fun), kicked me into gear to actually get working on it again. the result is bsuite , which is probably what bsuite should have been all along. the big news is that i’ve finally revamped stats tracking to work with caching mechanisms like wp cache, wp super cache, varnish, or whatever else. json on rhel & php . . stuck with php . . on rhel or even centos (and a sysadmin who insists on using packages)? need json? i did. the solution is easy: yum install php-devel<br /> pecl install json the pecl install failed when it hit an mb memory limit, and i was clueless about how to fix it until i learned that the pecl installer ignores the php.ini. turns out the best solution is to use the pear installer (which does follow php. happy birthday wordpress wordpress was released to the world five years ago today. celebrate in sfo, sydney, or with me at whatever bar i find myself at in new hampshire tonight. dm me with any ideas. another gun control analogy “gun control is like trying to reduce drunk driving by making it tougher for sober people to own cars.” via many eyes, bugs being shallow, all that wordpress . . added a really powerful feature to register_taxonomy(): automatic registration of permalinks and query vars to match the taxonomy. well, theoretically it added that feature. it wasn’t working in practice. after some searching yesterday and today, i finally found the bug and worked up a fix. i made a diff and set off to open a ticket in trac. on the one hand i’m glad i searched first, because it turns out that a ticket on the very same issue was opened on may th and it already has a fix. where do they find the time? clay shirky recently posted (wayback) a transcript of his web . expo keynote. …if you take wikipedia as a kind of unit, all of wikipedia, the whole project — every page, every edit, every talk page, every line of code, in every language that wikipedia exists in — that represents something like the cumulation of million hours of human thought. then shirky asks us to compare that to television. roflcon turns me on to ustream.tv i was amused to learn nathan was officially at roflcon on behalf of his library. i wasn’t representing my work and wasn’t on the lookout for work-related tools, but i found some anyway. universities have been anxious to get into live video casting for a while. our first effort eventually became pbs (net, ets and pbs histories). later, we invested huge amounts of money in interactive television (itv), but enormous costs and complexities limit the use of such facilities. anglia ruskin university faces criticism . anglia ruskin university is in cambridge, but it’s not cambridge university. it’s likely that none of us would even know of anglia ruskin‘s existence if it wasn’t for naomi sugai, but she’s not interested in promoting the school. she’s got complaints, she’s fed up, and she’s taking her case to youtube. well, she took her case to youtube, and then she got suspended. the video that’s up now doesn’t seem suspension-worthy, but the telegraph story suggests there’s a different version that may slander an aru administrator, and that’s the reason aru gives for suspending her. honda civic ipod/iphone install last weekend, while i was putting an ipod interface into my scion i did the same thing for my honda civic. using ben johnson’s story as a guide, i bought a pie hon -aux interface and dove in. aside from tools (screwdrivers and and mm sockets), you’ll need: the interface adapter audio wiring — i used a ′ rca to / th inch cable from radio shack power — i used a belkin car charger plugged into this v extension cord i picked up from radio shack i also recommend a sufficient quantity of good beer or other beverage. snakes on a plane it was only after i’d taken my seat and david weinberger began his roflcon keynote that i realized there was a box of t-shirts at the side of the room with a sign over them that said something along the lines of “free: t-shirts from worn out memes.” thinking that the internet might be old enough now that the old memes might be resurrected in some ironic way, i almost jumped over jessamyn to rifle through the box and claim a prize. retro atari video game cover art sure you played asteroids and defender, but did you play these? scion xb ipod/iphone install based on this story about an ipod interface install i purchased a pie toy -aux aux input adapter so i could finally listen to my iphone without using the lousy fm transmitter. sure, i coulda bought a new car, as the manufacturers seem to have finally come to their senses and started including such inputs, but i refuse to buy another car until i can have one that gets well over mpg. barbed wire, the deeper history of it turns out that, like most everything else, barbed wire shows up at auctions. not just shiny new stuff, you’ll find used stuff too. expect it to be at least a little rusty, and look out for clumps of hair or other things stuck to it. whether that adds value or not is unclear. where could we look to find out? the antique barbed wire society‘s barbed wire collector magazine might be your best source. my flickr complaint some whine about movies on flickr, others about the switch to yahoo ids, i simply want better rendering of transparent pngs as jpgs. cats want to eat your brains nyt: parasites in your brain are driving you to raise cats in hopes that they eat you. hat tip to cliff. flickr adds video i asked for it in , before youtube, vimeo, viddler, or revver appeared on the scene, and before myspace and facebook added video sharing as a feature. four years later they finally added it. neil rickards should get credit for creating the theme of “long photos” (neil called them “moving photos”). and anybody who was around then isn’t the least surprised at how angry some are now about the new feature (see sarcastic response to that). the internet, according to mememolly identity management going commodity? atlassian’s crowd sso and idm solution has the kind of online pricing you’d expect for word processing software. i don’t know if it’s any good, but it’s a sign that identity management getting boring. why can’t i re-check spam with akismet & wordpress . ? (workaround) i recently installed wordpress . and among the changes i noticed was a loss of akismet‘s “recheck spam” button (or something like that. it didn’t seem like such a problem at the time, but then i got swamped with so much trackback and comment spam that the flood dos‘d my server. i had to disable comments and trackbacks for a time, which brought my server back, but my moderation queue still had over comments waiting for me. christian nymphos not that you’d mistake our sites, but christian nymphos uses the same theme i use here at maisonbisson. well, i modified the theme quite a bit for my use, but…. well, regarding the title of the site: pastor bob snowdon probably approves of any and all efforts to reclaim “nympho” from its pejorative purgatory. cargo aircraft safety who knew fedex and ups planes crashed so often? (blame the intronetz for making this too easy to discover.) ups plane catches on fire, lands in philadelphia ( ). apparently the source of the fire remains a mystery, as with a few other ups fires. fedex planes have crashed and burned in tallahassee ( ) and memphis ( ). in a fired fedex pilot attempted to murder flight crew with hammer and hijack the plane. swift: another ham handed attempt at social networking all yesterday and this morning i’ve been seeing tweets about swift, so i finally googled it to see what it was about. the service promises to help organize conferences in some new . way, but it looks to be about as preposterous a social network as walmart’s aborted attempt at copying myspace. there are some real lessons here, however, about how to court the early adopters that are essential to making an application that depends on user activity successful: wordpress . out, maisonbisson upgraded wordpress . is out (and the wordpress site got a facelift), and i’ve already upgraded maisonbisson using svn. the changes are exciting, and seem to reflect a tradition that’s developing in wordpress of delivering some really revolutionary features in the x. release. the loss of file-based object caching was a bit of a problem, as my vps‘s load average jumped to over pretty quickly after the upgrade. i tried mark jaquith‘s apc-object-cache enabler and saw load average drop back to or so, but i also saw tag and category names disappear and discovered other weirdness. make your own sign i had fun with the signs in taiwan (jet powered baby stroller and men’s bathroom signs, for example), but why travel around the world for these things when you can make them at home? create warning signs, protest signs, church signs, library catalog cards, or whatever. tibet open letter and other innovative uses of wordpress all things digital is interesting. parents would say my baby our baby.com is a little more important. but tibet open letter is as real as the violence. two things to note: all of them are based on wordpress, and those who discuss tibet probably risk being listed by the chinese government as a trouble maker. evil google aaron swartz‘s bubble city, chapter : he sent the report to his superior and wandered off for a bit to dwell on the power he had as a faceless person deep inside an office park in mountain view to know every detail of another person’s life. he wondered what it would be like if he came across that person on the street, he would know every detail of his life, his household budget, the secrets he confided over im, even what he looked like naked. interesting wordpress plugins wp contact manager turns wordpress into a contact manager. it’s a combination of theme and plugins (including custom write panel) that allows you to enter and manage contacts as blog posts (familiar, eh?). use members only to secure access. tdo mini forms “allows you to add highly customisable forms to your website that allows non-registered users and/or subscribers (also configurable) to submit posts. the posts are kept in ”draft“ until an admin can publish them (also configurable). best restaurant in taipei i ate here. it’s every bit as good as the review suggests. seb’s description and photos tell more, i’ll post my own photos soon. update: posted. short story: there’s a restaurant in australia with a three month waiting list, but a sydney morning herald reporter says the restaurant i ate at is its equal or better, but without the waiting list and at us$ per meal. google pagerank is/is not/is all machine generated google’s always been in the awkward position of claiming that pagerank is algorithmic, not editorial, while also explaining that they’re constantly adjusting their algorithms to ensure that pagerank reflects editorial judgments of quality. here’s a peek inside the machine. zach houston’s poem store walking north on valencia i heard the characteristic snap snap snap of an old manual typewriter’s hammers striking paper on the platen. i was more than a bit curious about who might still use such a classic machine even before its operator called out to ask if i wanted to buy a poem. still, it’d been a full day exploring the mission with a fabulous host and the time for my flight home was nearing. no mo w stolen from jessamyn‘s photostream. where the previews are i announced yesterday scriblio‘s integration of google’s new book viewability api that links to full text, previews, or additional book information (depending on copyright status and publisher foresight). now that it’s live with plymouth’s full catalog, i spent a moment browsing the collection and taking note of what books had what. i get no preview for a baby sister for frances, but another of russell hoban‘s books, a bargain for frances. scriblio integrates google book search links (crossposted at scriblio.net) using the newly released book viewability api in google book search, plymouth state university’s lamson library and learning commons is one of the first libraries to move beyond simply listing their books online and open them up to reading and searching via the web. take a look at how this works with books by plymouth authors bruce heald and joseph monninger. the “browse on google” link in the new features section leads to extended previews of their works where you can browse excerpts of the books and search the full text. great name, but is it any good? “spork” is a great name for a restaurant, but is it any good? yelp says it is, but most of the reviews mention the burger, putting me in the position of having to review the reviewers and wonder if a hamburger person can recommend a restaurant to a vegetarian. not that i am a vegetarian or not a hamburger person, but please tell me there’s more to the retrofabulous-looking place than a cool name and a hamburger. geographic tweeting twittervision and twittermap show new tweets wherever they appear on the map, twitterwhere let’s you follow tweets at a specific location, and ask people has nothing to do with twitter but does show you global opinion. live. while you watch (so they say, anyway). warming if this doesn’t warm your heart, check to see that it’s not made of stone. netflix for audio books netflix for audio books: simply audiobooks. though it makes me wonder why we don’t say “like a library for audiobooks where they send you the stuff you want.” wordpress . offers built-in gravatar support nobody doubted that full gravatar support would make it into wordpress eventually. weblog tools collection shows what they look like, how they’re managed, and how theme designers can implement them. quaint vs. libraries this slashdot post asks the same question a lot of people do: “can libraries be saved from the internet?” slate has an interesting photo essay exploring the question of how to build a public library in the age of google, wikipedia, and kindle. the grand old reading rooms and stacks of past civic monuments are giving way to a new library-as-urban-hangout concept, as evidenced by seattle’s starbucks-meets-mega-bookstore central library and salt lake city’s shop-lined education mall. buddypress: the wordpress of social networks? andy peatling, who developed a wordpress mu-based social network and then released the code as buddypress has just joined automattic, where they seem to have big plans for it. i’d been predicting something like this since automattic acquired gravatar: it’s clear that the future is social. connections are key. wordpress mu is a platform which has shown itself to be able to operate at internet-scale and with buddypress we can make it friendlier. parse html and traverse dom in php? i spoke of this the other day, but now i’ve learned of php’s dom functions, including loadhtml(). use it in combination with simplexml_import_dom like this: ``` $dom = new domdocument; $dom->loadhtml(' one two three sublist item ' ); if($dom){ $xml = simplexml_import_dom($dom); print_r($xml); } parse html and traverse dom in php? i love how easily i can traverse an html document with jquery, and i’d love to be able to do it in php. there are a few classes, but the php binding for tidy seems to be where it’s at. the zend dev pages make it look that way, anyway. movable type to wordpress scot hacker (yes, that’s really his name) posted a story about migrating china digital times (published by berkeley school of journalism) from movable type to wordpress: we’ve launched with a lovely new design, reduced story publishing times from by orders of magnitude, been able to re-enable a bunch of features we’d previously had to disable for load reasons, and added new features that were never possible before. the team of authors and editors is in heaven, and i’m considering bringing the site back onto the main j-school server. scriblio feature: text this to me take note of the “new feature: text this to your cellphone” line above. adam brin of tricollege libraries explained that the “text this to me” feature he built to send location information about items in the library catalog as text messages to a user’s cell phone is being used as many as times a day. that was the news i needed to decide to offer the feature in psu’s scriblio implementation. web design frameworks? i’m a fan of the sandbox wordpress theme because it does so much to separate application logic from design, and a few small changes to the css can make huge changes to the look of the site. i think that’s the idea behind yahoo! developer network’s grids css library. that is, well structured html allows very sophisticated styling. all you have to do is plug in your content. to wit: give up your civil rights (and your laptop and hard drives) at the border can the feds take your laptop? yep. be prepared to give up your civil rights and your laptop at the border, says a recent article in the washington post. this came to the attention of music fans earlier, when mtv news reported that a hard drive seized at the border contained studio recordings for chris walla’s (guitarist for death cab for cutie) latest album. there was some suggestion that it was all a publicity stunt, but the post story suggests that it’s a real and not uncommon problem. apache reverse proxy apache mod_proxy does most of the work, nick kew’s howto on running a reverse proxy with apache explains it. now, can i tack on some authentication and make it replace iii’s wam or ezproxy? moscow subway’s underground palaces photographer farflungphotos describes: all the stations in moscow’s metro are completely different from one another. some of them are so opulent, with grand marble halls and chandeliers, all hidden away underground. people seemed to be using them as places just to hang out and meet up with friends. the trains were really frequent too, practically on each others tails. you never have to wait more than a few minutes for one to come along. western north carolina library network’s classification outline western north carolina library network‘s lc outline is full of detail. lc outline, classification, western north carolina library network, libraries changes to wordpress object caching in . jacob santos‘ funcdoc notes: the wordpress object cache changed in wordpress . and removed a lot of file support from the code. this means that the object cache in wordpress . is completely dependent on memory and will not be saved to disk for retrieval later. the constant wp_cache also changed its meaning. i’ve just started using the object cache and i’m happy with how it works now, so these changes are somewhat concerning. iphone strobe light strobe light is clearly the perfect app for your new gb iphone. mysql on multi-core machines the devshed technical tour explains that mysql can spawn new threads, each of which can execute on a different processor/core. what it doesn’t say is that a single thread can only execute on a single core, and if that thread locks a table, then no other threads that need that table can execute until the locking thread/query is complete. short answer: mysql works well on multi-core machines until you lock a table. looking ahead from : top tech trends i’m excited and honored to be joining meredith farkas and david j. fiander in a roundtable discussion of top tech trends, an olita program at superconference. we’ve made a pact not to share our trends with each other in advance (no peeking), so it’ll be interesting to see how much overlap we have and how differently we approach the issues where we do have overlap. sophistication the search box with its flashing cursor is a powerful tool, but it’s positively pre-cambrian when compared to our hyper a/v culture. ola superconference presentation: scriblio i’m honored to be invited to the ontario library association superconference to present my work on scriblio today (session # ). a pdf of my slides is online. scriblio has had about a year of use in production at each of three sites, and the lessons suggest that web . technologies really do work for libraries. and the best news: we can do it without breaking the budget: i’ll be demonstrating how to install scriblio and reinvent a library in about ten minutes. microsoft threatens to buy yahoo! i like yahoo!. i really hope the shareholders decline microsoft’s offer. blech, ms has wanted a piece of yahoo! for a while. never forget, - - paranoia if it’s not an american flag, it’s probably a bomb. what do coots eat? turns out that coots are omnivorous, but prefer plant matter. why. forget time capsule, i want a space ship apple’s time capsule is great. seriously. when has backup been easier? but i need more. the macbook air’s small storage highlights a problem i’ve been suffering for some time: there’s never enough storage. the slower processor and limited ram expansion are sufferable, but storage isn’t. the gb drive in my macbook pro now is stuffed with gb of music (and that’s after spending hours paring it down a few weeks ago), and almost gb of pictures. camera found in cab starts digital goose chase what would you do if you found a camera in a cab? lcsh news: “mountain biking” replaces “all terrain cycling” even though mountain bike sales and participation are down (as a percentage of market share, biking has been declining for ten years), the library of congress has just issued a directive to change the subject heading from “all terrain cycling” to “mountain biking.” the term was apparently first coined by charlie kelly and gary fisher in . stephen king doesn’t hate kindle stephen king writes at entertainment weekly.com that he doesn’t hate the kindle: will kindles replace books? no. and not just because books furnish a room, either. there’s a permanence to books that underlines the importance of the ideas and the stories we find inside them; books solidify an otherwise fragile medium. but can a kindle enrich any reader’s life? my own experience — so far limited to . books, i’ll admit — suggests that it can. mcqualifications bruce pechman earned his credentials, but you could get yours at mcdonald’s. yes, the fast food chain is apparently offering diplomas in britain now. dangerous grains call for drastic measures “the office of emergency management, the new york city fire department, department of buildings, nypd, health department, and department of agriculture” all apparently showed up to evict tenants from a building called the “kibbutz” in the williamsburg section of brooklyn. why? “dangerous grains,” and a matzoh bakery. it’s been labeled matzo-gate, and speculation is rampant that the eviction was spurred by developers eyeing the now fashionable neighborhood. gothamist has a picture. apache, mysql, and php on macos x p ps harlow tweeted something about trying to get an amp environment running on his mac. conversation followed, and eventually i sent along an email that look sorta like this: if you’re running . (i doubt it, but it’s worth mentioning because i’m most familiar with it), here’s how i’ve setup dozens of machines for web development and wordpress: install mysql http://dev.mysql.com/downloads/mysql/ . .html#macosx-dmg install marc liyanage’s php package usability experts are from mars, graphic designers are from venus this an old one, but it just caught my atention. in a list apart tells us usability experts are from mars, graphic designers are from venus. is this still true? haven’t the last several years been about the triumph of good design in both the usability and graphic senses? or are rounded corners not actually useful? dancing with the nerds richard stallman‘s soulja boy dance, mit style (via). wordpress to_ping query optimization the wordpress team has taken up the issue of performance optimization pretty seriously, and i look forward to the fruits of their efforts, but i’m also casting a critical eye on my own code. thanks to caching and a hugely optimized query architecture, scriblio is now performing better than ever, and i’m now looking at the next tier of problems. first among them is a wordpress query that runs to find which posts have pingbacks or trackbacks waiting to be processed. this would _so_ cramp my style the new hampshire house is considering a ban on texting while driving. please, no. even cheetah moms have to argue with kids about dinner mother cheetah wants kids to learn to hunt gazelle, but cubs want to nuzzle it. signs of user-centric shift at ces? doc searls in linux journal compares previous ces expos to and finds a shift from talk of “broadcasters and rights-holders extending their franchise” to a web . enlightened user-centricity. at every ces up to this one, i always felt that both open source and user-in-charge were swimming upstream against a tide of proprietary “solutions” and user lock-in strategies. this year i can feel the tide shift. lots of small things point toward increased user autonomy, originality, invention and engagement. introducing phonepedia, a voice-activated wikipedia mashup the phonepedia concept is simple: take wikipedia’s rich content and add voice recognition. it’s as easy as calling a number and asking your question, the answer will be returned via sms and email. go ahead and try it for yourself. phonepedia. the voice recognition is powered by jott, and thanks are due to heidi for writing so glowingly about it (cluetrain moment: i’d heard about jott before, but hadn’t been stirred to look at it until i saw heidi’s post speaking in the voice of a real person). like mr. ranganathong said… like mr. ranganathong said: “the intellect cannot be tied down with a decimal thong.” (via) i can haz ice cream and booze? this thread says you can get booze and ice cream in the same joint! places to know in nyc: otto, the chocolate room (beer & wine only?), chikalicious, clinton street baking company, blt burger, homer’s, and liquor & ice cream. staring contest shirow masamune himself couldn’t draw manga eyes like hers. google pumps openid too following news that yahoo! is joining the openid fray, it appears google is dipping a toe in too. while those two giants work out their implementations, others are raising the temperature of the debate on idm solutions. stefan brands is among the openid naysayers (<a href="http://daveman .livejournal.com/ .html” title="david recordon’s blog - stefan chooses to take the “fox news” approach to openid blogging">david recordon’s response), while scott gillbertson sees a bright future. let’s watch the openid directory to see how fast it grows now (count on january : ). harvard film archive’s wild movies of s pre-code films were apparently something of a spectacle. harvard film archive this weekend is exploring their depths in a series titled vice vs. virtue. just in case anybody else wond… just in case anybody else wonders why a wordpress initiates extra mysql activity http://tinyurl.com/ nkplo balloon organ, yes, a balloon organ in a piece that will have some people eagerly looking for some afro celt sound system, others singing where do they make balloons, and some people just shaking their heads, this fellow, apparently standing in his bathroom, introduces us to another guy and his balloon organ. really. check this for more homemade organ fun. eccentric chess champ bobby fischer dead eccentric, perhaps persecuted, bobby fischer is dead. news story. wordpress + invalid urls = extra database queries after reporting weirdness last week i finally sat down with a completely clean and virgin install of wordpress . . and traced what happens when you make a permalink request for a non-existent url. here are two sets of urls to use as examples and context: these are valid urls: http://site.org/archives/ http://site.org/page-name these are _not_ valid urls: http://site.org/archivezorz/ http://site.org/favicon.ico valid urls get parsed, the expected mysql queries get executed, and the results are processed and returned to the browser. yahoo! pumps openid ars notes that yahoo! supports openid. yeah, that openid. southwest’s in-flight magazine doesn’t suck, they say derek powazek likes it, but is it worth flying southwest for? @jblyberg: i had to look it up… @jblyberg: i had to look it up a while ago too http://tinyurl.com/z sg sifting results of error_log( … sifting results of ``` error_log( $_server['request_uri'] ."\n". $_server['remote_addr'] ."\n". print_r( debug_backtrace(), true ) ); ``` trying to figure out why wp hi… trying to figure out why wp hits db for all posts query _after_ it determines the url is a is facebook really the point? a post to web lib alerted me to this u mich survey about libraries in social networks (blog post) that finds % of students don’t care for or want libraries in facebook or myspace. the biggest reason being that they feel the current methods (in-person, email, im) are more than sufficient. % said no because they felt it was inappropriate or that facebook/myspace is a social tool, not a research tool. @tinfoilraccoon: take the pled… @tinfoilraccoon: take the pledge: http://tinyurl.com/ x qye @tinfoilraccoon: is it really … @tinfoilraccoon: is it really so complex that it requires training? pls tell them amazon and itunes don’t require training, ask why od does. fancy up your website with web clip icons aaron schmidt alerted me to this how to sweetening up your site with fancy iphone web clip icons. impeach cheney now you’ll feel better after signing the petition. bits of mysql query syntax i’ve learned this week watching the wordpress hacker list this week, a couple messages related to selecting information about users schooled me on mysql syntax. i obviously knew the following would work, but i’d previously used the union syntax in similar situations and somehow hadn’t thought of writing it this way: ``` select (select meta_value from wp_usermeta where meta_key = 'first_name' and user_id = ) as first, (select meta_value from wp_usermeta where meta_key = 'last_name' and user_id = ) as last, wp_users. user posts antisemitic content… user posts antisemitic content to wikipedia, then crosses out my comment in the requests for deletion page!?!? http://tinyurl.com/ytt zh @edventures: their hardware an… @edventures: their hardware and operating system operations are getting squeezed. they’ve gotta look elsewhere. i like mysql. i like sun. this… i like mysql. i like sun. this could work well: http://tinyurl.com/yr rl tried sleep, failed. surfing w… tried sleep, failed. surfing web oniohone in bed while sandee sleeps soundly. just a tiny example of a commu… just a tiny example of a community trying to figure out its boundaries http://tinyurl.com/ytt zh drove home clicking iphone map… drove home clicking iphone maps locate button like walt mossberg on meth. works great in cities, crap in woods new iphone maps locate circle … new iphone maps locate circle has yet to locate me macbook air is sealed like ipo… macbook air is sealed like ipod. can’t replace battery, no ram upgrades. iphone update finally download… iphone update finally downloading. not leaving office until i get a locator button on my maps. iphone update server overloade… iphone update server overloaded nh primary fraud? two very important things: i have every confidence that the nh primary results were correct and accurate, and, most importantly, unmolested. and, i’m also quite happy with them. but that doesn’t mean i’m not anxiously awaiting the results of the hand recount that congressman kucinich has requested. conspiracy theories abound, and diebold is a despicable company worthy of general derision, but at least our accuvote os machines have paper ballots. @awd: wasn’t sure if there was… @awd: wasn’t sure if there was a specific meeting your sarcasm was directed toward, though i’ve been following the drama all along @mstephens : bring cigars and … @mstephens : bring cigars and ask if prez has has scotch in the office? getting ready for the stevenote i can’t go to the parties laughing squid names, and world of apple’s live video coverage seems about as likely as a kucinich becoming president, but the unofficial apple weblog‘s keynote predictions are out, ars’ keynote bingo is set, and half the blogaverse will likely offer some updates about the action, some of them live. the stevenote is coming, and at the end of the day, or at least later that day, it’s likely that apple will broadcast the recorded event in quicktime (judging from this url, you might find it here). dead men don’t cash checks virgilio cintron was the happiest corpse in the city… chris “long tail” anderson on open source open source and the long tail: an interview with chris anderson the shift of software from the desktop to the web will really be the making of open-source software. the long tail side of software will almost certainly be web-based because the web lowers the barriers to adoption of software. there will always be some software best delivered as packaged bits. but the big problem with packaged software–or one big problem–is the risk associated with installation. how do i create a semantic web site? a member of the web lib mail list asked: how do i create a semantic web site? i know i have to use either rdf or owl but do i use either of these to create a mark up language which i then use to create the web site or, with the semantic web do we move away from mark up languages altogether? am i right in thinking that owl and rdf do not contain any information on how the document is to be displayed or presented? live in mehran karimi nasseri, sanjay shah and alex ervasti all made their names living in airports. now, comedian mark malkoff is hoping his one week stay at the paramus, nj ikea store will do the same. the state of democracy what does it mean about the state of democracy when viral video darling obama girl amber lee ettinger shows up in nh? and chuck norris too? (chuck norris political facts.) it probably surprises no one that kucinich’s press secretary’s year old daughter is more articulate than amber and chuck combined. ugh. wordpress admin redesign progress happy cog‘s liz danzico introduced it at wordcamp (her slides are online), but it’s been only recently that the fruits of the admin control panel re-thinking have started to appear in code. though there’s much work yet to be done and it’s not uncontroversial, i think i like it. maisonbisson chocolate martini the holidays are past, but we still have a sweet tooth here. chocolate shavings for rimming part crème de cacao parts vodka dark chocolate garnish warm a martini glass over a small flame, then roll the rim in chocolate shavings. put a square of dark chocolate in the glass, then prepare the liquor. shake vodka and crème de cacao with ice and strain into glass. for additional flavor, sprinkle the top with cocoa powder or chocolate shavings. wiimote (wii remote) + projector + computer = homebrew multitouch display you’ve got the hardware, you’ve got the skills, go build a multi-touch electronic whiteboard with your wiimote and a data projector. building in a (big) bubble dcdead‘s photo of the central station of strasbourg, france reminds me of something i’d long wanted to do in (or around) my old house: put it in a dome. apparently, this dome doesn’t fully cover the building, just enlarges it without obscuring the facade. still, square meters of glass looks pretty good, eh? back to my old house, however. here’s the plan: forget the lack of insulation and the drafty windows (and the dying roof, before i replaced it), solve all of that by putting a greenhouse up around it. wordpress . performance, timeline the good news is that performance is a big goal for wp . , the bad news is that it’s been delayed to the end of january at the earliest. gmail imap vs. previous pop users google mail now supports imap, but what if you’ve been using pop all along and have a gajillion messages on the server, all marked unread and waiting in your inbox? how can i tell apple mail not to download the [gmail]/all mail imap folder without an ugly hack? [update, the hack just causes mail to crash a lot.] free report on accessible web design from jakob nielsen free from nielsen norman group: beyond alt text, making the web easy to use for users with disabilities, a report on web design for users with disabilities. “seventy-five best practices for design of websites and intranets, based on usability studies with people who use assistive technology” according to the blog post, usability is three times better for non-disabled users. bsuite machine tags there can be no arguments about it, machine tags are cool and they solve problems. and now they work in wordpress with bsuite too (svn only, for the moment). it’s not just because flickr popularized them that i like them, though it helps and you should definitely look at that stuff: the announcement excitement from o’reilly radar, programmableweb, and dan catt (who championed the concept at flickr, i think). inside your head video found via a photo in soffia gisladóttir‘s photostream. the suggestion that things go rotten inside a person’s head is very sad, but i’ve also suggested it to zach for moldy snack.com css transparency settings for all browsers ``` .transparent_class { opacity: . ; /* the standards compliant attribute that all browsers should recognize, but... */ filter:alpha(opacity= ); /* for ie */ -khtml-opacity: . ; /* for old safari ( .x) */ -moz-opacity: . ; /* for old skool netscape navigator */ } ``` (via) a boy and his cabbage of significant size from the la crosse tribune, a boy and his cabbage of significant size: wisconsin ten-year-old douglas mezera grew a -pound cabbage for a competition sponsored by bonnie plant. the alabama plant company’s program aims to promote gardening as fun and rewarding. what do you do with so much cabbage? “we made it into homemade sauerkraut,” douglas’ mom said. “it’s good.” (via) language translation icon we all need a recognized icon to represent “translate this.” we’ve got one for feeds and social bookmarking, but where’s our translate icon? a lot of folks simply use flags, but that’s a bad idea because they’re “nationalistic, and represent ideals, boundaries, and political beliefs, but do not represent a language.” joe lee has developed a few icons for use in the olpc project, and they look good. the only problem i have with them is in trying to make them work at × pixels. in flight wifi back in the air? i thought the matter was dead after boeing shut down their much hyped in-flight wifi plans (yep), but engadget got a seat on jetblue’s private introductory flight for their wifi service. the good news is that it’s free, the not surprising news is that yahoo! is partnering in it (and it requires a yahoo! account), the bad news is that all you get is yahoo! im and email. no web browsing, or anything else useful. scriblio . v released scriblio . v is out. see it. download it. install it. join the mail list. what’s new? lots of small bug fixes. implemented wp_cache support. revamped sql query logic for better memory efficiency. new widget options. search suggest/autocomplete support (implemented in the new theme). new theme. new theme! by jon link. home libraries, amateur libraries the library problem: in march of my wife mary and i owned about , books. we both have eclectic interests, voracious appetites for knowledge, and a great love of used bookstores. the problem was that we had no idea what books we had or where any of them were. we lost books all the time, cursed late into the night digging through piles for that one book we knew must be there, and even bought books only to find that we already owned them. usb-connected monitors? displaylink is licensing technology that promises to make adding a second (or sixth) monitor as easy as plugging into a spare usb port. samsung’s ux “ lcd (under $ , review) is among the first to employ it, though iogear’s usb to vga adapter is also available (about $ , review). this isn’t without problems, though. image quality is said to be sharp until it moves, then it stutters and chops, more from cnet labs. seven person bicycle: the conference bike i saw this bike here, here, and here on flickr, but nobody said what it was or where i could learn more. some googling revealed it was eric staller’s conferencebike, first sold by hemmacher schlemmer. one person steers while all seven riders peddle, and it looks like a lot of fun if you’ve got a spare $ , . the eight foot long bike is six feet wide and weighs about pounds. compress css & javascript using php minify it was part of a long thread among wordpress hackers over the summer and fall, but this post at vulgarisoverip just reminded of it: minify promises to be an easy way to compress external css and javascript without adding extra steps to your develop/deploy process. no, really, look at the usage instructions. (to be clear, the vulgaris and google code versions are different, one derived from the other and backported to php compatible. old romans knew how to make glue we’ve known about the birch bark glue romans used on their clay pots and jars for a while, but now researchers in germany are calling it “caesar’s superglue.” researchers at the rhine state museum in bonn apparently found it used to bond silver plate to an iron helmet in a year old repair job. the superglue part: the bond was still good. people make scriblio better it’s way cool to see lichen‘s scriblio installation instructions translated to hungarian. even cooler to have sarah the tagging librarian take hard look at it and give us some criticism (and praise!). but i’m positively ecstatic to see robin hastings’ post on installing scriblio (it’s not easy on windows, apparently). part of it is pride in seeing something that i’ve been working on for so long finally get out into the world, but scriblio really does get better with every comment or criticism. roadside attractions fading away? roadside attractions fading from landscape: a staple of the american road trip could be slowly disappearing from the nation’s interstates and byways. owners of some roadside attractions are deciding that interest is waning bsuite released [innerindex]i started bstat in when i ported my blog from pmachine to wordpress and needed to bring over the tools i’d built to identify popular stories and recent comments. i renamed it bsuite when i added tagging and other features to it. now it’s bsuite . get it here. get installation details here, and users of previous versions should check the upgrade instructions here. features tracking of page loads for each post and page. my iphone commercial (or, the night we almost died on a mountain) it was cold. the air carried no scent, ice squeaked under our boots, and every little leaf and twig crinkled and snapped as we walked over it. but this was louder than that. much louder. neither jon nor i saw it actually happen, but when i found will he was mostly upside down between a boulder and tree. the trail at that point was elevated by some rocks and bordered by pines that grew from the forrest floor some distance below. tabbed chatting in ichat among the missing features i hear the most complaints about regarding ichat is the lack of tabbed chatting. today i discovered it’s part of leopard. simply go to the ichat prefs, click on the messages pane, and selected “collect chats into a single window” and you’re set. a nation marketing itself japan‘s the ministry of foreign affairs english-language web japan is a bottomless trove of in-flight magazine-quality stories like antibacterial epidemic and j-culture-hyping love-fests like honoring the world’s manga artists. if american propaganda efforts are this bad, why do foreign governments even bother blocking them? is this really worth protesting? it can only be taken as evidence of our wealth and privilege that two years after macy’s bought marshall field’s people are planning a black friday rally and holiday boycott to protest the name change. wp rewrite instructable dan’s instructable for custom rewrite rules in wordpress is better than the docs in the codex. how expensive does commercial software need to get before we consider open source? open source software of the free as in free beer and free as in free speech variety has matured to the point that there are now strong contenders in nearly every category, though that doesn’t make them easy choices. it’s often revealing when people criticize oss as being free as in free kittens, which is true in the sense that f/oss does require continued care and feeding to make it work, and false in that it suggests commercial solutions don’t. themes i like matt has updated his site with a less blog-like front page and i just discovered unsleepable, which is very bloggy, but seems like a good start for what i want to do next. remix remix remix: the tracey fragments i guess the criticism is that it’s one thing for somebody to open up their music for remixing, but an entirely different thing to do the same with a movie. or is it? is it (click re-fragmented)? [insert word here] is hurting your network corporate networks are defenseless against the growing threat from instant messaging, and the government warns wifi is insecure and easily sniffed. experts suggest we take precautions against the growing risk of p p software that’s exposing sensitive documents and threatening national security. businesses blame security problems on their employees, their mobile devices, and other consumer technologies. and now we have myspace. tidens hotteste it-trends my presentation for today’s hottest it trends is nearly completely new, though it draws a number of pieces from my building web . -native library services and remixability presentations. what it adds is an (even more) intense focus on the people that make up the web. denmark is among the most wired countries of europe, and it’s especially interesting that more than half of danes over use the web at least once a week. remember the good old days? the first article database i remember using was dialog, sometime in the late s or early s. today i found myself amused that we used to call such things “interactive.” that is, you poked the command line interface with questions and it usually beeped a syntax error, all while they charge $ per minute, plus the connection fees. (the image above is from a later cd-rom version.) a article in phrack reminded me of some of the details and fun of such systems: european internet usage statistics eurostat : internet usage in the eu : “nearly half of individuals in the eu used the internet at least once a week in and a third of households and three-quarters of enterprises had broadband internet access.” statistics denmark : access to the internet: % of population has home internet access. going global with my iphone i can use my iphone pretty much anywhere, but att is going to charge me $ . a minute for calls, $. per text, and $. per kb for data while in denmark. att requires international activation but they do offer some tips for international roamers. i bought an international iphone data plan ( mb for $ ), but i also learned that visual voice mail counts against that (regular voice mail counts against minutes, at the $ . wordpress vs. drupal i’m a wordpress partisan, so i agree with mark ghosh’s criticism of this wordpress vs drupal report. still, it reminds me that i should point out xxlmag, slam online, and ford among the very non-bloggy sites built on wordpress. fish tacos oh decadence! veterans day provided not only a chance for reflection but also a rare day free from the classroom. so what to do with this open period of time? the answer was easy, dinner party. i have wanted to have my colleagues roxanna and john over, but time is always an issue. i phoned them up and they accepted. now the fun began — menu planning. while vacationing with my parents in vegas last summer we went out to marvelous food chain, the cheesecake factory. design anxiety all i know about denmark is what gets imported: legos, of course, but also a tradition of exquisitely clean and functional design. that’s why, as i prepare for my talk in copenhagen later this week, i’m incredibly conscious of my own design and a bit jealous of jessamyn’s outstanding use of orange. anyway, that’s where i’ll be all week. any tips? anybody up for a drink? gender gaps connect the dots: boys vs. girls in us colleges and too many men in east germany. object-based vs. ego based social networks vs. wow and second life there are so many cool things in fred stutzman’s recent post, but this point rang the bell for me just as i was considering the differences between world of warcraft and second life. more on those games in a moment, first let’s get stutzman’s description of ego vs. object networks: an ego-centric social network places the individual as the core of the network experience (orkut, facebook, linkedin, friendster) while the object-centric network places a non-ego element at the center of the network. internet safety npr : back to school: reading, writing and internet safety as students return to school in virginia, there’s something new in their curriculum. virginia is the first state to require public schools to teach internet safety. freaking mysql character set encodings derek sivers‘ plan, with all it’s bin hex and regexp and back and forth between mysql and php almost looks good compared to what i’m about to do. really, why is it so difficult to go from latin (tables created back in mysql ) to utf ? not only do you have to set the charset on the table, but also the connection, in php, and flipping everywhere. and then you’ve gotta deal with all this old data that’s in the wrong character set. pick up lines how to pick up girls in the library. indeed, it’s picking up girls made easy. internet librarian presentation: building web . native library services the conference program says i’m speaking about designing an opac for web . , and i guess i am, but the approach this time is what have we learned so far? and though it’s the sort of thing only a fool would do, i’m also planning to demonstrate how to install scriblio, a web . platform for libraries (foolish because i plan to do it live and in real time). is the answers.com api public? answers.com is throwing a bone to wordpress users with their new answerlinks plugin written by alex king. but wait, there’s an answers.com api? a few pokes at the google machine reveal nothing relevant, and asnwers.com’s site is mum too. taking apart the code, i get the following (modded enough to make it run-able if you drop it in the base of your wordpress install): ``` require_once('wp-config.php'); require_once(abspath.wpinc.'/class-snoopy.php'); $snoop = new snoopy; $snoop-read_timeout = ; $snoop-submit( 'http://alink. maisonbisson and unapi thanks to mike giarlo‘s unapi server for wordpress. now if only there were a library catalog built on wordpress, i could probably just drop it in. panorama stitchers: calico vs. doubletake i’ve been using doubletake to stitch panoramas for a while, but when i discovered p ps harlow’s photos and learned he was using calico panorama, i figured it was worth taking a look. doubletake has done a great job for a number of my photos (mt. moriah, san francisco motorcycles, mt. mondadnock), and when the automatic stitch failed, i could manually reposition (or re-order) the photos. i could also adjust the individual images to make them better match each other. mac os x . comes with apache and php yep. leopard comes with new stuff. lazeez says it works fine, but commenters here are having trouble. memory, intimacy, and the web i’ve been thinking about it since troy mentioned to me that he thought google was ruining his memory. and i thought i found confirmation of it when i read gladwell’s description of daniel wegner, et al’s transactive memory in close relationships: when we talk about memory, we aren’t just talking about ideas and impressions and facts stored inside our heads. an awful lot of what we remember is actually stored outside our brains. library . subject guides ellyssa kroski‘s librarian’s guide to creating . subject guides is good introduction for librarians who think know “there has to be a better way.” but why no mention of blogs and blogging tools? (i’m still really happy that when you search our catalog for something, a subject guide for that term appears (if we have one that’s relevant)). book autopsies via ryan: brian dettmer: book autopsies at centripetal notion. site crashed…recovered…sort of my hosting provider lost a server, and their most recent backup of my database was from wednesday. that was newer than what i had, so that’s what i’ve got. any comments submitted between then and mid afternoon today have been lost. i was luckier with my posts: i write most of them in ecto and had them backed up on my lappy. at least the sox won. the war on zombies from kim to zach to me to you: bush vs. zombies. now we know: the guy doesn’t understand the difference between fact and fiction. most people thought shaun of the dead was horror/comedy, not documentary. poor w probably read the zombie survival guide as an instruction manual (don’t show him how to survive a robot uprising, please). gah. the guy hired a cannibal, fears animal-human hybrids, and flip-flops on evolution. gravatar acquired, more features & better reliability ahead matt pointed out that automattic has purchased gravatar, the globally recognizable avatar service. om speaks of the economics and matt’s cagy, but it’s hard not to see the possibility of creating a larger identity solution around this. wordpress’ market penetration is huge, a service that connects those nearly two million blogs could offer real value, especially in connection with automattic’s akismet. aside: now that gravitar’s reliability is up, i’ll probably get sexy comments running here soon. stupid trademark law story: timbuk develops a new line of messenger bags that features fabric made of <a href=;http://www.treehugger.com/files/ / /dont_shoot_the.php">recycled material (engineered by rootphi). some of the fabric contains a symbol that target lawyers say is their logo. target lawyers cease and desist timbuk . thing is, the trademarked target logo is a roundel, commonly used around the world (easily recognized in british aircraft of wwii). the particular design target has chosen appears to be a copy of peru’s official insignia. screencasting on mac i’m as annoyed as the next guy about how hard it is to find a decent screencast app for mac. the forthcoming mac os . ’s new ichat theater (and the built-in screen sharing/control features) should create some new opportunities for developers, but right now it’s hard to know what works or is worth trying. further, i narrowed the field with the following requirement: i need an app that records to quicktime-compatible files, not flash. not just hip when a writer goes looking for young turks (my words, not scott’s), you should expect the story to include some brash quotes (writers are supposed to have a chip of ice in their hearts, after all). on the other hand, we’re librarians, so how brash can we be? scott carlson’s young librarians, talkin’ ‘bout their generation in the chronicle this week did it better than most articles: rather than showing how hip or geeky we are, it asks us about the future. friends, photos, favors, feeling ill i practically begged will and karen to get on a carnival ride with me so i could get portraits with the lights streaking behind them. will warned me that he doesn’t do well on rides; i argued that no ride with so many kids under four feet tall could be too dangerous for us. we boarded, it started. from the ground it looked gentle, much like the teacups. that was misleading. corrosion test facility not as rusty as expected corey, will, and jon were all as excited as i was to see the fabled point judith corrosion test site, just south of narragansett, but we were all surprised at how un-rusty the goods were. don’t laugh, corrosion is a big deal. according to the national materials advisory board: corrosion of metallic structures has a significant impact on the u.s. economy. in a congressional study, the total economic impact of corrosion and corrosion control applications was estimated to be $ billion annually, or . fools on the beach [[slideshow|height= px|farm -static-flickr-com- _ e c .jpg farm -static-flickr-com- _f bd da b.jpg farm -static-flickr-com- _ ea fede .jpg farm -static-flickr-com- _fb c b .jpg farm -static-flickr-com- _dbaee d .jpg]] we were there because of the point judith corrosion test facility — the rust museum — but who can resist chasing seagulls? and who can resist posting the sequence? assuming you’ve got a recent browser with javascript enabled, you should see a bit of a slideshow above. photos on flickr, slideshow powered by jquery and bsuite. cocktail manifesto we’re huge fans of the new joy of cooking by marion rombauer becker, irma s. rombauer, and ethan becker. hardly a meal goes through our kitchen that isn’t shaped in some part by the recipes and general information in its pages. a recent discovery was joy’s description and defense of cocktail parties. so, when a book as serious and valuable as the new joy of cooking raises alarms about the declining future of cocktail parties, we listen. who owns the network? note: this cross-posted item is my contribution to our banned books week recognition. we’ve been pitting books against each other, hoping to illustrate that there are always (at least) two sides to every story. most of the other books were more social or political, but i liked this pair. wikinomics authors don tapscott and anthony d. williams tell stories of how the the internet’s unprecedented collaboration opportunities are changing the rules of economics. banned books week dilemma our intention is to feature “a series of books that challenge our beliefs and test our commitment to free speech,” but on this post about holocaust denial i found myself unwilling (and unable) to link to the free, online pdf full text of david irving‘s hitler’s war. and when we discovered it wasn’t in our collection (though it may have been lost/stolen, not replaced, and the record deleted), we decided not to purchase it. business . too tired? magazines fail all the time, but it’s hard not to look at them as signs of something larger. macweek‘s fizzle was claimed to represent the demise of the mac, computer shopper has lost more weight than a slim fast spokesmodel ( pages to in ten years!). and now business . magazine is shutting down and sending cancellation notices to readers. perhaps the lesson here is that there’s nothing too . restaurant review: cotton first impressions how much is too much for an entree at a place that plays the kind of anonymous muzak that kenny g calls jazz and is decorated like applebee’s? trust me, i like renovated mill buildings, but why confuse it with faux grecian columns and too many pictures of dead celebrities? i mean, the interior was clean and pleasant, but lacked attention to detail. if you’re so afraid your customers are going to walk off with the poorly framed prints of old hollywood darlings that you nail them to the wall through the frame, how much can you expect them to pay for dinner? smashitup smashitup smashitup! after all my agitating for small, cheap, fuel efficient cars (and automotive metaphors), i figured i had to post this picture (and a few others) from the demolition derby at the hopkinton fair a couple weeks ago. my video of the four-cylinder event is at youtube. extra: i don’t know where it fits in your stereotype of the demolition derby audience, but i was happy to find somebody wearing a css_descramble. “to ascertain if the applicant is still living” whose library is it anyway?: a visit to the lenox [tags]library, libraries, humor, lennox library[/tags] don’t mistake me (please) over at kle’s web . challenge i was surprised to learn: both bisson and stephens are so excited about this concept of web . they have not taken a good look at what they can’t do for our libraries. …with all this new technology we can not forget that what is the most important in our libraries is the personal touch. we are one of the few institutions left that still offers individual attention. checkouts vs. gpa? cindy harper, systems librarian at colgate university, posted to the iug list with this notion today: i’m clearing out a large group of expired student records, and wonder if anyone else has had the same idea that has occurred to me. [our ils] keeps track in the patron record of totchkouts (total checkouts). at the expiration of the students’ record at the end of their four or so years, this represents a measure that is not perfect, but could distinguish heavy library users from non-users. copyleft: defending intellectual property anybody who thinks free software is anti-copyright or disrespectful of intellectual property should take a look at mark jaquith’s post, what a gpl’d movable type means. let’s be clear, anil dash takes issue with jaquith’s interpretation, but the point is jaquith’s offense at what appears to be six apart’s grabbiness for any code somebody might contribute. freedom was one thing, the willingness of a person to pour his or her sweat into something, then watch somebody else (or even risk watching somebody else) profit from it is another. mullenweg on wordpress and open source i wish i’d seen this from wordpress maven matt mullenweg before i finished my ltr on open source software for libraries. mullenweg is brushing off some of the mystique and praise the media has been giving him, and giving an honest sense of what makes open source software work: the real story is more exciting than the cookie-cutter founder myth the media tries frame everything in. it’s not just one or two guys hacking on something alone, it’s dozens of people from across the world coming together because of a shared passion. it’s standard playtesting, everybody does it in another sign that my generation’s culture is gaining dominance, npr gave video games a bit of coverage this morning. unfortunately, the story that makes it sound like the company invented playtesting doesn’t suggest that microsoft’s behemoth investment in the halo franchise makes that testing (and, perhaps, blandness) necessary. (meanwhile, msnbc last year ran an off-message story about how playtesters declared the wii the top console.) reality: playtesting is one of those dream jobs that people scour craigslist for or start questionable-looking services around. developing and testing mobile content read: a list apart: articles: put your content in my pocket and part ii. test/simulate: opera mini, lynx, a variety of mobile phones, internet explorer (because even with parallels, who really wants to infect their machine with windows?), and iphone. a message from the establishment to the establishment we must stop thinking of ourselves as a good-idea factory whose every thought has greater merit than those of our customers. procter & gamble doesn’t even do that. — paraphrased nh’s virtual learning academy the ceo of nh’s first online-only, distance education high school expects about students to enroll in its first semester, to start in january. so says a report at nhpr. four years of music industry lawsuits & madness marketplace reminds us the storm of riaa lawsuits began in september . in that time they’ve sued a thousands of people, and most lawyers apparently advise those caught in the madness to simply roll over and take it. but tanya andersen, a year old disabled single mother didn’t. after years of litigation (and mounting legal bills), it finally came out the riaa’s lawyers had misidentified her and dropped the case, casually saying “sometimes when you go fishing with a driftnet, you catch a few dolphins. obligatory talk like a pirate day post perhaps talk like a pirate day has been too successful when npr hosts are doing it, but anything that’s so important to our children’s future success is important enough for me. and if you need a brush up on your skills, don’t miss this instructional video. nyt: the link is the currency of the web the new york times has struggled with timesselect, now they’re killing it. but the news here isn’t that a media giant is giving up on a much hyped online venture. the news is that a media giant is endorsing what we now call web . : since we launched timesselect in , the online landscape has altered significantly. readers increasingly find news through search, as well as through social networks, blogs and other online sources. closed formats are bad for libraries, stop ooxml now microsoft just won’t quit. now they’re trying to make ooxml an iso standard. please help stop this. here’s how i explained it in open source software for libraries: the state of massachusetts in announced new it standards that required its , employees and agencies to adopt open file formats. the decision didn’t specify the applications to be used, just the format of the electronic documents they created, stored and exchanged #. nebraska state senator ernie chambers sues god the following, quoted from daily kos: accodring to chambers, god has caused fearsome floods, egregious earthquakes, horrendous hurricanes, terrifying tornadoes, pestilential plagues, ferocious famines, devastating droughts, genocidal wars, birth defects, calamitous catastrophes resulting in the wide-spread death, destruction and terrorization of millions upon millions of the earth’s inhabitants including innocent babes, infants, children, the aged and infirm without mercy or distinction. so, you think “yeah, he’s got a point. building libraries with free software sarah houghton-jan‘s review of my ltr on open source software for libraries reminded me i wanted to blog this related piece i’d written for american libraries. tim spalding cocks his head a bit as he says it to emphasize the point: “librarything.com is social software.” however we categorize it, spalding’s baby has become a darling to librarians, and as we sat chatting over lunch in spring , the web application that had begun life just to months earlier was to catalog its -millionth book. the “show of force” brand a pentagon commissioned $ , rand study, enlisting madison avenue: the marketing approach to earning popular support in theaters of operation, concludes “the ‘force’ brand, which the united states peddled for the first few years of the occupation, was doomed from the start and lost ground to enemies’ competing brands.” small is beautiful will found this on the side of the road, and after he told me about it i begged him to show me. it’s tiny, rusty, and a little older than i expected. like a very, very small vw bus, it has a rear-mounted engine. i think it’s a subaru sambar, but that’s mostly based on the details i gleaned from the subaru article, which reveals that engine was probably air cooled, displacing ccs, and producing under hp. a shadow lifted, berlin’s smokestacks felled corey and i went to berlin to watch the stacks fall today, but bad weather, confusion, and some dud explosives conspired to leave me with no usable pictures of the event. we arrived early and lined up a perfect view of two out of three towers that were to be felled, but as the explosions started it became clear that i was mistaken about which smokestacks were being destroyed, and instead we had a really good view the one stack that was supposed to be left standing at the end of the day. mildly funny scenes i’ve come across recently not lmao, certainly not roflcopter-ingly funny, but funny enough to want to snap a picture, and good enough for casual friday here. the boat in the parking lot, ups vs. fedex, and hoe for hire are all easy enough to understand (though they leave me open to easy criticism). the fourth photo is of some books on an anonymous shelf: look closely at “library trends, ” and others. lessons in change from ford motor company i probably spend too much time considering competition and change management, but just as i figured i was done with it for the week, a comment from kathryn greenhill regarding model ts got me going again. just like railroads, those “any color as long as it’s black” model ts looked like freedom, until general motors showed the world they could get their cars in color and with curves. every car came with four wheels and an engine, and they’d drive you down the block and around town, but the moldy model t suddenly looked pretty old next to a sleek green chevrolet. onewebday have you thanked the internet lately? onewebday, our opportunity to celebrate “one web, one world, one wish” is just about a week away (though it falls on yom kippur). this video explains a bit and tim berners-lee is planning his own video (worth mentioning: his net neutrality post). if things work out, i’ll be posting a video too, even though i’ll likely be offline most of that day (not observing yom kippur, at a friend’s wedding). first they ignore you, then they ridicule you, then they fight you it’s an aside to kathryn greenhill’s larger point, that all this . stuff is about a shifting power to the user, but she places l somewhere on ghandi’s continuum of change between ridicule and fight. the photo above (original by monster) is in support of greenhill’s larger point: control is shifting. trains were once seen as icons of freedom, but that view changed with the development of the automobile — and the way it shifted control of routes and schedules from the railroad to the driver. playing with food like all well bred women, my mother always told me not to play with my food. however, as we get older we realize that sometimes ignoring the rules is just as important as, generally, following them. food is fun. it has wonderful tastes, smells, colors, and textures. something with so many wonderful attributes is just begging to be played with. for me, breakfast is not just the most important meal of the day, its also the most wonderfully yummy for one specific reason — maple syrup. jumping from airplanes a guy walked into the student newspaper office and asked “does anybody want to jump out of an airplane?” without a moment’s hesitation, i said “i’m your man.” it was only afterwards that i confirmed a parachute would be involved. well, that was ten years ago (can’t you tell, i look young — young!), but the video is still laying around and i just uploaded it to youtube. actually, this video has been through the wringer. hawkish is bush really so hawkish that he refuses to formally declare an end to the korean war? launch! a little more than two years after i realized how (really) bad the problem was and about months after i <a href=;http://maisonbisson.com/post/ /wpopac-an-opac- -testbed">prototyped my solution, our new library website, catalog, and knowledgebase launched last week — just in time for the fall semester opening. it’s all built on scriblio, includes a very simple new books list that you can narrow by subject and get via rss. and if you search for subject areas like anthropology, economics, english writing, or any of a few dozen other topics, you’ll find our librarians’ subject guides listed at or near the top to help you out. cliffy’s office prankd office pranks are a bit of a thing here. well, at least in it. last year matt took charge and put together a quartet of pranks that got the attention of the london daily mirror. this video is from a may prank that put a golf cart with fuzzy dice and bobble headed jesus in cliffy‘s office along with a vote bush sign and other things. he was mad, to be sure. add tags to flickr photos while uploading via email the short story is that you simply put “tags:” in the subject or body and anything that follows becomes a tag. it’s worth remembering that the subject of the email becomes the title and the body becomes the description. the longer story is at flickr. make it official before he forgets in a development that even foxnews couldn’t ignore, us attorney general alberto gonzales has resigned, he thinks. would princess diana have been a blogger? in an interview on npr, the diana chronicles author tina brown says “diana had represented feeling, and the end of the stiff upper lip,” but the princess comes off sounding a bit like a harbinger of the cluetrain. yes it’s all about the royals, the glamor, and her dramatic death ten years ago, but take note of this exchange: renee montagne: “the royal family is probably stronger than it was when she died. vicar’s delight hot weather demands cool drinks. lemonade is fine for the kids, but adults need a pitcher of something more entertaining. parts vodka part orange juice parts lemonade dash lime juice prepare in a pitcher with ice and share. adjust quantities to taste. enjoy safely. iphone unlocked if the news is to be believed, separate teams have found hardware and software-based solutions to unlock an iphone. it’s worth noting that all this is legal because of an exemption, <a href="http://www.onthemedia.org/transcripts/ / / / ” title="on the media: transcript of “mobile malcontent” (march , )“>much needed and hard fought. scratch-n-sniff hey, i’m a fan of that old book smell too, can i get some scratch-n-sniff stickers? meebome + pidgin = a match made in heaven meebome + pidgin (formerly gaim) = a match made in heaven. (via.) color blind safe web design check etre‘s colour check. a good day to land the shuttle? a hurricane, high crosswinds at the landing site, a nitrogen leak, and two damaged tiles. watch the shuttle land live on nasa tv. allagash wilderness, maine will, jon, joe, ted, and i arrived at telos landing with plans to run the allagash wilderness waterway. as we prepared to embark, the park ranger appeared with a tape measure and told us our kayaks weren’t canoes. section . of the allagash rules and regulations is quite clear: “a canoe is defined as a form of small watercraft long and narrow…. the width at the widest point shall not exceed % of the craft’s overall length. , structurally deficient bridges about , vehicular bridges nationwide, and , are “structurally deficient.” sources: national bridge inventory compiled by the u.s. department of transportation, american society of civil engineers‘ infrastructure report card, and gannett. p ps’ panoramas shot with iphone i’m coming to learn that p ps has a number of interesting things going on, but it was his panoramas stitched from pictures taken by iphone that caught my attention first. above is the j train somewhere between fulton and city hall. i’d thought the iphone’s camera was pretty decent, p ps’ work shows it off. bad joke friday [innerindex] beginning of a bad day… i rear-ended a car this morning. i knew it was going to be a really bad day! the driver got out of the other car and i looked down and realized he was a dwarf!!! he looked up at me and said “i’m not happy!” so i said, “well then, which one are you?” and that’s how the fight started. our diets, our health a doctor was addressing a large audience in tampa. mac + cell phone + bluetooth + sms old instructions that connect the mac os x address book app to a phone via bluetooth from o’reilly and sillydog. once paired, the address book can initiate dialing, notify the user of incoming calls, and send sms texts. bluetooth texter sms widget, message net, and bluephoneelite all offer further tools to interact with your bluetooth-connected mobile phone. the list of compatible phones (bpe & m n) offers some leads for those trying to make the connection. fuel economy: is diesel an option? in response to my previous kvetching about the scarcity of cheap fuel efficient cars, jwk commented that his golf tdi gets mpg (it’s rated for ). meanwhile, treehugger pointed out that volkswagen’s polo bluemotion gets mpg (volkswagen uk claims the current polo hatchback gets up to mpg in diesel (i assume that’s about mpg in us measures), and treehugger points out the mpg loremo ag). iphone + newton + emate pr n {#set_thumb_link_ .image_link}{#set_thumb_link_ .image_link}{#set_thumb_link_ .image_link}{#set_thumb_link_ .image_link}{#set_thumb_link_ .image_link} {#set_thumb_link_ .image_link}{#set_thumb_link_ .image_link}{#set_thumb_link_ .image_link}{#set_thumb_link_ .image_link}{#set_thumb_link_ .image_link} it’s likely phil carrizzi could make a broken tire iron look good, but his series of the iphone with the newton message pad and emate is geek-sweet eye candy. i want a cheap fuel efficient car i’m looking for a new car, but i’m finding that the market for cheap and fuel efficient cars is no better now than it was in . i drive about miles round trip to work (all highway), so i’m looking for the best available highway fuel economy. i can drive a standard, but sandee can’t, so we’ll need automatic. i like small cars, but no so much that i want to pay a lot for one. moving a subversion repository i foolishly just moved a subversion repository by importing the contents of a current checkout into a new repository. wrong. a friend pointed out these instructions that make it easy and preserve the revision history. here’s the trick: svnadmin dump /path/to/repository > repository-name.dmp and svnadmin load repository-name < repository-name.dmp [tags]svn, subversion, move, repository[/tags] castro sued for wrongful death of cia operative, guantanamo bay prisoners taking notes the bangor daily news is reporting a maine woman has sued fidel castro for her father’s death. sherry sullivan of stockton springs accuses fidel castro, his brother raul, the cuban army, and the republic of cuba for the wrongful death of her father, who has been missing and assumed dead since he was last seen at a mexican airstrip in . according to the lawsuit, from until their disappearance, sullivan and rorke participated in numerous covert anti-castro operations in central america and cuba. chocolate white chocolate chip cookie and vanilla bean ice cream sandwiches so once again, my husband called on my assistance with a friday food fiesta challenge. this week’s theme was cookies and biscuits. i scoured my pantry, but alas, like old mother hubbard, my cupboards were practically bare. the one interesting thing i did have was a bag of hershey’s white chocolate chips. so, between my meager rations and a quick trip to our town’s tiny market for butter, i cobbled together the ingredients needed to make the chocolate, white chocolate chip cookies on the hershey wrapper. is it that they don’t care? or just don’t want it from us? &tjessamyn asks “do library users care about our new initiatives?” it comes from a survey done by the wisconsin public library consortiumon one hand, if you interpret the results literally you could make a decision to reject technology and focus on building a collection around personal enjoyment for wisconsin residents. on the other hand, these same results may suggest that initiatives and library services need to be marketed in such a way that resonates with current conceptions of a public library. the fbi and irs are a series of accountants alaska senator ted—the internet is a series of tubes—stevens (mockingly so, listen) returned to find the fbi and irs searching his alaska home. iphone complaints cliff and vasken wrote up some link bait complaining about how the iphone doesn’t meet their expectations or is a lesser competitor to a crackberry. but i challenge them to find a device that offers what they say is missing or even matches what the iphone has. still, i’ve been using mine for a month now, and i can say there are few things it’s missing or could do better. ingmar bergman dead at swedish film director ingmar bergman is dead at . the local calls his work immortal, as did many of his colleagues. until now i’ve been misremembering the title of one of his movies as three smiles of a summer night, a romantic comedy. i’d say that most of his works i’d seen were depressing and that smiles was one of the few that wasn’t. but i couldn’t even remember the title properly, so perhaps i should keep that to myself. sour cream berry bread my wonderful neighbor, wendy, went berry picking and dropped me off a large container with luscious, fresh blueberries and raspberries. i decided to try a bit of an experiment and use the batter for one my favorite cakes with the berries. the result was this heavenly sour cream berry bread. preheat oven to degrees. grease and flour an -cup loaf pan. melt tablespoons of salted butter, pour into a large bowl, let cool. what is that thing kent wien posted this photo of the tail of a boeing showing what looks like the exhaust end of a turbine. i had to ask what it was all about, and kent explained: ahh, very good question! there actually is an engine back there. it’s the apu (auxiliary power unit) and it’s what keeps the airplane cool on the ground without being plugged into the gate. it also provides electrical power and high pressure air that starts the engines after we push back from the gate. poet-bot doug savage‘s take on frost. iphones around the world a long time ago somebody started the newtons around the world gallery, and it came to symbolize the love we newton users had for the little device as well as our geeky pride. the trend seemed to continue with ipods around the world, and now ilounge wants to start a gallery for the iphone. i was about to submit when i noticed the legal fine print: by submitting, you agree that all photographs, and private information you submit are entirely yours at the time of submission, become the property of ilounge upon submission, and that you have not submitted and will not submit such images to any other contests. iphone troubled, replaced on thursday i had trouble answering a call. by friday night it was clear my iphone was seriously porked. a visit to the nearby apple store got me a swift replacement, and a promise that once i synchronized the new device it’d have all the info the old one did. hrm. well, the mac genius did ask if i had any photos i hadn’t offloaded, as those would be lost in the swap. liz danzico on wordpress usability liz danzico of happy cog studios spoke today about her consulting with automattic on the design of the wordpress admin interface. as with so many of the presentation today, i’m really hoping the slides will be published soon, as there are some great ideas coming out. liz spent a lot of time watching wordpress users at blog. at work, in cafes, and in their homes with coffee and cigarettes, liz saw real users of all types doing everything they do with wordpress. scriblio goes to wordcamp scriblio is based on wordpress, an open source content management system, and the community that uses, supports, and builds it is what makes it great. wordcamp started last year, when the community was about , , and it’s even more important now that it’s grown to nearly two million. the first day of the schedule focuses on how to better use the software, and included a great session by lorelle vanfossen. tomorrow is more technical, with discussions about performance, usability, and development. designing the obvious robert hoekman, jr is speaking now on designing the obvious, his book and philosophy: these principles include building only what’s necessary, getting users up to speed quickly, preventing and handling errors, and designing for the activity. i just added the book to my must read list, but what i’m hearing here sounds like instructions to a sculptor: chip away all that is not david. calliope gazetas design calliope gazetas works for the fontshop and freelances under the name monsters. one of her projects includes skinning the burning man environmental blog. jason brightman design portfolio jason brightman’s work includes xxlmag. wordcamp wordcamp wordcamp i’m at wordcamp again. this time i dragged matt and zach with me. dan kuykendall, author of podpress, is first on the schedule, and i’m just now learning how he’s built in support for a variety of media types (more than mp ) and for premium content. those who showed up early got to pick over last year’s t-shirts. this year’s shirts are way different, having given up the somewhat cleaner and simpler design of that has characterized wordpress so far. peanut butter burger now matter how depressed i got in new orleans, i still had to eat. a tip from the ladies at molly’s on toulouse led me to yo moma’s with instructions to try their peanut butter burger. yes. peanut butter. on a burger. i was also told that if i don’t like mayo, i should tell them to hold it because they’ll put it on thick if i don’t. yes. peanut butter, on a burger with mayo. when you can’t say it in english… when you can’t say it in english, say it in german. the reconstruction of new orleans it wasn’t until after my presentation that i had a chance to see the city. and i have to admit it was so depressing that i’ve been having trouble writing about it. i have a sick interest in abandoned theme parks and the like, but seeing the neighborhoods of all classes so destroyed, the symbols marking search and rescue attempts, and the general vacancy of the city left me confused and uncomfortable. presentation: bringing the library to the user i’m at aall in new orleans as part of a program organized by june liptay and alan keely, speaking with u of r’s david lindahl and ncsu’s emily lynema. from the description (see page in the program): traditional library online catalogs are being marginalized in an increasingly complex information landscape. …better methods are needed for mining the wealth of information in library systems and presenting it clearly and concisely. yes it’s laughable, but… i get as frustrated with airport security as the next guy (and i’m plenty doubtful of its effectiveness), but really, if you don’t yet know liquids aren’t allowed, and you hold up the one security line at a small airport at an ungodly early hour, it’d be nicer if you didn’t laugh like a kid at a theme park about it. yes it’s farcical, but not funny. usage instructions <img src="http://farm .static.flickr.com/ / _ce f .jpg” width=" ” height=" ” alt="“tear open packet and use”” /> what’s really angering about instructions […] is that they imply there’s only one way […] their way. and that presumption wipes out all the creativity. actually there are hundreds of ways […] and when they make you follow just one way without showing you the overall problem the instructions become hard to follow in such a way as not to make mistakes. the rarin in librarian i’m going to violate my rule against linking to nyt (because) and give a shout out to this article. not just because it quotes my friend jessamyn, but for what it says: libraries are full of smart, hip people. [tags]library . , jessamyn west, new york times, libraries, hip, smart[/tags] essential iphone apps rush in [innerindex] games tilt, described in programmer joe hewitt‘s blog: …christopher introduced me to a very talented video game designer, nicole lazzaro, who had an endless stream of ideas for games that would use the iphone’s accelerometer. nicole’s ideas quickly ran into the limitations of the phone, as we discovered that the browser doesn’t rotate when you hold it vertically upside down, nor is it possible to distinguish the two horizontal orientations. whose technology is it anyway? i wasn’t planning on posting much about keen’s cult of the amateur, but i did. and now i find myself posting about it again. thing is, i’m a sucker for historical analogy, and clay shirky yesterday posted a good one that compared the disruptive effects of mechanized cloth production to today’s internet. yes, that’s actually the birth of the luddite movement, or at least where it got its name. and, though i was aware of the story, shirky’s study offered details i’d not know previously. ironic moments in law enforcement new hampshire’s deputy chief of liquor enforcement caught drunk driving. keen says i’m killing culture, byte by byte andrew keen‘s the cult of the amateur__; how today’s internet is killing our culture is getting a lot of attention from usually quiet corners of the web, and i’ve had to quell the urge to write a story under the headline “andrew keen tells youtubers to eat spinach.” keen’s argument rests on the belief that “culture” is the sole provence of established media, and falls flat as soon as you get past the bombast of the subtitle. why is pdf inferior to html? html and postscript are both page description languages, but one is designed to convey the look of the page, while the other to convey the meaning of its content. pinch me i’ve been away from my computer for a couple days, but very much online with my iphone. today, as i looked at something on my laptop in google maps i found myself trying to pinch and flick my monitor to manipulate the position and scale. felonious dancing naked == lewd lascivious conduct == felony crime. (better, however, than riding a gondola naked.) celebrate independence day with a drink ok, the truth is that at maisonbisson we celebrate all holidays with a drink. since we take cocktails quite seriously, i wanted something very pretty for the little fourth of july soiree we were having. i have found that the secret to a perfect strawberry daiquiri is using frozen strawberries. i also use lots of crushed ice and a ripe banana — it adds a nice creaminess. i garnished with whipped cream, blueberries, and star fruit. cold cucumber soup my beloved husband went off on a boy’s adventure weekend. this left me with the entire house and kitchen to myself. when this happens, i become a bit like a mad scientist left alone in my laboratory. so, it was just me, the cats, and that most dangerous invention, food network. after some house work, chick flicks, and visiting with my parents, i spent an hour putting away laundry and watching emeril. sweet bike sweet bike originally uploaded by misterbisson. sent from my iphone iphone accident big accident on highway leaving mall…was somebody unboxing their iphone while driving? so much sweetness in so small a package zero hour + minutes: the iphone rocks. minutes to go minutes to go. guy from store: “being in line doesn’t guarantee you’ll get one.” two hours, people. two hours to go, people in line. blackout they just put up black vinyl over the windows and gate. the line has grown to about . still no word of quantity, but somebody shared a story that they asked “what happens if there are people in line?” the answer was supposedly: “even if they buy two we’ll have enough.” retail status check does your apple store have iphones? about the rumors are that the at&t store here has about phones. nobody is talking about how many our apple store has. people in line people in line. at least one is hoping to auction his, three are being paid, and nobody wants the cheap one. fake iphone pic at first believed, then quickly called out by the true believes in line this pic elicited gasps, then indignation. we’re loved, we share the love suited security guy with square jaw and angry expression grunts at us as he confirms plans with store manager. he’s from management, and though we couldn’t overhear much, we did realize he was headed off to the at&t store next. all of us remained silent as we watched him stomp off in the wrong direction. waiting for iphone arrived at am to find four parties ahead of me. the first arrived at am, after repeatedly being chased out of the mall parking lot last night. june : tony day it’s tony day, not just because joe’s book has garnered some good reviews—“the only excuse for the continued existence of boxing is that its battles have occasioned some of the best writing any sport has ever inspired”—or because he likes telling the story. it’s tony day because “galento [is] a champion of everyone who’s ever gotten in over his head, shrugged, and said ‘what the hell? i’ll give it a shot. apple iphone vs. internet tablets sure, the iphone is a sweet phone (even at $ ), but how does it compare to the less definable internet tablet category? i’ve actually used a pepper pad and held an olpc in my hands (yes, they exist), but what i know about the nokia n (the successor to the n ) is limited to what i’ve been told. all four devices have feature-complete browsers and can take advantage of the rich web . presentation: faceted searching and browsing in scriblio i was honored to be a panelist at the lita/alcts ccs authority control in the online environment interest group presentation of “authority control meets faceted browse.” what is faceting? why is it (re)emerging in use? where can i see it in action? this program is intended to introduce the audience to facet theory, showcase implementations that use faceted approaches for online catalogs, and facilitate discussion on the relationship between structured authority data and this type of navigation. the iphone cometh; haters swarm some are calling it the jesus phone, but jason chen calls it a moral quandry, gartner group is <a href="http://www.techworld.com/mobility/news/index.cfm?newsid= &pagtype=samechan” title=;techworld.com - gartner warns it to avoid apple’s iphone">telling it to avoid it (really, because itunes is scary to enterprise), business . ’s joshua quittner is reminding the peeps it’s just a regular phone, and wayne smallman is whining that it doesn’t have a flash or telephoto lens. (humor alert: one of those is supposed to be funny, and another is supposed to be hilarious. presentation: transforming your library with technology [innerindex]part of the transformation track, transforming your library, and your library’s future, with technology, program coordinators alan gray and john blyberg (both of darien public library) described it like this: technology can transform your library and its services, as it is transforming the lives of your patrons. from do-it-now technology improvements to next-generation implementations, from software to sopacs, from in-your-face competition to over-the-horizon transformations, three accomplished experts will instruct, enlighten and challenge you to use technology to make your library more relevant to your patrons — today and tomorrow. iphone service plans and coverage? at&t’s current (reasonable) voice and smartphone data plans offer minutes for $ and unlimited data for an additional $ , but previous reports about the iphone suggested that consumers should expect to pay $ /month for service, so we’re left to wonder what’s up. meanwhile, i’ve been asking at&t users about their signal coverage. i’m on verizon now and enjoyed pretty solid coverage throughout dc, even underground. folks on at&t, however, had spottier coverage, even above ground. “as dead as elvis” “the librarian as information priest is as dead as elvis,” needham said. the whole “gestalt” of the academic library has been set up like a church, he said, with various parts of a reading room acting like “the stations of the cross,” all leading up to the “alter of the reference desk,” where “you make supplication and if you are found worthy, you will be helped.” via. down the up escalator running down the up escalator = fun. landing upright = difficult. escalator, running, up, down an almost-manifesto masquerading as a presentation… context: below is the text of my virtual presentation to the lita bigwig (it stands for blogs, wikis, interest group, and stuff) social software showcase. the presentation is virtual, but the round table discussion is going on today, june rd, from : - : p.m. in the renaissance mayflower cabinet room. i won’t be there, though. my bad scheduling got me double-booked and i’m presenting in the transforming your library with technology track. cider drinks black adder = cider + guinness snakebite = cider + harp th century information architecture one hundred years ago the country was in the middle of a riot of library construction. andrew carnegie’s name is nearly synonymous with the period, largely due to his funding for over , libraries between and , but architectural historian abigail van slyck notes that the late th century was marked by widespread interest in community development, with broad recognition of libraries as a means of promoting individual development. trains vs. seat belts i’m not saying i want seat belts, but it always takes me a moment to get used to them not being there on a train. the sky is falling myspace, second life, and twitter are doomed. the rules, [innerindex]web . has matured to the point where even those who endorse the moniker are beginning to cringe at its use. still, it gave me pause the other day when cliff (a sysop) began a sentence with “web . standards require….” web . is now coherent enough to have standards? we used to joke about rounded corners and gradient blends being the rule, but something more has indeed emerged. o’reilly defined web . google gears google gears: create web apps that work offline two books on a shelf… two books that just happened to be sitting next to eachother in the lc files: dlc . s xx dut _a _a . _a(ocolc) _adlc _cpbm _ddlc _apremarc _ad .n _br _atoonder, jan gerhard, _d - _ahet puin aan de rotte, _cdoor j. gerhard toonder. _aamsterdam, _ba. j. g. strengholt _c[c ] _a p. cake robed in chocolate and strawberries like so many women, there are days when my desire for chocolate is nearly overwhelming. however, perhaps because i am a tad high maintenance, my cravings are not satisfied by a mere candy bar. when i crave chocolate i want something rich, decadent, and freshly baked, i want chocolate cake. when one of these cravings coincided with finding the first of the year’s native strawberries i decided to combine the two, the result was the cake you see above. arm wrestling, dung throwing, lawnmower racing, and seed spitting i don’t know whether to thank the pheonix or the fair organizers for this great ad copy, but i hope the washington county fair is as good in as it sounded in : an agricultural fair featuring tractor pulls, stage shows, crafts, and livestock, plus games and children’s contests. adult events include arm-wrestling contests, dung throwing, lawnmower racing, and seed spitting. live country concerts every night. open wed through sat from am to pm, and on sun until pm. go together? just spotted: do hippie skirts and bluetooth headsets go together? star wars stamps found at post office star wars stamps found at post office. will the merchandizing ever end? flag day the us flag with all its stripes and a few of its stars was adopted by a resolution of the second continental congress in . but today, overpriced textbooks and underpaid schoolteachers have sanitized most of our history and hidden the early controversies while fluffing half-truths, leaving us unclear about what that flag really stands for. fortunately, this is america and we’ve got movies to tell us what our teachers didn’t. a three year high report: civilian and military death toll in iraq is up strongly after us “surge.” roy pearson sues custom cleaners roy pearson sues custom cleaners for $ million over lost pants. millions! pants! new hampshire ranks local pride: new hampshire ranks near the top of the list for quality of healthcare services, according to new report. climate change vs. budget planning just as climate change makes hurricanes more frequent and dangerous, noaa says its best tracking satellite is failing and there’s no plan to replace it until . desoto report leaked. desoto report leaked. the highest ranking un official in israel has warned that american pressure has “pummelled into submission” the un’s role as an impartial middle east negotiator in a damning confidential report. echos abound. the neocons were right, so far… the neocons were right so far: civil war is erupting throughout the middle east and iran is feeding the flames. is this really what we (or anybody) wanted? paralyzed paralyzed: they can blow our helicopters out of the sky, and now they’re [destroying the roads and bridges][ ]. are we prepared for [another surge in iraq][ ]? [ ]: http://www.plenglish.com/article.asp?id=% bb c ad- cbf- ab- b e- b a d % d)&language=en [ ]: http://www.huffingtonpost.com/ / / /no- -us-commander-in-ira_n_ .html installing mysql with yum how to install and configure mysql database server wordpress blogging by email the built-in tools don’t support secure pop , but gmail requires ssl pop . the fix? postie. carbon neutral living apm marketplace: news of a british model home. highly insulated, carbon neutral, just % more$. not just a demo, it’s going to be the law: all new uk buildings to must be carbon neutral by . economies of scale are said to reduce or eliminate the added cost by then. down for fifteen years straight, up like a rocket now after being down for fifteen years straight, milk consumption is up. up big, and prices are rising to meet it. stand alone appletv? new gb appletv. how far away are we from a standalone unit that can download from itunes store directly, sync ipods, and write to usb-attached burners? iphone apps = web apps; web apps = iphone apps wwdc: safari for windows!?!? leopard looks sweet, but delayed ’till october. iphone apps = web apps. the new plazes plazes, a kinda-cool, formerly networked-based geolocation tool has just been revamped. they’ve been promoting this change for over a month (i got a cool invite to the launch party, but couldn’t make the flight to germany), and they’re continuing the push now that it’s live. i’ve used the new service for a few days, the company has sent me an email soliciting feedback, i’m offering it. i submitted the following via the site’s contact form, but the message seems to have disappeared, and i prefer public discussion, so i’m reprinting it here: presidential candidates chasing rural votes? presidential candidates chasing rural votes? worth remembering that % of us libraries serve towns of , or fewer people. -fed r-fed defeated, k-fed mourns. missed the paper airplane contest… i missed the paper airplane contest in concord nh today!?!? ultimate frozen mud slide recipe who wouldn’t enjoy a frozen mud slide on a hot summer day? typical recipes call for crushed ice and cream or ice cream. for some reason, we decided to try making them from ice cream, from scratch. the maisonbisson frozen mud slide this recipe requires an ice cream maker, we used the deni scoop factory. . cups heavy cream cup milk cup sugar . cups bailey’s dash vanilla mix ingredients in bowl, then pour into ice cream maker’s freezer container. wheelchair ride mph wheelchair ride in michigan people invent funny words: schaedenfatte okay, now that we all know what a muffin top is, let’s learn about schaedenfatte: schaedenfatte: shaw-den-fah-tuh, etym. from the german, schaedenfraude. (n.) . the feeling of pleasure upon seeing someone for whom one once held unrequited romantic and/or lustful feelings who has now become fat. . the taking of such pleasure. with summer being the season of weddings (and, along with reunions, weddings being the place where people people who haven’t seen eachother for years cross paths…), i suppose you might also call it the season of schaedenfatte. students want libraries iblee points out that students want libraries. asdasd asdasd they vaccinate ducks… they vaccinate ducks against h n bird flu, but not enough. it’s active again in vietnam, where the first human case since has now appeared. regime change… why isn’t the us supporting regime change and democracy in packistan? we’ve given general perv us$ b in aid since ! queasy stomach bush gets queasy stomach when facing other world leaders at g . the poor fellow is being shamed by his peers. open source software and libraries; ltr . , finally the most selfish thing about submitting a manuscript late is asking “when is it going to be out?” so i’ve been waiting quietly, rather than trouble judi lauber, who did an excellent job editing and managing the publication. ryan and jessamyn each contributed a chapter, and i owe additional thank yous to the full chorus of voices that answered so many of my questions, participated in interviews, and generally made the book/journal/thing what it is. what’s up with police? “prosecuting a woman for ‘staring’ at a police dog is absurd,” said her lawyer. “people are allowed to make faces at police dogs and officers to express their disapproval. it’s constitutional expression,” said public defender kelly green, who represented jayna hutchinson. more: what’s up with police? this is the liberal media? what liberal media author eric alterman arrested, mocked at gop debates. poke your tech staff with sticks, and other ideas what a difference a year makes? jessamyn was among those sharing her stories of how technology and tech staff were often mistreated in libraries, but there’s a lot of technology in this year’s ala program (including three competing programs on saturday: the ultimate debate: do libraries innovate, social software showcase, and transforming your library with technology. and still, not all is well. ryan deschamps seems to have hit the button with a post from april of this year. months libby to scoot in for months. is it enough? good for? “what is an atomic bomb good for?” easy mysql performance tips yes, i’m still trying to squeeze more performance out of mysql. and since small changes to a query can make a big difference in performance… here are two really easy things to be aware of: never do a count(*) (or anything *, says zach). instead, replace the * with the name of the column you’re searching against (and is hopefully indexed). that way some queries can execute entirely in the keycache (while * forces mysql to read every matching row from the table). what’s so bad? congressman sensenbrenner: “what’s so bad about shorter winters and global warming?” ironic: lightning strikes church steeple lightning struck the steeple of the saint john the baptist church in allenstown nh saturday. men at work… men at work lead singer has new album: “are you lookin’ at me?” biofuel: good idea, bad practice yes, gas prices are high, and gas doesn’t grow on trees (well, in geologic time it does), but that doesn’t mean that it’s a good idea to run on cars on corn, even if it does grow on, um, trees (yes, alright, cornstalks). i mean, people talk about photovoltaics being inefficient, but wow, think of how much energy it takes to turn a seed into corn, then turn that corn into ethanol and truck it to a gas station. the lawnmowers in ohio from associated press and wavy tv: police said a drunk man drove a lawnmower to a store about a mile from his house. they arrested him on his way home. dondi bowles, , of vermilion was arrested friday night as he drove the mower on a sidewalk. police said a breath test showed that bowles’ blood alcohol level was . percent, nearly twice the legal limit of . percent. industrialized transportation vs. individual choice thought: industrialized transportation first aggregated passengers onto railroads, the broke up into cars…technology empowered the individual, and they embraced it. wish alanis a happy birthday i’m wishing alanis morissette a happy birthday not just because we share a birth month and year, but because it’s a good reason to look back at her cover of my humps and get another smile. but, as long as we’re talking about events in june, we might as well remember that we’re now just days away from paris hilton’s retirement. youniversity “youniversity” big issue… huh, the nasa administrator doesn’t think global warming is big issue. what’s his stance on evolution? speedy php: intermediate code caching i’ve been working on mysql optimization for a while, and though there’s still more to done on that front, i’ve gotten to the point where the the cumulative query times make up less than half of the page generation time. so i’m optimizing code when the solution is obvious (and i hope to rope zach into giving the code a performance audit soon), but i’m also looking at optimizing how php works. bragging about my new office it’s taken a while (we moved in two months ago), but my new home office is finally usable. the big hurdle was my desk. i prefer to stand (or walk) while working, but there aren’t many desks for that, and those that are available are very pricey. so i put together the above from a recycled base, a matching pair of table tops from ikea, and some decorative wall-boxes that elevate the upper surface. books i now want to read… the problem with working on scriblio is that i end up running into so many interesting looking books. just this morning i discovered a number of recent acquisitions in the th century and th century subject feeds in my development instance (also available via rss). all of this is under active development, so those links may or may not work, and the site is definitely changing urls soon. street-level photos in google maps! thanks to ryan eby for tipping me to this. go try it out. whatever you think of them, they do keep delivering. i wonder if people will ask for stack-level photos of our libraries? burninator: kinetic sculpture never looked so hot this is what i get for not following gizmodo faithfully: flaming industrial art. they introduced it saying “do you enjoy fire? do you also enjoy very intricate rube goldberg machines? of course you do.” though a reader there exclaims: it didn’t do anything. for it to be a true rube goldberg doesn’t it have to accomplish some task, like cracking an egg or pouring a glass of milk or something? kids need bowling coaches, desperately there is little doubt that the great diversity of styles and techniques of bowlers from countries enjoying test match status has helped to shape the history of [the sport]. with the recent world-wide implementation of professional coaching schemes, which generally teach only one, or perhaps two optimal ways…, bowling could be in danger of losing its technical diversity. are we therefore on the verge of a new era in which the art of bowling is irretrievably lost? harry potter finale out soon, does book embargo have details? student gets restraining order over facebook photo the associated press reports a composite nude posted to facebook has earned a unh student a restraining order: a university of new hampshire student got a temporary restraining order against another student who combined an image of her face with an explicit photo of another woman’s body, then posted the composite on his facebook page. a judge ordered owen sanborn, of laconia, to stay at least feet away from the woman and barred him from posting her “likeness or name on any internet site,” pending a final hearing. a fair(y) use tale from the chronicle: copyright law, a constant thorn in the sides of scholars and researchers, is generating a lot of public discussion this week, thanks in part to a new -minute video that parodies the law. “a fair(y) use tale” has been downloaded from youtube about , times since it was posted online friday. the video uses cuts from different disney films to mock copyright law as overly protective of the interests of copyright owners — disney among them. google to psyc profile users!?! there it is in the guardian: internet giant google has drawn up plans to compile psychological profiles of millions of web users by covertly monitoring the way they play online games. yep, “do no evil” google has filed a patent on the process of building psychological profiles of its users for sale to advertisers. details such as whether a person is more likely to be aggressive, hostile or dishonest could be obtained and stored for future use, it says… players who spend a lot of time exploring “may be interested in vacations, so the system may show ads for vacations”. redhat selinux gets in my way ack, my wordpress suffers connectile dysfunction on a fresh install of redhat ! not only did i get the above message, but dmesg was filling up with errors like this: audit( . : ): avc: denied { name_connect } for pid= comm=“httpd” dest= scontext=user_u:system_r:httpd_t:s tcontext=system_u:object_r:mysqld_port_t:s tclass=tcp_socket it turns out that i was getting stung by selinux, which is enabled by default in redhat . all the extra security is probably a good idea, if i knew how to configure it, but for the moment it was breaking a live site. surf ‘n turf salad my computer geek husband, who i do adore, joined a flickr photo group called friday food fiesta. a new theme is announced every friday, and everyone contributes a single photo that illustrates that theme. the first themes he contributed to were burgers and pizza, but when salads came up, he needed help. luckily for him, i love making salads. so casey, my husband, asked me to be his partner in crime and create a salad for him to photograph and submit. bringing up the cute quotient of this blog if you ever tire of the kittens on flickr, it turns out there’s no shortage of bunnies on youtube. are you a certified asshole? sure it’s a promo for his new book, but bob sutton is offering us all a chance to see if we’re assholes with the asshole rating self-exam (arse). after questions like “you secretly enjoy watching other people suffer and squirm” (hey, what’s wrong with a little schaedenfreud?) you’ll find yourself placed somewhere on the scale from possible liar to full-blown certified asshole. you don’t sound like a certified asshole, unless you are fooling yourself. customer relations done right rebekka guðleifsdóttir is one of my favorite photographers on flickr. her photos are amazing, and it’s clear a lot of people agree. that’s the easy part. then two problems arose: first rebekka discovered that somebody was selling her photos for profit, and she posted about it. the community was shocked, and angry. and then, and this is the second thing, flickr removed her post about it. and then the storm got worse. increased fuel economy, easy here’s an irony: i used to live in the country, a small town with fewer than residents, and i used to speed. now i live in the city, well, as much of a city as new hampshire can manage, and i’m driving slower. driving slower not just because manchester‘s traffic lights are on timers they leave me listening to crickets chirping at empty intersections while they blindly tick tick tick through the cycles before finally giving me the green (usually just as somebody arrives at the newly reddened light on the other street). wordpress . out wordpress . is out and available for download now! i’m excited because this version includes widgets (by default), some xml-rpc hooks to edit pages (so you don’t need my hacks), a switch to jquery from scriptaculous (matty got me excited about this), full atom support (enough of the different versions of rss!), and the ability to set your mysql character encoding (go utf- !). if that isn’t enough, . is planned for release in september. plasticlogic’s flexible e-paper display plastic logic is a developer of plastic electronics – a new technology for manufacturing (or printing) electronics. the plastic logic approach solves the critical issues in manufacturing high resolution transistor arrays on flexible plastic substrates by using a low temperature process without mask alignment that is scaleable for large area, high volume and low cost. this enables radical new product concepts in a wide range of applications including flexible displays and sensors. people ask me questions: web design software (or is it website management software?) the question: what’s a good user-friendly macintosh web development program? a friend called. she’s thinking of buying dreamweaver, but is afraid it will be overkill. she found frontpage to be easy and needs something similar. my answer: if the intent is to design individual pages on an unknown number of sites, then i don’t have a recommendation. if the intent is to build a site (or any number of sites), then i’d suggest looking at wordpress. wordpress strips classnames, and how to fix it wordpress . introduced some sophisticated html inspecting and de-linting courtesy of kses. kses is an html/xhtml filter written in php. it removes all unwanted html elements and attributes, and it also does several checks on attribute values. kses can be used to avoid cross-site scripting (xss), buffer overflows and denial of service attacks. it’s a good addition, but it was also removing the class names from some of the elements of my posts. it’s not about technology, stupid inside higher ed asks are college students techno idiots? slashdot summarized it this way: are college students techno idiots? despite the inflammatory headline, inside higher ed asks an interesting question. the article refers to a recent study by ets, which analyzed results from , students who took its ict literacy assessment. the findings show that students don’t know how to judge the authoritativeness or objectivity of web sites, can’t narrow down an overly broad search, and can’t tailor a message to a particular audience. l.a. burdick’s cafe and chocolate my favorite place to eat in all of new hampshire is la burdick’s in walpole. it’s a chocolate shop and cafe and i’ve never had anything there that isn’t sinfully delicious. we took my mother-in-law there for mother’s day this year. we started the meal with their delightful cheese plate. this featured four cheeses in a range of intensities, a delightful fruit chutney, olives, seasoned nuts, and crackers. the cheeses were all wonderful and could be purchased at the market next door, many are by local artisans. sausage: the other ground hog the photo is from jessamyn, who declared it groan-worthy. i’m still grinning about it. reminds me of the time homer said “yeah, right lisa. a wonderful, magical animal.” sweet meatcake first it was meat hats, then supermodelmeat. now it’s meat cakes. yes. three layers of meat, with ketchup and potato frosting. it all happened when the groom announced that a man’s cake should be made of meat, ’cause “wedding cackes are all girly.” apparently a red velvet armadillo groom’s cake isn’t manly enough. funny thing, now there’s a growing gallery of meatcakes. (via.) wikipedia the wonder middlebury college banned it, but % of college students and % of college grads use it. twelve year olds point out errors in its competition, while those over are among its smallest demographic — just % (just! %!) say they’ve used it. it’s wikipedia, of course, and the numbers come from a recent pew internet project memo reporting that wikipedia is used by % of the online population and is one of the top ten destinations on the web. is automated metadata production really the answer? (it’s old, but i just stumbled into it again…) karen calhoun’s report, the changing nature of the catalog and its integration with other discovery tools, included a lot of things i agree with, but it also touched something i’m a bit skeptical about: automated metadata production. some interviewees noted that today’s catalogs are put together mainly by humans and that this approach doesn’t scale. several urged building or expanding the scope of catalogs by using automated methods. centos released at work i use red hat enterprise linux, but my personal stuff is served from machines running centos. both distros were just bumped to version , bringing with them support for current components of the lamp stack. i care because i want apache . . , and while it’s pretty easy to get mysql & php on a centos/plesk box, apache . is a bit more of a struggle. gary sims at linux. leopard beta to be released at wwdc those of us hoping for an early release of mac os x . leopard might be disappointed to learn that apple will just be getting around to giving out a “feature complete” beta at wwdc in mid-june. if you really must have it, conference badges are $ , . the leopard beta. available first at wwdc. at the apple worldwide developers conference, we’re planning to show you a feature-complete version of mac os x leopard, and you can take home a beta copy. world’s hottest peppers tabasco thinks their peppers and eponymous sauce are hot. anybody who’s just ate a habanero thinks that’s a hot pepper. but earlier this year, paul bosland of new mexico state university said “damn, i’ve got a hot pepper.” and the guiness world records folks agreed. world’s hottest pepper? bosland had identified the naga jolokia pepper and measured it at over one million scoville heat units, quite a bit more than three times the burn of a hot hot habanero. dewitt clinton on the birth of opensearch opensearch is a common way of querying a database for content and returning the results. the idea is that it brings sanity to the proliferation of search apis, but a realistic view would have to admit that we’ve been trying to do that since before the development of z . in libraries decades ago, and the hundreds of apis that have followed have all well intentioned and purposeful. so what makes makes opensearch something more than an also ran in a crowded herd? awkward moments in social software we all know social networking may be a feature, not an application, but one person’s feature can become another’s bane. so when netflix offers a handy friends feature that makes it easy to share your viewing history and recommendations, it opens itself up not only to the value of social interaction, but also the awkwardness it can sometimes be rife with. titration’s story is instructive: so i have this friend who has invited me to become her “netflix friend” twice now. david halberstam on competition speaking at uc berkeley’s school of journalism last month, david halberstam struck the chord of competition journalists must struggle with. as a newspaper man who started at the smallest newspaper in mississippi and worked his way up to the new york times, where he won a pulitzer for his reporting on the vietnam war, he learned that television’s constant stream of images offered “drama and excitement,” but perhaps incomplete reporting. not that he was criticizing tv, no, he praised it for bringing images and awareness into our living rooms nightly, raising questions among the viewing audience that “we [in newspapers] had the chance to answer if we used our skills properly. mysql error : temp tables and running out of disk space bam: mysql error , and suddenly my queries came to a stop. error is about disk space, usually the disk space for temp tables. the first thing to do is figure out what filesystem(s) the tables are on. show variables like “%dir%” will return a number of results, but the ones that matter are tmpdir and datadir. `show variables like “%dir%”; basedir / character_sets_dir /usr/share/mysql/charsets/ datadir /var/lib/mysql/ innodb_data_home_dir innodb_log_arch_dir miles hilton-barber flies blind from britain to oz i learned of it last night on the cbc’s as it happens: miles hilton-barber, blind since age , has flown from biggen hill, south of london, to gosford, outside sydney, by ultralight in a journey that took almost two months. aviation regulations required he take a sighted co-pilot, but in the as it happens story he explained how his instruments were geared up to give him audio and voice feedback such that he could do most of it on his own. php libraries for collaborative filtering and recommendations daniel lemire and sean mcgrath note that “user personalization and profiling is key to many succesful web sites. consider that there is considerable free content on the web, but comparatively few tools to help us organize or mine such content for specific purposes.” and they’ve written a paper and released prototype code on collaborative filtering. vogoo claims to be a “a powerful collaborative filtering engine that allows webmasters to easily add personalization features to their web sites. remixability vs. business self interest vs. libraries and the public good i’ve been talking a lot about remixability lately, but nat torkington just pointed out that the web services and apis from commercial organizations aren’t as infrastructural as we might think. offering the example of amazon suing alexaholic (for remixing alexa’s data), he tells us that apis are not “a commons of goodies to be built on top of for fun and profit, like open source software.” here are his “six basic truths of free apis:” boris yeltsin: the most colorful, drunk politician since churchill sure, clinton played his sax on tv, bush groped angela merkel, but boris yeltsin gave speeches drunk, tossed women into the water, danced on stage, and generally did all manner of laughable things. but he also turned back a hardline coup by jumping atop a tank and dragged russia kicking and screaming toward democracy. not since cigar chomping, scotch drinking winston churchill led britain through world war ii has the world had a more colorful leader. atomic test photos from los angeles this renewed talk of building nuclear weapons here in the us reminded me of an old report of photos of the sky glow from nuclear tests done in nevada seen over los angeles. this one includes the following description: atomic explosion, the largest yet set off on the nevada test range, was clearly visible in los angeles. staff photographer perry folwer was ready with his camera on a tripod on the roof of the herald-express building when the blast occurred at : a. nukerator, we’re nukrawavable will, cliff (both above), and i recorded this song in one take in late . though, calling it a “take” is overstating it. we were beyond silly drunk and lacked any talent for the task, but we had a mic in front of us, a guitar, and a willingness to open our mouths and let something — anything — fly out. it wasn’t until will said “this song is called nukerator” that we knew what we were supposed to be singing about. csi jumped the shark i’m a newcomer csi: crime scene investigation, i started watching it with season six while suffering a flu that immobilized me for what seemed like a week or more. dumb with illness, i went searching for a diversion at the itunes store and stumbled into the series. i had the entire season downloaded quickly; it took me two marathon days to watch them all. i got hooked. now i’m following season seven, again via itunes. how to: zip files on mac os x it couldn’t be much easier. i’d previously posted command line instructions, but it turns out that there’s a huge number of people who don’t know the easy way: just ctrl-click on the file and select “create archive…” you’ll also find the option in the file menu. either way, you’ll end up with both the original and a zipped copy. decompressing that zip — or any other — is as simple as double-clicking it. ncaa set to ban text messaging between recruiters and high school students college sports are big business, so recruiting student athletes is big business. the ncaa limits the times coaches and recruiters can call or visit athletes, but text messages are all fair game. for now. the chronicle of higher education explained in an october story: before chandler parsons committed to play basketball for the university of florida, his cellphone buzzed more than times a day with text messages from college coaches. are we there yet? still waiting for decent ipod car integration even bob borchers, apple’s senior director of ipod worldwide product marketing, calls most ipod car setups an “inelegant mess of cassette adaptors and wires.” indeed, while apple aparently doesn’t want to get into the car audio business, they do want to improve the in-car ipod experience: what apple really wants you to buy is a car that’s designed from the ground up to interface with the ipod,” the web site said. please, not another wiki ironic secret: i don’t really like most wikis, though that’s probably putting it too strongly. ironic because i love both wikipedia (and, especially, collabularies), but i grit my teeth pretty much every time i hear somebody suggest we need another wiki. putting it tersely: if wikis are so great, why do we need more than one of them? i think my concern is that wikis appear to depend on either very large or very, very active communities. claims of prior art in verizon/vonage patent infringement case vonage has been saying verizon’s patent claims are overly broad for some time, but now people have dug up some prior art. one of the patents verizon is complaining about is # , , , what they call an “enhanced internet domain name server.” in short, it’s all about linking phone numbers to ip numbers, and jeff pulver says he was doing that in with free world dialup, an early, noncommercial voip service. the high cost of innovation: vonage’s patent woes vonage will be in court again tomorrow defending itself against verizon’s claims of patent infringement. the innovative voip company had lost the trial and was ordered to pay $ million in damages in early march, when a jury found them to have violated thee of seven related patents held by verizon. vonage appealed of course, but it’s uncertain if the company, which has yet to turn a profit, has the stamina for a drawn out battle. eco-friendly web design for earth day mark ontkush at ecoiron did some math starting with the department of energy data that showed crt monitors consume less power displaying dark colors than light and determined that redesigning google’s site in black would save megawatt-hours per year (assuming that % of computer users still haven’t upgraded to lcds and are using power-hungry crts). the results were so dramatic he redesigned his own site and developed a low wattage palette that uses only about three or four watts more than a completely black screen (white is to be used only as a text or accent color). “i want my money” my nephew checked his email while he was here this morning and this was the first thing in his inbox. maybe it’s because he’s and my humor is at about the same level, but both of us were cracking up over it. miserable attempt at recovering my dignity with serious criticism: will farrell and landlord prove there is no meaning (or humor) without context. would it be as funny without will farrell (with full afro! reminder: paris hilton to retire in days amid all the “zomg paris hilton is pregnant!” rumors, it’s worth remembering that the girl famous for doing nothing (except repeatedly having her racy photos and video leaked) is retiring in two months. yep, on june th , paris is give up on public life. at least that’s what she said in newsweek: she’s certainly managed to turn herself into an icon and a conglomerate for essentially being a party girl—that is, for doing nothing. deloreans are back in this future if the delorean looks at all like a lotus esprit, it should. both of them were designed by giorgetto giugiaro, and much of the engineering work was done by lotus founder colin—to add speed, add lightness—chapman. amusingly, john de lorean also owned a company that manufactured snowcats under the dmc name. owners and wannabes can join the fun at the delorean motor company open house, being held june — in humble, texas. moveon: we can’t afford bad song parodies in yet another lesson about how a bad joke in front of one audience can trouble a larger public, moveon wants mccain to know bombing iran is no laughing matter. music and bombing, it could be said, really only go well together when joined in criticism. wordpress, permalinks, mod_rewrite, and avoiding s i made a mistake in changing my wordpress permalinks, but by the time i’d discovered it my blog had already been indexed. fixing the permalinks meant breaking those indexed urls, leading to a bad user experience, but leaving them as is wasn’t really an option. last night, after getting ’d while using google to search my own blog, i realized i had to do something. first i looked at apache mod_rewrite and the url rewriting guide (as well as this cheat sheet from ilovejackdaniels), then, frustrated, i found some items in the wordpress codex, including this one about conflicts between . some needs, some of the time i don’t know why i love this quote from a post in panlibus: serve some needs of some parts of the population, some of the time …though my love for the quote may have something to do with my embrace of what opensearch creator dewitt clinton describes as the “ % case,” the solution that would work for the great majority of applications most of the time. it’s one of those things that’s easy to see in retrospect, but difficult to aim for: building a tool that is specific enough to be useful, but not too specific. joost brings television to the internet age (finally) on demand internet tv has been just around the corner since the dawn of the popular internet, but like flying cars, it’s still not here. the problem is how tv streams clog the internet’s tubes. bandwidth may be cheap, but there’s still never enough of it. well, that’s true if your metaphor for the internet is a hub and spoke system. not so if you think of it as a mesh. usability, findability, and remixability, especially remixability it’s been more than a year since i first demonstrated scriblio (was wpopac) at ala midwinter in san antonio. more than a year since ncsu debuted their endeca-based opac. and by now most every major library vendor has announced a product that promises to finally deliver some real improvements to our systems. my over-simplified list said that our systems failed us in the categories of usability, findability, and remixability, and now people are asking me what i think about what i’ve seen from the vendors so far. my boston library consortium presentation speaking thursday at the boston library consortium‘s annual meeting in the beautiful boston public library, my focus was on the status of our library systems and the importance of remixability. my blog post on remixability probably covers the material best, but i define it as: remixability is the quality of a system or data set to be used for purposes the original designers or owners didn’t predict or intend. bsuite bug fixes (release b v ) [innerindex]work on bsuite is progressing well, thanks to help from zach and matt, who are collaborating with me on completely rearchitecting how stats are collected and reported. this, however, is not bs . it’s a transitional release intended to fix some bugs in b and make upgrading easier. this upgrade is recommended for all current bsuite users and new users. bsuite features tracks page loads (hits) tracks search terms used by visitors ariving at your site via search engines reports top-performing stories via a function that can be included in the sidebar reports recent comments via a function that can be included in the sidebar reports top search terms via a function that can be included in the sidebar outputs a pulse graph of activity on your site or specific stories lists related posts at the bottom of the current post’s content suggests posts that closely match the search criteria for visitors who arrive via search engines integrates bsuite_speedcache does some stuff with tags fixed/changed/added as mentioned above, a huge-but-invisible feature here is that this version includes some pieces that will make it easy to transition to the new plugin. mysql errors while creating the tables should now be fixed. it’s my shame that these have persisted so long. the plugin now “rebuilds the tags table” as soon as you activate it. this is a good thing, but if you’ve got a huge number of posts (or a really short max execution time) it might cause a problem (please leave a comment if it does). the related posts feature now works even if you aren’t tagging your posts. if there are no tags, the post’s title is used as a search string. this list is probably incomplete and in some other way inaccurate. it’s not intentional, i’m just sloppy. please leave comments with bug reports or corrections, i’ll do what i can to fix them. finally, i’m now hosting the download on a new server, so it won’t be subject to .mac’s bandwidth consumption limits. is the moller skycar a fraud? will i ever get my flying car? a recent comment here reminded me to check in on our options for flying cars, now at least seven years overdue. it turns out that moller international, the folks developing the m skycar aerodyne, are accepting deposits: as a result of the recent successful hovering flights of the m skycar, moller international is accepting deposits to secure delivery positions for our m skycar until after the skycar has flown from hover to full aerodynamic flight and returned (transitioning flight). yep, skulls are office products, brains not included i don’t know what’s funnier, that amazon sells skulls (just $ , get one now!), or that they’re classified as “office products.” extra: more office weirdness in this video. i’m a fonero, are you a fonero too? now that i’ve moved i’ve finally set up my fonera. i had hoped to offer a story about the process, but it was so simple i can’t really say much more than “i plugged it in, i registered it, it worked.” the fonera is a tiny little router/wifi access point that looks worlds better than the average linksys/netgear/belkin job, but the real sweetness is in what it does that they don’t do. google mymaps and georss o’reilly’s where . conference isn’t until the end of may, but google just released two sweet new map-related features: georss support and mymaps. the georss support means that any application that can output it’s geocoding — as simple as <georss:point> . - . </georss:point> — can now be linked to a live map with no more effort than it takes to paste the feed url into google maps’ search box. google holds this up as the exemplar, but i’m a fan of the cheese photo map here. twitter twitter anti-twitter my own feelings about twitter have gone back and forth across indecision street for a while, and despite a moment of excitement it’s still not part of my life-kit. so i was amused to see blyberg pointing out kathy sierra’s poo-poo-ing of twitter. ironically, services like twitter are simultaneously leaving some people with a feeling of not being connected, by feeding the fear of not being in the loop. by elevating the importance of being “constantly updated,” it amplifies the feeling of missing something if you’re not checking twitter (or twittering) with enough frequency. dawn of the citizen professor? it should be no surprise that journalists are talking about citizen journalism, but what of the disintermediation of other industries? man-on-the-street mark georgiev told marketplace: i didn’t want a certificate, i didn’t want any kind of accreditation, i really just wanted the knowledge. and i also wanted to work at my own pace. georgiev, the story explains, has a masters from yale but wanted to learn programming. that’s when he found foundations of software engineering in mit’s opencourseware. pranks international matt tells us the office pranks he masterminded a couple weeks ago got reported in saturday’s daily mirror (scan above): joker matt batchelder had the last laugh after he was left out of an office conference trip. alone at his desk for a week, the snubbed computer geek dreamed up a series of pranks to greet his boss and three colleagues as they returned… on april fool’s day. cut and paste is a skill too [update: keith pointed out that my small disclaimer at the end isn’t clear enough. this post is copied, stolen, cut and pasted in its entirety from keith’s blog, istp dad. i was glad to learn of the story, and this was meant to be ironic and funny.] an editorial in the washington post is explicit about a topic close to my heart: students think plagiarism is fine, and teachers (high school? moving and shaking and shimmy-ing it’s sort of late by now, and others have been offering their congratulations to me for a while (thank you, thank you, thank you, thank you), but i only just got the paper copy myself and this morning had a chance to browse the list. mover & shaker alumnus john blyberg asked me if i preferred moving or shaking better, but now that i’ve seen the names and read the profiles, i can say i’m just proud to be among such a distinguished group. [good|bad] covers: my humps, interpreted by alanis morissette i’m one of those guys who almost never actually hears the lyrics to the music that’s playing constantly. then somebody covers the song in a beautiful-but-ridiculous way, and i finally clue to them. example: tori amos’ cover of smells like teen spirit. now i hear alanis’ interpretation of the black eyed peas my humps, and i realize that, while not meaningless, it’s on par with lene alexandra’s current single. does it make me old to say that bad grammar in lyrics hinders my understanding of them? economics of open source two fairly old papers on the economics of open source. the news recently has been that open source allows companies to bring in better, more innovative talent and saves marketing costs, but these papers are interesting nonetheless. the simple economics of open source: the nexus of open source development appears to have shifted to europe over the last ten years. this paper explains why this trend undermines cultural arguments about “hacker ethics” and “post-scarcity” gift economies. “smart networks” are a stupid-bad idea this story in mit technology review scares me. instead of letting all computers within the network communicate freely, ethane is designed so that communication privileges within the network have to be explicitly set; that way, only those activities deemed safe are permitted. “with hindsight, it’s a very obvious thing to do,” mckeown says. no matter how obvious it seems, it’s still a really bad idea. it’s hard to imagine a world without the internet now, which makes it especially easy to dismiss the critical features that made it possible. sweet vespa scooter with sidecar on ebay greenstemstudios is selling a sweet-looking vespa with sidecar. in gleaming cinder red and house of kolar black, riding on white wall continentals, “the scooter gets to miles to the gallon and can easily maintain mph even with the sidecar attached.” the starting price is $ , . i’m plenty happy with my scooter, but this is very tempting. emi and apple/itunes to offer drm-free music downloads following steve jobs’ ant-drm post, people began to wonder if apple was just pointing fingers or really willing to distribute drm-free music via their online store. yesterday we learned the answer. apple and emi announced yesterday they would offer drm-free bit aac premium downloads, priced at $ . each. bisson tower siezed with plenty of moving help from zack, matt, cliff, justin, jon, will, and karen, bisson tower went from empty to full quickly enough that we all had plenty of time to sit around and enjoy the lunch sandee cooked up, then retire to the roof with cocktails. the cats were traumatized by it all, but i’m happy to be done with construction and finally be able to enjoy the new place, with all its quirks. web based genealogy software interesting, a lamp solution that promises “the next generation of genealogy sitebuilding.” it does pretty charts and pages, and as any web app should, makes it easy to edit or add information. but it also makes me wonder if there’s an xfn attribute to indicate parent/child relationships. could our work on network identity and social software solve this? for april fools… those looking for this year’s april fools gags should look at the office pranking from last week (pictured above). this blog will henceforth be very serious. not. dance around the world among the pop-culture viral videos i apparently missed is matt harding‘s dancing. i had to turn to wikipedia for an explanation: harding was known for a particular dance, and while videotaping each other in vietnam, his traveling companion suggested he add the dance. the videos were uploaded to his website for friends and family to enjoy. later, harding edited together dance scenes, all with him center frame, with the background music “sweet lullaby. whoosh boom splat bill gurstelle thought the exploding balloons were as funny as i did, and now i understand why: the contributing editor of make magazine knows his way around improvised munitions. he also knows youtube videos of oppressed geeks getting back at the man with potato guns is a good marketing ploy for his audience. whoosh boom splat appears to be his latest book. amazon doesn’t let me look inside, but how can you go wrong with projects like these? who will be first to put a metronaps pod in their library? metronaps started business in with a boutique in nyc’s empire state building, selling minute naps for $ bucks. the company has slowly been opening franchises around the world, but metronaps co-founder arshad chowdhury says overwhelming interest from office folks who wanted to install the pods on-site as an employee perk. so the company redesigned the pods to fit through the smaller doors common to office environments (trust me, retail doors are big), and has started selling direct. apis are big business programmableweb pointed out an informationweek story that claimed % of amazon’s sales in early were attributable to amazon affiliates. and c|net claims amazon now has , aws developers (up from the , amazon was claiming about a year ago). (note: not every amazon affiliate/associate is an amazon web services (aws) developer, but amazon hasn’t shared more specific numbers.) these slides, from amazon’s aws developer relations team explain a lot about what aws is. office prankd! when ken, zach, dan, and dee all went off to a conference without matt, al, cliff, tim, laurianne, and me (but especially matt), they had to assume something would happen in their absence. something. and it did. to each one of them in turn. , square feet of tinfoil covered everything in ken’s office. , post-it notes were tiled over everything in zach’s. cups (many had water in them) covered dan’s floor and desk. idm, openid, and attribute exchange the conversation on code lib about openid reminded me to finish a draft i’d started at identity future on the topic. the short of it is that marc canter says that single sign-on is good, but “we need the attribute exchange to make this thing really take off.” then all the skeptics will realize that the authentication layer had to come first – but was just a first step. along the way we’ll figure out standards for user intrerface and usage flow. japanese lessons from william rowe: zetcho = the apex of the mountain tonsei = to shave one’s head and forsake the world i learned the literal meaning of “karaoke” early last year. heavy skies newley purnell pointed me at this astronomy picture of the day by antti kemppainen: sometimes the sky itself is the best show in town. on january , people from perth, australia gathered on a local beach to watch a sky light up with delights near and far. nearby, fireworks exploded as part of australia day celebrations. on the far right, lightning from a thunderstorm flashed in the distance. near the image center, though, seen through clouds, was the most unusual sight of all: comet mcnaught. world’s smallest horse thumbelina is smaller than a decent dog. so small, in fact, that the guinness folks — no, not those guinness folks — recognize her as the smallest. from boing boing: thumbelina is the world’s smallest horse. she weighs lb and is five years old. she was born on a ranch that specializes in breeding miniature horses. she is thought to have dwarfism, which makes her even tinier. but she’s not alone. spring! spring flowers! uploaded from before the days when flickr would keep the original size photos, this is one of my favorite, most spring-y shots. and with weather like we’re having here now — ° in northern new hampshire! — it’s very appropriate. my personal crisis of digital preservation for a long time i was a big fan of dantz retrospect backup. for while i was so committed that i would do an incremental backup of my laptop and most every other computer in my house every day, but i’ve been using it one way or another since or or so. all those backups have added up, and they’ve even saved me a couple times. i wish, of course, that i’d been using it previously, when my laptop was stolen in , or when my hard drive failed catastrophically in . uc berkeley proud of powerpoint bob gaskins, a former berkeley ph.d. student, conceived powerpoint originally as an easy-to-use presentation program. he hired a software developer, dennis austin, in to build a prototype program that they called “presenter,” later changing the name to powerpoint for trademark reasons. powerpoint . was released in for the apple macintosh platform; later that year gaskins’s company forethought and the program were purchased by microsoft for $ million. the first windows and dos versions of powerpoint followed in . nyt struggles to find young audience, online audience, audience the new york times last week announced that it’s giving away timesselect to students and faculty that hold a .edu email address. timesselect, of course, is the paid access site that debuted in january to a confused and critical web. editor and publisher repeated the times’ claim that they’re doing this for the good of democracy: “it’s part of our journalistic mission to get people talking on campuses,” says vivian schiller, senior vice president and general manager at nytimes. snow spider karen found this spider in the snow yesterday when she wasn’t running for the camera. will spied several more, all moving laboriously over the crystalline landscape. none of us had ever seen spiders on snow before, but it’s likely we’d never looked. charlie the unicorn meg was never shy about asking me what rock i was found under when i stunned her with my complete ignorance of major pop culture touchstones, so i put my mind to it and after significant remedial work i thought i’d caught up. but, no. i’d not seen this video and only discovered it when blyberg pointed at it as an icon of network-enabled pop culture. the candy mountain video has been circulating for almost a year now and it’s a prime example of how network effects are allowing society to disseminate, in this case, popular culture, and ultimately the bulk of information deemed “important” by our fellow citizens snow thrower in my favorite action photo since will cut a woody, karen hit the snow with fury. i missed lebowski fest!?!? as usual, beatnikside had to tell me what i missed: lebowski fest. it looks like everybody was there. the dude jeffrey lebowski, theodore donald ‘donny’ kerabatsos, walter sobchak, maude lebowski, bunny lebowski, the rich jeffrey lebowski with no legs, and his lacky brandt. and don’t forget jesus quintana or treehorn’s thugs. and certainly don’t forget nihilists uli kunkel, karl hungus, kieffer, and franz. twittter twittter twittter ryan tried to tell me about it a month ago, jessamyn gets the idea but uses facebook instead, dewitt fell for it, ross said it tipped the tuna, and now i’m finally checking twitter out. i signed up yesterday and immediately went looking for ways to connect twitter, plazes, and ichat. tweet is an applescript that works with quicksilver (a launcher) and twitterrific (a desktop twitter client) to make updating even easier. oss saves marketing costs, protects business va linux founder larry augustin on oss in augustin’s view open source development became a necessity in the s when the cost of marketing a program came to exceed the cost of creating it. “my favorite is salesforce.com. in they spent under $ million in r&d and over $ million in sales and marketing. that doesn’t work.” “open source enables people to reach all those customers. it’s a distribution model. beyonce and swimsuits not appropriate for librarians my ala email newsletter arrived today with this story: sports illustrated decides libraries don’t need swimsuit issue librarians on publib and other discussion lists discovered in the first week of march that none of them had received the february “swimsuit issue” of sports illustrated. inquiries to publisher time warner eventually resulted in a statement from spokesman rick mccabe that the company had withheld shipment of that issue to some , libraries and schools because for years the magazine had received complaints it was too risqué. linux leads on world’s top supercomputers the real map of the world’s top supercomputers isn’t nearly as us-centric as my screenshot suggests, but the operating system stats are seriously tilted toward linux. over of the top supercomputers in the november report run some form of the free operating system. generic “linux” leads the pack, but redhat and suse are the two most named distributions. non-free operating systems include ibm’s aix, hp-ux, and macos x. spam getting more personal? the viagra and cialis knock-offs being pushed in so much of the spam i get may be directed at things the recipients feel very personally about, but the message itself has never been personal. well, it had never seemed personal to me, anyway, until now. clay shirky pointed out what i’ve started to see, and wonder about, myself: many of the subject lines in the spam i’ve received recently sound familiar, and plausible as a real message. the future of library technology is free, cheap, and social delicious = endoeavor’s course content integrator opensearch = metasearch flickr = digital collections management damn daylight saving doesn’t save npr covered it like an eclipse or astronomic curiosity, and did little to question the claimed energy saving benefits. but, as michael downing asks in spring forward, how can something understood by so few be done by so many? and why go through this twice annual madness? supposedly, we subject ourselves to the rule of time to conserve oil, but even the most wildly optimistic predictions suggest only a % drop in consumption. firecrackers for troops via npr this morning: a michigan man strapped more than , firecrackers onto himself, and lit the fuse. john fletcher publicized it as an effort to support u.s. troops. it was an event to collect cell phones for soldiers. the daily press and argus, in livingston county, mich., shows fletcher standing calmly as the firecrackers explode. afterward he did say he needed some tylenol. livingstondaily.com has has video as well as photos of the fiery seconds of firecracker fury, which worked out a whole lot better than this other soldier-related firecracker stunt. : a torrent of awesomeness or just too much? so, is really the “torrent of blood and awesomeness” that matt says it is (and the preview supports), or does it run out of steam as npr’s film critic, kenneth turan, suggests? unless you love violence as much as a spartan, quentin tarantino, or a video game playing teenage boy, you will not be endlessly fascinated. the problem is that the visual panache that made snyder an acclaimed director of commercials works better for second spots than two hour features. and he-man screams from the top of his lungs “what’s goin’ on” the what’s up? cover would be funny enough on its own, with the he-man video it’s golden. now, you know you want to sing along with the chorus. go for it, here are the lyrics: and so i wake in the morning and i step outside and i take a deep breath and i get real high and i scream from the top of my lungs “what’s going on? charges put internet radio on pause in early the copyright arbitration royalty panel (carp) set royalty rates for webcasters that were twice as high as for regular radio broadcasts. the library of congress reset those rates in late summer (yes, the loc oversees those things). now it’s , and the riaa is at it again. techdirt reports the copyright royalty board is adopting royalty rates the riaa has been asking for, “and making them effective retroactively to the beginning of — meaning that many small independent webcasters are now facing a tremendous royalty bill they’re unlikely to be able to afford. the true spirit of copyright i wrote to c|net, owner of techrepublic and builder.com, asking if i could quote their ten commandments of egoless programming in an issue of library technology reports journal on open source software for libraries and got the following canned response: thank you for your interest in including cnet content on your website. […] there would be licensing fee of $ . associated with use of the cnet logo or text excerpt on your website, or $ . ingenious and almost unusably different lars wirzenius’ linux anecdotes: in january, linus bought a pc. he’d been using a sinclair ql before that, which, like much british computer stuff, was ingenious and almost unusably different from everything else. dell tells linux users where to put it holy smokes. as dell’s sales slump and stock remains flat, the famously unimaginative company is trying to tap into the mob for ideas about what new shade of grey to deliver its hardware in next. and what did the dell ideastorm mob say? “give us linux!” “give us openoffice.” and how did dell respond? “no. no. and, no.” john naughton reports on the story for the guardian, explaining: waiting for mac os x . leopard with rumors of a march release of mac os x . leopard, swirling, zach asked what was promised that he should be excited about, so i went looking to jog my memory. the announced features include time machine automatic backup of all your stuff (with integration to make finding and restoring stuff in applications easy and sweet, watch the video already), as well as a big leap ahead for ichat. internet awesomeness diagram by matthew batchelder above, matthew batchelder’s diagram showing the correct relationship of the internet, awesomeness, ninjas, pirates, dinosaurs, zombies, robots, and gummi bears (though, where are the superheros you might ask). this guy can draw circles around you (and me) found at baekdal.com, where the author expresses some amount of whiteboard-skills envy. the video shows alex overwijk, head of glebe collegiate high school‘s math department (more trivia: alanis morrisette went there) drawing what appears to be a perfect circle. this is something i do in my spare time. i draw freehand circles and then i found out there was a world championship…it’s like winning the masters. once you win, you automatically get invited back every year. google apps and roadshow i was supposed to go to the what i think is a google apps roadshow this morning, but i was also supposed to be at code lib this weeks and be doing a dozen other things that didn’t happen. so, in lieu of that i’m reading up on the company’s first new business strategy since adsense. phil wainewright is skeptical, even mocking at the likely prospects for the premium package that google is offering for about $ per person, per year. links from ryan eby encyclopodia – the encyclopedia on your ipod geocool! – rasmus’ toys page ie and opensearch autodiscovery information management now: social tagging for the enterprise let me show you my credentials “i’m bruce pechman, the muscleman of technology, let me show you my credentials.” this is the instructional video that comes with the dynaflex powerball gyro. the fan videos on youtube have got nothing on this. just click play and prepare to laugh. will and i have been asking to see people credentials since he shared this with me a week ago. middlebury college vs. wikipedia middlebury college is proud to have taken a stand against wikipedia this year: members of the vermont institution’s history department voted unanimously in january to adopt the statement, which bans students from citing the open-source encyclopedia in essays and examinations. without entirely dismissing wikipedia — “whereas wikipedia is extraordinarily convenient and, for some general purposes, extremely useful…” — the decision paints it with a broad brush — “as educators, we are in the business of reducing the dissemination of misinformation. wwan update brings higher speed-mobile connectivity apple’s wwan support update . brings support for the following new cell carrier-based based networking cards (wwan = wireless wide-area networking): available on the cingular network novatel merlin xu expresscard (hsdpa) available on the sprint network novatel wireless merlin ex express card (evdo rev. a) novatel wireless ovation u usb modem (usb adapter, evdo rev. a) available on the verizon network novatel xv expresscard (evdo rev. top ten times two for students back in august educated nation offered the following top ten list of web tools for college students: writely soundslides bluedot.us efax pdf online google calendar google spreadsheets bloglines technorati mynoteit not to be outdone, an anonymous-but-first-person story at nextstudent identifies their top ten: book finder mynoteit ottobib google docs tada list meebo wikipedia zoho show google reader del.icio.us quiet comfort that’s me on jetblue flight to long beach, wearing my noise canceling headphones. sandee saw me wanting them, so she was especially happy to make them a christmas present to me. and, with all the flying i’ve been doing lately, i was especially happy to have them. i wanted the quietcomfort s not just because i like big, old skool, over-the-ear headphones (i don’t, actually), but because i really wanted the extra noise reduction that design offers. let it snow! with over a foot on the ground already, and more falling now the through the night, we’re crossing our fingers for another snow day tomorrow. foods i want to try… despite the mystery, porklets are quite yummy, at least according to sandee‘s recipe. what i want to try next is bacon cheesecake or chili powder on french toast or maraschino cherries mixed with jalapeños. all of those sound delightful to me. extra: sausage man, don’t eat that, don’t try this at home. just pretend it’s all okay ryan im’d this to me, and it was pretty easy to find that northern sun sells them for $ a pop. this is serious stuff, but it’s hard not to laugh at the support our pants magnet or some of the stickers here. this blog is for academic and research purposes only this sign on a computer in the paul a. elsner library at mesa community college caught beth‘s eye and garnered a number of comments, including one from theangelremiel that seems to mark one of the most elusive aspects of library . . they know that none of their classes require gaming excerpting the above as a simple declarative may not be fair, but it gets to the point. let’s say they “know” (that is, let’s say they think they know) that none of the courses requires gaming. treo firmware, dun, frustration john commented to say he’s been using his for dun over bluetooth for a long time now, and that all it takes is the latest firmware. so i go looking and find treo updater . from october and i have to wonder “what firmware does my phone have?” here’s how to check: open the phone application, press ‘menu’, navigate to ‘options’, then ‘phone info’ of course nothing is simple, and a treoaddicts story notes trouble with the update, and the installation instructions are daunting (really, look at ’em). a visual explanation of web . kansas state university‘s digital ethnography group — “a working group of kansas state university students and faculty dedicated to exploring and extending the possibilities of digital ethnography” — posted this visual explanation of web . . it’s by michael wesh, assistant professor of cultural anthropology, and it rocks. text is unilinear…when written on paper. digital text is different. hypertext can link. with form seperated from content, users did not need to know complicated code to upload content to the web. steve jobs’ thoughts on music, music stores, and drm steve jobs’ thoughts on music is surprisingly open and frank, almost blog-like, for the man and the company especially know for keeping secrets. jobs is addressing complaints about apple’s “proprietary” drm used in the itunes music store. there is no theory of protecting content other than keeping secrets. in other words, even if one uses the most sophisticated cryptographic locks to protect the actual music, one must still “hide” the keys which unlock the music on the user’s computer or portable music player. no one has ever implemented a drm system that does not depend on such secrets for its operation. and after offering his view of the situation, he offers three possible futures. the first alternative is to continue on the current course, with each manufacturer competing freely with their own “top to bottom” proprietary systems for selling, playing and protecting music. and the case for doing more of the same is pretty clear. apple’s ipod and itunes music store are successful, and though there are competitors, they’ll have to convince would be buyers to give up their ipods. the second alternative is for apple to license its fairplay drm technology to current and future competitors with the goal of achieving interoperability between different company’s players and music stores. and that’s exactly what people have been asking for. it’s hard to know who wants to use a player that’s not an ipod, but there are some things that don’t play on ipods. but… apple has concluded that if it licenses fairplay to others, it can no longer guarantee to protect the music it licenses from the big four music companies. perhaps this same conclusion contributed to microsoft’s recent decision to switch their emphasis from an “open” model of licensing their drm to others to a “closed” model of offering a proprietary music store, proprietary jukebox software and proprietary players. and finally… the third alternative is to abolish drms entirely. and how does that work? in , under billion drm-protected songs were sold worldwide by online stores, while over billion songs were sold completely drm-free and unprotected on cds by the music companies themselves. the music companies sell the vast majority of their music drm-free, and show no signs of changing this behavior, since the overwhelming majority of their revenues depend on selling cds which must play in cd players that support no drm system. so if the music companies are selling over percent of their music drm-free, what benefits do they get from selling the remaining small percentage of their music encumbered with a drm system? there appear to be none. if anything, the technical expertise and overhead required to create, operate and update a drm system has limited the number of participants selling drm protected music. if such requirements were removed, the music industry might experience an influx of new companies willing to invest in innovative new stores and players. this can only be seen as a positive by the music companies. connectile dysfunction no sooner do i lay down a rant about how bad sprint wifi is than do they run an ad telling us how great their service is. well, not only that, but they promise to save us from “connectile dysfunction.” angela natividad described it best: it’s hard to position broadband ads. you can be like earthlink, which kind of laughs at the whole idea of marketing in general, and you can be like comcast, which takes the easy way out with off-colour humour. wasn’t like for those who watch the ads as intently as the game, it’s hard not to think of apple’s commercial. and from that thin thread, i’m reminded of the ministry of re-shelving and, now, the ministry of love. i discovered the last from a comment here, and after looking them up, i decided to contribute a few copies to the cause. the notes i sent along requested the following: sprint wifi sucks i’m back in oakland airport, but this time i’m bringing my own network and i don’t have to deal with sprint’s wifi mess. see, the problem isn’t just that it costs too much. the problem is that once you pay, you’re plopped at the login page where the login i just created doesn’t work. and worse, the error offers absolutely no clue about why the username i just just created (and paid for! social internet sharing it all started as a simple idea. why should you pay for internet access on the go when you have already paid for it at home? exactly, you shouldn’t. so we decided to help create a community of people who get more out of their connection through sharing. the deal is that you get a special wifi router and use it to securely open your connection to the world. ecto vs. wordpress ecto is finally available in intel optimized form, but wp . ‘s xmlrpc breaks it. cliffy, of all people, tells us how to fix it. now, when is ecto coming out? aside: this blog post explains how to hack up the xmlrpc to extract the tags ecto is sending. this was interesting to me a long time ago, but bsuite handles tags entirely in the post content. open source shifts costs does open source free your budget up for the best talent? i asked her if the choice to go with open source is helping her to keep costs in check, here’s what [dabble ceo mary hodder] said: what happens with open source is you actually spend the same amount of money, but you don’t have lock-in and you pay for really good people to run it. and so you still end up paying. neg’s urban sprinting i might watch more tv if i didn’t live in the us. well, i used to like watching world’s wildest police chases on spike while knocking back a few at the bar after work, but they re-arranged the schedule a while back and it’s just not the same. so clearly i have to sit around waiting for people to forward me goodies like this. yeah, it’s neg’s urban sprinting, which apparently aired on a show named “balls of steel,” and it’s just one in a brilliant series. sealand for sale principality of sealand, a wwii-era gunnery platform called roughs tower, in the north sea outside britain’s pre- three nautical mile claim of sovereign waters, is for sale. yep, the “land” declared by some as the world’s smallest micronation will go to the highest bidder. ravage by fire ( ), beset by marauders ( ), and generally ignored by the world’s governments (all time), it’s, well, it is what it is. and now the pirate bay hopes to buy sealand. communities are as communities do right there are the beginning of esther dyson‘s ten-year-old book, release . , she alerts us to the web . challenge we’re we’re now beginning to understand: the challenge for us all is to build a critical mass of healthy communities on the net and to design good basic rules for its public spaces so that larger systems do self-organize and work effectively. rule-making is not the job of legislatures and governments alone. presentation: collaboration, not competition ala midwinter , alcts future of cataloging presentation: collaboration, not competition. (slides: quicktime & pdf.) stir my writings on the google economy and arrival of the stupendous post with frame four of the alcts and the future of bibliographic control: challenges, actions, and values document: in the realm of advanced digital applications, we are interested in collaboration, not competition. we take as axiomatic the idea that library catalogs and bibliographic databases on the one hand, and web search engines on the other, have complementary strengths. presentation: faceted searching and our cataloging norms ala midwinter , alcts cataloging norms discussion group presentation: metadata and faceted searching: an implementation report based on wpopac. (slides: quicktime & pdf.) faceted searching such as that made possible by wpopac (look for the new name soon) improves the usability of our systems and findability of our materials, but also puts new demands on how we catalog them. my favorite search example is sociology of education, both because it’s a common search in our logs, but also because it demonstrates how our systems can help bridge the gap between what our users know and what our catalogs know. casual friday: the ala midwinter + music video edition the above circulated a while ago, but i post it today to recognize this special ala midwinter edition of casual fridays. and while i’m not suggesting libraries will or should become st century dance halls, lichen’s title, “ . -> . , the video” has some resonance here. and on the theme of music videos that tell stories comes miranda’s yo te dire, which i like both because it’s funny and because i’m instantly attracted to foreign pop culture. let the silence roar okay, before anybody inquires if i’ve gone into boat sales or brings up the bisonboom story again, i need to ask for your understanding. it’s not that i’ve been spending my days trying to pick out just the right shade of red for my new corvette (really i’m not, it’s the lotus i like), or that i’ve been moving to sunny california to take up my new job at google (a year ago i would have been twitching with excitement, now i’m more likely to agree with this). sweet jquery matty discovered jquery at the ajax experience, and his enthusiasm has rubbed off on me. jquery makes coding javascript fun again. well, at least it makes it possible to write code and content separately. and that means that sweet ajaxy pages can be made more easily, and it sort of forces designers to make them accessible from the start. resources: jquery: javascript library getting started with jquery visual jquery . pes films i’ve been loving the pes films i found via this design observer post, and despite featuring his films for christmas day and new year’s eve, there’s still a lot to see. animated peanut butter is about as cool as it gets, even if i can sympathize with the peanut here in drowning nut. casual friday extras that tickle my inner -year-old: roof sex, beasty boy, pee-nut, and prank call. apache . .x on mac os x i’m lazy, that’s all i can say to explain why i hadn’t put any serious thought into upgrading from the . .x version of apache that ships with mac os x to the much more feature rich . .x or . .x. but today i found reason enough to switch my development to . . , and i went looking to the community for information about the switch. a post in marc liyanage’s forums made it clear how easy config/compile was. rusty nail: the maison bisson winter drink the holidays are long since past, here’s a drink to carry you through ’till spring. rusty nail parts scotch part drambuie serve over ice in an old fashioned glass. please enjoy it responsibly. lies, damn lies, and statistics thanks to metafilter for pointing this out, and matty, for putting it to good use. yes, you really can use this to make authoritative looking reports on anything. new year’s fireworks pes offers these fireworks for any occasion, but when better to celebrate than the new year? and thinking of that, if all these clocks are correct, the new year has already started in gmt, which means i’m probably a few drinks behind and need to catch up. holiday violence by the end of it, all the wrapping paper and other material affects of the holidays really do take on air of violence. well, at least they do in pes‘s kaboom. and if you’re amused by that, you might want to see how it was made. happy holidays one goat down, one goat to go cliffy got excited about the gävle goat when his pal derek emailed him about it all. derek was in town, or something like that, and got caught up in the frenzy first hand: “last year some other guy was a bit smarter, hitting it with a flaming arrow from a bow, and he wasn’t caught. it went up in flames!” the goat, of course, is a year holiday tradition. great white solstice while northern-hemisphere inhabitants are enjoying their first day of winter, our cousins in the southern hemisphere are just beginning summer. and in south africa’s shark bay, near gansbaai, the great whites are departing for other waters. the great whites make their way to shark bay annually between september and january, though they are not hunting, and, as rob mousley reports, they “ignore bait slicks (and bathers), swimming through them without any reaction–in contrast to their behaviour at other locations such as dyer island” [link added]. competition, market position, and statistics watch this video a few times. it’s funny. it’s catchy. it’s kitsch. now watch it a few times more. the ad, for a lada vaz , appeared sometime in the s. it reflects the influence of mtv and other cultural imports from the west, but the details betray it’s command economy provenance. the snow appears trodden and dirty, the trees barren, the background architecture bleak. the car has headlights that flash in time to the music, but their dim yellow glow fails to dazzle. welcome to your world in pointing this out to me, lichen noted “if this isn’t evidence that web . is an undeniable force, i don’t know what is.” “this,” of course, is time magazine‘s announcement of the person of the year. and the answer is you. yes, you. michael stephens was right on top of it, pulling this quote: …but look at through a different lens and you’ll see another story, one that isn’t about conflict or great men. helsinki complaints choir though some people prefer the birmingham choir to helsinki’s, there’s certainly something to be said about complaining in song, and something more when it’s in a language i can’t begin to understand. one blogger remarked of the video: to think of what might of been. what if i’d moved in with a bunch of angst ridden finns,instead of pseudo-happy baptists, and been forced to sing their rants along with them. wish i could be there… harry shearer and judith owen are performing their holiday sing-a-long at the concert hall at the society for ethical culture in nyc with guests tmbg and others. it’s a go on friday, but why can’t these things happen closer to me? actually, maybe they should all come to warren afterwards. memcached and wordpress ryan boren wrote about using memcached with wordpress almost a year ago: memcached is a distributed memory object caching system. wordpress . can make use of memcached by dropping in a special backend for the wp object cache. the memcached backend replaces the default backend and directs all cache requests to one or more memcached daemons. you must have a memcached daemon running somewhere for this to work. unless you’re managing the server on which your blog is running, you probably can’t run a memcached daemon, making this backend useless to you. wordpress . + wpopac i’ve been following wp . development, but aaron brazell’s post in the development blog wrapped up a lot of questions all at once. the short story is that . is going to bring some really good changes that will allow more flexibility and better optimization of wpopac. of the four changes brazell names, the last two, the addition of the post_type column and a change in usage of the post_status column, are where the money is. woot! woot! the press release: making libraries relevant in an internet-based society psu’s casey bisson wins mellon award for innovative search software for libraries plymouth, n.h. — you can’t trip over what’s not there. every day millions of internet users search online for information about millions of topics. and none of their search results include resources from the countless libraries around the world—until now. casey bisson, information architect for plymouth state university’s lamson library, has received the prestigious mellon award for technology collaboration for his ground-breaking software application known as wpopac. flightplan perhaps it’s just because i’m in the air again today, but i’m fascinated by aaron koblin‘s animation of aircraft activity, illustrating the pulsing, throbbing movements of aircraft over north america. nah, this is hot. you’ll love it too. also worth checking out: koblin’s other works. flickr interstingness patent…application it’s old news (boing boing and slashdot covered it a month ago), but flickr’s patent application is a bit troublesome. it’s not that they’re trying to patent tagging (they’re not), it’s that they’re trying to patent the things library folks have been wanting to do (and in some cases actually doing) for some time. media objects, such as images or soundtracks, may be ranked according to a new class of metrics known as ”interestingness. lemurs movin’ it thank jon for pointing out the above. actually, you should go read his post on the matter because, well, it gave me a chuckle and it’s certainly better than going shopping today. and then the feds blocked me via a friend who coordinated a program i presented at not long ago i received this message about difficulty accessing my blog post with notes from the presentation: do you have the notes electronically that you could send? believe it or not our federal government internet filter is blocking access to the blog site below…..big brother is truly at work these days….. jessamyn has been dealing with this for a while now, but this is the first i’d learned that i’d been blocked. will it blend? go now to willitblend.com and offer your suggestion for something new. want to see a bacon cheeseburger with pickles and grilled onions? go for it. parsing marc directory info i expected a record that looked like this: leader nas ia c mau u p uuua eng /rev dlc|caug psmm f . |b.a f . |b.a appalachian mountain club the a.m.c. white mountain guide :|ba guide to trails in the mountains of new hampshire and adjacent parts of maine amc white mountain guide white mountain guide a. second school? rebecca nesson, speaking via skype and appearing before us as her avatar in second life, offered her experiences as a co-instructor of harvard law school‘s cyberone, a course being held jointly in a meatspace classroom and in second life, and open to students via harvard law, the harvard extension school, and to the public that shows up in second life. nesson has an interesting blog post about how it all works, but she also answered questions from the audience about why it works: social learning on the cluetrain? they don’t want to engage in chat with their professors in the classroom space, they want to chat with other students in their own space. — from eric gordon’s presentation this morning. hey, isn’t that the lesson that smart folks have been offering for a while now: “nobody cares about you or your site. really.” how could learning environments not be subject to the same cluetrain forces affecting the rest of the world? social software in learning environments it’s really titled social software for teaching & learning, and i’m here with john martin, who’s deeply involved with our learning management system and portfolio efforts (especially as both of these are subject to change real soon now). aside: cms = content management system, lms = learning management system. let’s please never call an lms a cms…please? on the schedule is… social software in the classroom: happy marriage or clash of cultures? displaying google calendars in php ical php icalendar solves a couple problems i’m working on, but i needed a solution to fix the duration display for gcal-managed ics calendars. as it turns out, a fix can be found in the forums, and the trick is to insert the following code in functions/ical_parser.php. case 'duration': if (($first_duration == true) && (!stristr($field, '=duration'))) { ereg ('^p([ - ]{ , }[w])?([ - ]{ , }[d])?([t]{ , })?([ - ]{ , }[h])?([ - ]{ , }[m])?([ - ]{ ,}[s])?', $data, $duration); $weeks = str_replace('w', '', $duration[ ]); $days = str_replace('d', '', $duration[ ]); $hours = str_replace('h', '', $duration[ ]); $minutes = str_replace('m', '', $duration[ ]); $seconds = str_replace('s', '', $duration[ ]); // convert seconds to hours, minutes, and seconds if ($seconds > ) { $rem_seconds = $seconds % ; $minutes = $minutes + (($seconds - $rem_seconds) / ); $seconds = $rem_seconds; } if ($minutes > ) { $rem_minutes = $minutes % ; $hours = $hours + (($minutes - $rem_minutes) / ); $minutes = $rem_minutes; } $the_duration = ($weeks * * * * ) + ($days * * * ) + ($hours * * ) + ($minutes * ) + ($seconds); $first_duration = false; } break; hopefully this gets worked into the baseline with the next release. rock paper scissors this weekend’s fifth annual rock paper scissors world championships have ended, and brit bob cooper has come out a winner. the toronto event drew a reported competitors and spectators from u.s. states, four canadian provinces, norway, new zealand, australia, wales, the uk and ireland and paid a top prize of can$ . “i went through extensive training, read ‘the official rock paper scissors strategy guide’, and studied the possible rps gambits before competing,” said cooper. mushaboom remix props to tim for offering linking me to a remix of feist’s mushaboom. i like the original better, but, well, i’m also a fan of remixes. i feel great transcipt: what? oh, yeah. i feel great. larry, i’m quittin’ the company and startin’ my own. and by the way, i feel great. steve, you’re a great guy with great skills, you’re gonna do great. *pounds fist* what the hell, i’m comin’ with ya. ooohhhhfff. hey, you’re hot and i feel great. let’s get married. alright, but i want lots of kids. me too. five hundred of them. *slams file drawer* ooohhhhfff. and fell the wall it’s worth taking a moment to remember that the berlin wall fell this day in . though orders had been been given, they were botched by east german propaganda minister günter schabowski, who mistakenly announced in a press conference that restrictions on border crossings would be lifted immediately. in fact, restrictions were to be lifted the next day. tens of thousands of east berliners heard schabowski’s statement live on east german television and flooded the checkpoints in the wall demanding entry into west berlin. art vs. the google economy in an anomaly that we would eventually recognize as commonplace on the internet, touching the void, a book that had gone out of print, remaindered before it hit paperback, was all but forgotten, started selling again in . chris anderson wondered why, and found that user reviews in amazon’s listing of publishing sensation into thin air had people recommending touching the void as a better read. today, touching the void outsells into thin air to . ministry of truth = george bush’s whitehouse the huffington post pointed out how the white house is doctoring video of bush’s “mission accomplished” speech from may . visitors to whitehouse.gov now get a video that crops out the mission accomplished sign. how orwellian will this president get? “the future of evil is in manipulating information.” i hope you’re all voting today okay, even if this diesel sweeties cartoon is a little disheartening, please vote. the fact is, vote suppression is probably more likely than vote fraud. a tip of the hat to lichen for alerting me to this, and for making the point that our users’ notions of “authority” are among the fastest changing features of our post-google world. arlington east the above photo and some others were forwarded to me by a friend. the body of the email included: a few friends of mine participated in this event on saturday. there wasn’t a lot of media coverage, but npr and the cct. the photos show markers representing american dead in the iraq war, and markers representing just a small percentage of the approximately , iraqi dead from a memorial held on cape cod on october th, . the political parties in vermont cliff took a picture of his absentee ballot because the new parties were just too good: dennis morriseau is the impeach bush now candidate for congress and peter moss is the anti-bushist candidate for senate. midterms mentioned earlier, but worth mentioning again: truemajorityaction’s take it back campaign. among the videos and political graffiti of the moment, don’t miss freedom, beat box bush, and <a href="http://video.google.com/videoplay?docid= ” title=" / , shock & awe: clip from “hijacking catastrophe” - google video">hijacking catastrophe. and as funny as the brazillion joke is, we need a government that doesn’t lie, a government that’s smart, a government that cares for its people, its soldiers and foreign civilians and our elections. network-enabled snooping in the physical world we’ve got ocr. we’ve got cameraphones. we’ve got web-based license plate lookup services. amazon japan has a fancy cameraphone-based product search feature. what’s more naive, imagining that somewhere somebody has a sms/mms-based license plate snooping and facial recognition services and fingerprint scanners, or imagining that they don’t? political graffiti found by lorelei in copenhagen. discovered by kieran’sphoto’s’ in cork. freedom (video) karen forwarded mgarthoff‘s freedom, tagged: bush war election midterm iraq katrina on youtube. presentation: designing an opac for web . maiug philadelphia: designing an opac for web . (interactive quicktime with links or static pdf) web . and other “ . ” monikers have become loaded terms. but as we look back at the world wide web of , there can be little doubt that today’s web is better and more useful. indeed, that seems to be the conclusion millions of americans are making, as current estimates show over million users in the us, including % of youth - . advice you didn’t ask for on writing: first figure out your story, then tell it. anything else is masturbatory. the solution is in your hands currugated_film‘s photo of graffitti in oaxaca. the caption at flickr notes that the text to the right says “the solution is in your hands, the rocks are on the ground.” two ton: one night, one fight tony day is june th, but today is the day i received my copy of joe monninger’s latest work, two ton: one night, one fight — tony galento v. joe louis. i learned a lot about the characters and times during the two years of research joe invested in the book, but other than sneaking peaks at the manuscript, i’ve not had a chance to learn the whole story of how tony galento ended up in the ring against joe louis — and knocked him down. all about atlatls…or…humans need to throw things in classic wikipedia-voice, an atlatl is… an atlatl (from nahuatl ahtlatl [?ah.t?at?]; in english pronounced [???t?l??t??] or [??t?l??t??] ) or spear-thrower is a tool that uses leverage to achieve greater velocity in spear-throwing, and includes a bearing surface which allows the user to temporarily store energy during the throw. […] a well-made atlatl can readily achieve ranges of greater than meters. atlatl bob describes it more passionately: damn that’s big the switzerland‘s verzasca dam is now added to the list of places i’d like to visit. linkability fertilizes online communities redux i certainly don’t mean this to be as snarky as it’s about to come out, but i love the fact that isaak questions my claim that linkability is essential to online discussions (and thus, communities) with a link: linkability fertilizes online communities i really don’t know how linkability will build communities. but we really need to work on building support platforms for the public to interact with the library and promote social discussions, whether offline or online. googlesmacked at a time when people are still wowing over the google-youtube deal (and wondering why their . company didn’t get bought for $ . billion), it’s good to know that marc cantor is dead down on it. not because of the copyright issues or “limited” advertising potential of youtube that others cite, but apparently because he just doesn’t like google anymore. to wit, he names orkut as a failed social network; knocks blogger as an also-ran; disregards google base as pointless; labels adsense a $ billion cash machine for sergey, larry and eric; tosses aside gmaps, gmail, gcalendar, gscholar, gbooks, and gtalk as “unrelated, random output of the labs, thrown up to justify their r&d expenditures;” and closes with an ominous warning: cheap and broken above, one of sandge‘s contributions to the the toy cameras pool reminds us that good photography is something that often happens despite the equipment, not because of it. of course, no sweeping generalization can go without argument, and in this case i think the toy camera enthusiasts would be joined by the glitch art aficionados, like roninvision, who apparently made a mistake while scanning to give us this: flipbook animation i love this flipbook animation on youtube (jump ahead to about : for it), even if the live-action preface is somewhat tiresome. and even with that, it still doesn’t rate as bad as some viewers think it is. this is the “making of” / behind-the-scenes sneak peak at my upcoming movie “annihilation”. i had hoped to finish annihilation in time to turn it in for my cinema class, but i didn’t… so i had to make a movie about my failure to complete the movie, and turn that in instead. cataloging errors a bibliographic instruction quiz we used to use asked students how many of dan brown’s books could be found in our catalog. the idea was that attentive students would dutifully search by author for “brown, dan,” get redirected to “brown, dan -,” and find three books. indeed, the expected answer was “three.” as it turns out, my library has all four of dan brown’s published books, including the missing digital fortress. what do you call a group of ninjas? from askmefi: “you know, like gaggle of geese, murder of crows, school of fish, all that. does a group of ninjas have some sort of descriptor? we’re talking many people in halloween costumes, how to address them together. the { blank }.” aside from the inevitable brush to ask a ninja, answers included: sir, sir, sir, and sir one ninja, many ninjim. and the collective is a flipout of ninjim a hedge of ninjas. the candy bar metaphor eleta explained it this way, and credited it to r. david lankes: your data: your _meta_data: butane handwarmer mt. moriah, this time better than last time. eat-rite diner, st. louis mo some time ago in st louis, i stumbled upon eat-rite diner. aparently i wasn’t the first to be taken in by its charms. yelp notes: this is a must in st. louis. however don’t go here for the friendly staff, good food, or fun atmosphere. this place is a joke! they will need to buzz you in the door to come in and try the delightful slinger. eat right or don’t eat at all! teddy bear kills , fish from associate press: concord, n.h. — a teddy bear dropped into a pool at a hatchery in milford, n.h., killed all , rainbow trout living in the pool. fish and game department hatcheries supervisor robert fawcett said the teddy — dressed in a yellow raincoat and hat — clogged a drain earlier this month, blocking oxygen flow to the pool and suffocating the fish. in a statement, fawcett noted: “release of any teddy bears into fish hatchery water is not permitted. what’s so great about adium? brian mann calls adium “one of the best multi-network [im] clients ever.” tim bray says it has a “wonderful user interface,” while also naming im generally “an essential business tool.” eric meyer, meanwhile, exclaims “adium is my new chat buddy.” what’s so great about adium? gaim is the engine behind the scenes, but the face of the application is xhtml and css. wit meyer: the entirety of an adium chat window is an xhtml document that’s being dynamically updated via dom scripting—all of it pumped through webkit, of course. isbn api followup a couple questions about my api to convert digit isbns to digits pointed out somethings i failed to mention earlier. first, the api actually works both ways. that is, it identifies and validates both and digit isbns on input, and returns both versions in the output. example: and - . and, as yet, i have no user agreement or usage policy. except for the disclaimer — don’t blame me if it’s broke — i’m leaving this open (though i’ll probably have to figure something out for future apis). inclusion is addictive lichen, who’s had a great string of posts lately, pointed out amy campbell‘s website, which opens with the following: so i guess this myspace thing is going to catch on. i resisted for a long time. these things make me nervous – myspace, messenger, emoticons… i can’t help but see it as some sinister forerunner of the complete degredation of language and of human interaction. i’m worried about a generation of people who’s definition of “friendship” consists first and foremost of an anonymous exchange of links. my own garlitz bob garlitz dropped by with a couple canvases yesterday — untitled and teng. it’s an honor i’d appreciate even if i wasn’t looking for something to cover my bare office walls. converting between isbn- and isbn- david kane asked the web libbers: can anyone tell me what the conversion between isbn- and isbn- is, please. i need to write a little conversion program. anything in php, for example. answers: “there is already an online converter: http://www.isbn.org/converterpub.asp;” some pointing at wikipedia on isbns, bookland, and eans; john blyberg’s php port of the perl isbn- / tool; some explanation that you have to watch the check digit, and discussion about why you’d need to do all this conversion. i am not a terrorist i am not a terrorist. i am not a terrorist. i am not a terrorist. democracy now! burning patriotism! beat box bush and dj cheney bush speech mashups rock. from google video: so, you wanna learn how to beatbox? gwb is back with another amazing performance. surprisingly he is actually very good. previously: state of the union? not good. also, note the tags on that video, and the way somebody snuck “????? ??? ? ???” past the filters. teddy bear cries red tears southtyrolean, who seems to take an interest in found graffiti posted this one (from graz)to his flickr stream, describing it: in the sackstraße, near kastner&Öhler (entrance to the car park for bikes) :: in der sackstraße, neben kastner&Öhler (eingang zum fahrrad-abstellplatz) i especially like this one. “this would make a really great blog post…” another great comic from xkcd: “i feel like i’m wasting my life on the internet. let’s walk around the world.” “sounds good.” [panels showing the world’s great beauty, a truly grand adventure] “and yet all i can think of is ‘this will make for a great livejournal entry.’” rocking wirelessly: verizon’s v evdo card after vacillating for a while (and waiting for it to become available), i finally purchased one of the verizon / novatel v express card evdo adapters that everybody’s talking about for my macbook pro. gearlog promised it would be easy — simply install drivers, plug in card — but they were wrong. truth was that i didn’t even have to install the drivers. mac os x asked me if i wanted to “activate” the card when i plugged it in, then automatically went about configuring everything. whitcher sawmill burned i described it to jessamyn in an im last night: lights flickering here, sirened vehicles passing frequently, smell of smoke hangs in air outside the globe reported it this way: warren, n.h. — a sawmill went up in flames during the night in warren (new hampshire). fire officials say they may never know what started the flames at the k.e. whitcher mill around ten o’clock last night. should universities host faculty or student blogs? (part : examples and fear) our cio is asking whether or not plymouth should get involved with blogs. not to be overly academic, but i think we should define our terms. despite all the talk, “blogs” are a content agnostic technology being used to support all manner of online activities. what you’re really asking is instead: what kind of content do we want to put online, and who do we want to let do it? library camp east lce was a success. let me quickly join with the other participants to offer my appreciation to john blyberg and alan grey for all their work planning the event, as well as darien public library director louise berry and the rest of the library for hosting the event. side note: darien is a beautiful town, but we all have to learn to pronounce the name like a local. michael golrick and john blyberg each have a number of photos on flickr, and i’m jealous of those like lichen rancourt who can live-blog events like this. scotchtober fest new hampshire’s highland games are back where they belong in lincoln nh. fittingly for the highlands theme, the weather saturday was cold and misty, with fogs rolling over the hills. i half expected lorna doone herself to appear. the games, of course, are “scottish heavy athletics” involving the throwing (though sometimes carrying) of just about anything that can be found. rocks… hammers… sheep… trees, they all count. well, the “sheep toss” is actually the “sheaf toss” and is intended to measure an athlete’s ability to toss hay to the top of the pile. with all voices now… preaching to the choir, or encouraging them to sing louder? truemajorityaction‘s take it back campaign amuses, but will it motivate the middle? will you join? kid koala’s fender bender while looking up bonobo — who is soon to have a new album out — i discovered not only some videos of his tunes, but also a path leading to videos from other nijna tune artists, including this goodie from kid koala. namiacs mr. pro-life and his wife, kirsten faith pro-life why not? does anybody know a way make a reverse-ordered — think countdown — ordered list without resorting to non-semantic (though ingenious) css tricks? wp ssl one wonders why ssl support isn’t built-in to wp. until then, this noctis.de post offers some tips. it be talk like a pirate day, matey hop to it, dogs. peer an eye at thar video and argue not w’the cap’n: tuesday september th is talk like a pirate day! talk like a pirate day only comes once a year (on september th), this year it falls on a tuesday. if you’re not ready yet, you can learn more about this international holiday on the about tlapd page or practice some phrases from the piratephrases page. our responsibility: teach our children how to talk like a pirate early for future success there’s no question that the video mentioned this morning is valuable resource for all of us, but our responsibility to our nation’s future demands more. the good folks at cook memorial library in tamworth nh are an example to us all with their series of instructional sessions in preparation for talk like a pirate day. microsoft vs. bloggers in accusations of msn spaces censorship i’ve been citing pieces of branding consultant james torio‘s master’s thesis for some time now. but because the thesis is long, and i want to cite a few small pieces, and those pieces aren’t directly url addressable, i’m quoting them here. clickable urls are added, but everything else should be exactly as torio wrote it. (also related: why there’s no escaping the blog and msn spaces isn’t the blogging service for me.) info on geo tags in the wp codex does this mean that geo stuff is built-in to wp? php array to xml i needed a quick, perhaps even sloppy way to output an array as xml. some googling turned up a few tools, including simon willison’s xmlwriter, johnny brochard’s array xml, roger veciana associative array to xml, and gijs van tulder’s array to xml. finally, gijs also pointed me to the xml_serializer pear package. in an example of how even the smallest barriers can turn people away, i completely ignored the two possible solutions at php classes, because navigating and using the site sucks. mysql fulltext tips peter gulutzan, author of sql performance tuning, writes in the full-text stuff that we didn’t put in the manual about the particulars of word boundaries, index structure, boolean searching, exact phrase searching, and stopwords, as well as offering a few articles for further reading (ian gilfillan’s “using fulltext index in mysql”, sergei golubchik’s “mysql fulltext search”, joe stump’s “mysql fulltext searching”). it’s one of a number of articles in the mysql tech resources collection. sysop humor i got tipped to this geeky-funny comic that deserves reposting here for casual friday: always san fran from the west coast comes this tale– a friend of mine is part of maxine hong kingston’s veterans writing group. they are publishing a collection of their work this october “veterans of war, veterans of peace,”, and he was invited to a reading in san francisco. they are a program up there called “drinks with writers” that moves from restuant to restuant once a month. people come, have drinks, writers read, they talk. sweet sumolounge omni a sumolounge beanbag chair is a beanbag like a maserati is a car. but even that doesn’t properly characterize the difference. for starters, it’s big — over five feet on one side. not big enough for the whole wrestling team, but big enough for cuddling. a bit bigger and i’d go looking for sheets and call it a bed, as it’s also comfortable. the website calls it a “crash mat, lounge chair, loveseat or floor pillow,” but whatever you call it, you’ll settle into it like an addictive personality to a bad habit. making plans for library camp east in the list of things i should have done a month ago is an item about making my hotel reservations for library camp east . fortunately, john blyberg notes that alan gray has arranged for a special rate doubletree hotel in norwalk, not far from the site of the event. apple’s itv — from ! the original apple press release is gone (and gone from the wayback machine too), but back in apple announced a different set-top box, also called the itv, for a six-state trial of interactive television services. apple’s itv system incorporates key technologies including a subset of the macos, quickdraw and quicktime. in addition, it includes an mpeg decoder and supports pal and ntsc video formats as well as e and t telephone protocols. the church of september th david moats did some hard thinking on oliver stone‘s world trade center. “[i]t occurred to me that the problem with the movie is that five years later we remain stuck in the moment. we haven’t really moved on.” we’ve not been able to move on from / because we’re still mired in the mistakes that followed from / . many people responded with bravery, including the service men and women who found themselves caught up in one struggle or another. top gun: a requiem for goose teamtigerawesome‘s top gun: a requiem for goose is more than funny, it’s the sort of thing a person should mine for insults and one-liners to use later. of course, the recent tom cruise flap doesn’t dampen it any. from the title cards: on march , president harding established the swingenest, scientologist, dew drop of a flight school in all . now, you boys may think that you are the high-hattenest group of flyboys ever to shoot down a mrs. laura veirs hey folks! good news. the young rapture choir cd is now available from raven marching band records. this album is an amazing collection of songs written by laura veirs, and performed by a choir of school children in cognac, france. it was recorded live in april by tucker martine. the packaging is all handmade and it’s a wonderful recording. this is a lovely, limited edition cd — we only made , — so get one quick at http://www. newertech firewire go pcmcia/cardbus card target disk mode? all my searching seems to confirm my hazy memory that my olf newertech firewire go card does indeed support target disk mode, but the old “hold t while booting” trick doesn’t seem to be working. another shady part of my memory is that the key command was different, but what is it? either google is failing me, or it really isn’t online anywhere. help? mac os x vnc, built-in sure it’s old news, but i am pretty happy that mac os x . has a built-in vnc server. you’ll still need a client, like chicken of the vnc, but it couldn’t be much simpler to make work. though, you could run a separate server app (even several instances of it) and work up a hack like this to allow you to have several people all logged in to the same machine (and getting different screens) simultaneously. crocodile hunter steve irwin dead tv star and crocodile hunter steve irwin is dead after being <a href="http://www.injurywatch.co.uk/news-and-groups/news/marine-incidents/australia-s-crocodile-hunter-steve-irwin-killed-by-a-stingray- ” title="australia’s “crocodile hunter” steve irwin killed by a stingray — injurywatch">stung by a stingray on australia’s great barrier reef. blue marlin spears fisherman from the royal gazette: an angler was almost killed when a giant bill fish leapt from the sea, speared his chest and knocked him off his boat in a freak accident at the weekend. ian card, from somerset, was impaled by the blue marlin and forced overboard during an international sports fishing tournament on saturday morning. his father alan, skipper of the commercial fishing vessel challenger, watched as the struggling creature — estimated to weigh about lb and measuring ft in length — flew through the air and struck the -year-old, who was acting as mate, just below his collarbone with its sword-like bill. remember, he’s really big in germany blame bentley for this. and, as noted in a comment there, “it’s so amazing how [david] hasselhoff has this entire other career that doesn’t exist in the us, except for mocking purposes.” lyrics: beware the pretty faces that you find. a pretty face can hide an evil mind. oh be careful what you say, or you’ll give yourself away, odds are you won’t live to see tomorrow. the competitive advantage of easing upgrades zdnet’s david berlind complains that upgrades are painful: upgrading to new systems is one of the most painful things you can possibly do. if you’re a vendor of desktop/notebook systems, it also represents that point where you can keep or lose a customer. today, most system vendors have pretty much nothing from a technology point of view that “encourages” loyalty. upgrading from an old dell to a new dell is no easier than upgrading to a system from a competing vendor. things i need to incorporate into various projects memcached, a “highly effective caching daemon, …designed to decrease database load in dynamic web applications,” and the related php functions pspell php functions related to aspell and this pspell overview from zend http_build_query, duh? current connected mysql threads * unix load average = system busy; reduce operations when $system_busy > $x missiles are the new ied i’m not going to make this point well, but let me try. now that we’ve recognized the long tail of violence and the “open source insurgency” and seen the hezbollah missile threat, it’s hard not to imagine a growing threat from enemy or terrorist missiles. in short, as technology becomes cheaper, the weapons people can use against us become more complex. iran and north korea have been developing and testing missiles for some time, but the pound gorilla here is russia. flickr to get all geotaggylicious? when dan cat gets cagey, and people are talking about mysterious map buttons in flickr a guy has to wonder…is this why the lines between dan’s hobby and day job are so blurry? update: ryan eby points out that the map is live! lurk, cut, paste and it is cutting and pasting but what other names are there now for it?? for looking at other websites, following the site and lifting off passages and putting them onto your own site– for one reason or another?? i found bookish.dk while looking up info on denmark about a year ago. finally this may, lifelong wish, i finally got to copenhagen for two days. karen b is a scotswoman who has seeger’s springsteen made the mistake of complaining about bruce’s new album. i knew i was risking the age thing, and sure enough– i downloaded finally, with too much anticipation, bruce’s new seeger sessions. i haven’t heard b much lately but his voice sounds like its shot?? seeger did his work with such a rich voice, deep and subtly modulated. this album is beautifully produced, the backup band is greatvoice and nearly too much. stranger than crazy every so often you want to know more about real gypsies. this film is where to start when that time comes round again. romanian gypsies portray themselves in gajo dilo. the crazy stranger in question is not a gypsy but a visiting young frenchman, played by the wonderful romain duris. we just have to go do the work nicholas lemann, in a story on blogging and citizen journalism in the august issue of the new yorker: [n]ew media in their fresh youth [produce] a distinctive, hot-tempered rhetorical style. …transformative in their capabilities…a mass medium with a short lead time — cheap…and easily accessible to people of all classes and political inclinations. and quoting author mark knights: …a medium that facilitated slander, polemic, and satire. it delighted in mocking or even abusive criticism, in part because of the conventions of anonymity. swimming in spam, but customer support comes through i awoke this morning to a bit of a mess. after enjoying months of spam-free bliss thanks to akismet, i found over a hundred spam comments for pills and free pictures to suit most any need or desire. spam has snuck through before, but never in this volume, and akismet has always been quick to learn from my manual corrections and stop further leaks. not this time. so i began to panic. reality television infects print media now that we’ve forgotten how deep the collected sludge on the bottom of our cultural barrel is since fox appears to have given up dredging it for entertainment like who wants to marry a millionaire? and <a href="http://imdb.com/title/tt /” title="“the littlest groom” ( ) (mini)“>the littlest groom, jane magazine (subscribe) has stepped up to explore what remains. the huffington post’s eat the press blog recently reported a story titled “girl, you’ll be a woman soon: the quest to deflower jane‘s -year old virgin” eaten alive books eaten alive books it’s a piece of cake to bake a pretty cake don’t hate me for this, it was mattyb who showed it to me and then setup the domain itsapieceofcaketobakeaprettycake.com. the clip comes from lazytown (<a href="http://www.imdb.com/title/tt /” title="“lazytown” ( )“>imdb), which airs in the us on nick jr. an excerpt of the lyrics: i’ts a piece of cake to bake a pretty cake if the way is ha-zy you gotta do the cooking by the book darwin, schmarwin are we ahead of turkey? yes. sign up now: library camp east library camp east is set for september at darien public library in darien ct. it’s an unconference, so the content is determined by the participants, and judging from the names on the signup page (john blyberg and jessamyn sound excited), there will be a lot of good discussion. catching bugs before they catch you i got itchy about magic quotes the other day because it’s the cause (through a fairly long cascade of errors) of some performance problems and runaways i’ve been seeing lately (pictured above). but i deserve most of the blame for allowing a query like this to run at all: ` select type, data, count(*) as hits from wpopac_wpopac_bibs_atsk <strong>where data like '%'</strong> and type in ('subjkey','author', 'title') group by data order by hits desc limit ` as executed, it’s trying to select all . false i had no words for it now wri- ting i am temp ted to say that i fe lt the wor ld had been giv en as a gi ft uni que ly to me and al so eq ual ly to ea ch per son a lone verse style of robert lax sentence by rory stewart treo as dial up network adapter sometime ago i started work on figuring out how to get dial up networking (dun) access via my treo . now i’m getting serious about mobile internet access and looking at this again. the plan is that you should be able to make a bluetooth connection between your laptop and the phone and then get piped onto the internet from the phone. trevor harmon wrote it up and has been following the issue as it relates to mac os x and sprint wireless service. dang addslashes() and gpc magic quotes somewhere in the wordpress code extra slashes are being added to my query terms. i’ve turned gpc magic quotes off via a php_value magic_quotes_gpc directive in the .htaccess file (we have far too much legacy code that nobody wants to touch to turn it off site-wide). and i know my code is doing one run of addslashes(), but where are the other two sets of slashes coming from? knockbox = wifi + real estate info in another sign of the arrival of the stupendous, i.e. that the internet is changing our world, engadget some time ago reported on the sellsmart knockbox real estate selling dohicky. what is a knockbox? a knockbox is a sleek, self-contained appliance that is placed unobtrusively inside your home for sale. it contains a photographic tour, custom buyer presentation, and other important details about your home, which potential buyers can access without ever having to enter your home. are you with me? this weeks free i-tunes down load is the song ” are you with me?” released by the band vaux. i like the song, it’s a little hardcore for my tastes but i can see my self mosh pitting to this. it’s not my favorite music but being a fairly open minded person i can find a place for it in my musical library. i give this song an . on pies listening pleasure scale. wpopac reloaded i’ve re-thought the contents of the record and summary displays in wpopac. after some experimentation and a lot of listening, it became clear that people needed specific information when looking at a search result or a catalog record. so now, when searching for cantonese slang, for instance, the summary displays show the title, year, format, attribution, and subject keys of each result. and when viewing the record for a dictionary of cantonese slang you’ll get all of that and more. longest book title ever geography made easy : being an abridgement of the american universal geography, containing astronomical geography, discovery and general description of america, general view of the united states, particular accounts of the united states of america, and of all the kingdoms, states and republics in the known world, in regard to their boundaries, extent, rivers, lakes, mountains, productions, population, character, government, trade, manufactures, curiosities, history, &c. : to which is added, an improved chronological table of remarkable events, from the creation to the present time, illustrated with maps of the countries described : calculated particularly for the use and improvement of schools and academies in the united states of america snakes on boards snakes on skateboards would not wear helmets nor would they swing hatchets, but snakes on snowboards might if they had just visited love land’s phallus garden verizon evdo service and the mobile office? the much anticipated novatel v express card evdo adapter is out. verizon is pimping them for $ with year contract and gearlog says it’s “almost too easy” to use these goodies with the macbook pros. then gearlog reader brad commented: “if you had to install a driver, i wouldn’t say it was the true mac experience. i have sprint evdo with a merlin s card. with os x . sweet bluetooth graphire tablet, bad portraits my graphire bluetooth tablet arrived last week as a bundled treat with some adobe software i needed. why do i need a tablet, especially as my days as a graphic designer are a distant memory? i don’t…at least not now. but somewhere on the long tail my um, unique, style of portraiture (above) will come into vogue and i’ll score it big. yup, there’s an unrecognized niche of people just waiting to be drawn with big cheeks, bulging eyes, and open mouths. carry-on restrictions to carry on? the mercury news’ qa on carry-on restrictions answered a big question i had: q can i still carry my laptop, cell phone and ipod on board? a those items are still ok as long as you’re not traveling to or through the united kingdom. but a reuters story posted at c|net suggests the restriction on liquids won’t be going away any time soon. draconian restrictions on carry-on baggage may stay in place for months, even years… all about non-profits i’ve been looking up information on non-profits, specifically c corporations. there’s this sales-pitch filled faq; the company corporation makes it sound easy, but this how to guide from the national mental health association (of all places) seems to offer the…um…most honest info i’ve seen yet. well, most honest sounding. dancing against the current you might argue with kevin lim‘s suggestion that terrorism depends on our emotional and psychological insecurity, but can you really argue with the notion that more happy people is a bad thing? i can’t. and i can’t criticize him for finding deep meaning in catchy pop songs and funny movies. he and brandtson might be right… “nobody dances anymore. everyone’s still playing safe and nobody takes chances anymore.” sxsw program proposals there’s programs proposed for sxsw interactive, march - . go vote for the ones you most want to see at lindsey simon’s super cool picker. round one voting is going now. (also note the really good use of semantic markup in the html download version (which i’m embarrassed to have sullied a bit in this representation).) podcasting – what’s it going to take to mainstream the technology? business / funding / entrepreneurial · web audio / web video over the past twelve months, podcasting has exploded among tech savvy individuals and organizations however, what’s it going to take for podcasting to evolve from its current state as a delivery system for specialized, longtail content to a widely-adopted media distribution system for mainstream users? hard math i found this at joe-ks.com. the title there is “mennonite longhand math,” but can anybody identify the source or context? can anybody work out the equation on the board? i’ve convinced my friend will, who teaches math and physics, to pose for a shot like this, but that means we’ll have find and fill a huge chalkboard…and he’ll have to grow his beard back. lawn mower speed record it’s late summer and the heat wave killed the grass on your lawn, so what better to do than challenge bob cleveland’s record for the fastest lawn mower yet? not sure your mower has what it’ll take to race down the salt flats at over mph? wimp. utah’s ksl tv quotes bob saying “we don’t need a whole lot of horsepower to go fast.” and when you look at the tiny wheels on that thing, well, you’ve gotta imagine you can do better. shakespeare, motivation, war, what are we doing here? i’m a sap. i can’t help but get choked up when i read or hear shakespeare’s st. crispin’s day speech in henry the v. ehow tells me that “saint crispin’s day is a good day to honor lives well lived, beliefs held dear and shoes well made.” but steve denning calls the speech a “magical, linguistic sleight of hand,” and warns us: …it may work for a battle, or even several battles. flight, hotel, spa “take a deep breath.” i did, and with it lisa souza, my massage practitioner at san francisco’s international orange, pressed into a knot just below my shoulder blade, deep in the latissimus dorsi. she worked along the length of it, not as a baker kneads bread, but rather as person wringing water from a damp cloth. each press was deliberate, powerful. i’d asked for the deep tissue treatment. eight hours in planes from boston (six hours to lgb, almost another two to sfo) had taken their toll, and this, i hoped, might spell relief. workflow goes social i was amused this week to see two examples of workflow getting sexy. that’s not how the developers describe their efforts, but the departure from old groupware notions is clear. in daring defiance of zawinski’s proclamation, jeffrey mcmanus, with approver.com, and karen greenwood henke, with nimble net (as reported yesterday), are tackling workflow and approval processes. combine the increasing numbers of people who are self employed or working in very small businesses that can’t afford those old enterprise groupware “solutions” (but who nonetheless have to get a job done) with the combination of luck, pluck and smarts these two seem to have applied to the challenge, and there’s a chance these new products — groupware . sweet coffee shop logo how can a person not like ritual coffee roasters [logo][ ]? the [laughing squid][ ] folks [apparently like the place][ ]. [ ]: ritualroasters-com-huge cup.jpg [ ]: http://laughingsquid.com/ “laughing squid” [ ]: http://laughingsquid.com/ / / /wordcamp-is-this-saturday/ dr. frankenstein’s stress-o-meter the scientologists regularly have a table on powell st., somewhere near union square. the game here, if it’s not obvious, is to invite people to take a free stress test, then sit them down and twiddle those unlabeled dials until the needle starts twitching. the blood red table cloth is sure to help. a technology for every niche way too many people are processing grant applications on paper. they spend a lot of time moving paper around and they don’t know much about who’s applying until after the deadline. that’s why we built nimble net. karen greenwood henke’s been working the world of grants and grantwriting for years. her site grantwrangler.com, and the new grant wrangler blog represent her efforts to connect grantors with grantees, but nimble net delivers the tools necessary to manage the process from announcement to award, and all the application and review processes in between. wordcamp kickoff woot! wordcamp kickoff party at taylor’s automatic refresher (no doubt selected in part because homophone to automattic), at the ferry building. but does it make up for missing wikimania, the librarything bar-b-que-thing, and napoleon dynamite night at the twig? go air scooter, go while we’re still waiting for flying cars (or even just fuel efficient cars) i’m keeping track of tiny helicopters like the gen h- and this one, the airscooter ii, pictured above. the company, airscooter corporation of henderson nv, introduces the new craft with a tip of the hat to igor sikorsky‘s earliest designs featuring counter-rotating blades. company founder woody norris (who won an award for acoustics) explains: “what we’ve done is package the coaxial design in a modern light-weight craft that allows for intuitive control and incredible maneuverability. the onion greets wikimania wikimania is about to start, but here, the ever-topical onion folk are poking fun at wikipedia. what is there to say when “america’s finest news source” casts aspersions on the world’s newest encyclopedia with the headline wikipedia celebrates years of american independence? extra: watch out for meredith farkas‘ panel presentation on wikis and enabling library knowledgebases. i should have thought of this in the context of ryan eby’s question about librarians going to non-library conferences. joe’s favorite novels will pressed joe, asking him to name his top ten favorite books. joe pressed back, saying such lists were ridiculous, but still, sometime later he emailed with the following: okay, here are the books that got to me at certain points in my life. not sure i would view them all the same now, but this is a list of sorts. i found this an interesting challenge, and of course impossible…i have more lists but i stuck to novels… opensearch progress i really need to keep better tabs on michael fagan, as his june opensearch update is full of goodies. the perils of flickr’s “may offend” button quite a while ago now, stepinrazor asked people to do some self-censorhip in a post in the flickr ideas forum. flybuttafly quickly joined the discussion, noting that she’d encountered some material she found offensive in pictures from other flickr members: “as i’m going through the pictures, one shows up of a protestor holding a sign with a vulgar statement on it.” though she refused to identify what she saw that was offensive, she did note in a later post that she “would never take my child to a pro-abortion rally. and now this is happening? when a gossip site has a picture of mel gibson that looks more like ted kaczynski, and a story about drunken, anti-semitic ravings, i think “eh.” but somehow i get more interested agitated when i learn the cops might have sanitized the police report of the whole affair. update: ooh, what about his endorsements? dooce and blogher bob, the occasional cultural affairs correspondent here, took me to task: how could you not? no link to dooce.com?? nor to blogher.org??? what can i say? my immediate reaction was that he’d found proof of danah boyd‘s point that male bloggers only link to male bloggers. anyway. the blogher conference just wrapped up, but as ryan notes, i don’t know of any library folk who attended. still, marianne richmond is on-blog, raising our awareness of dopa just like a lot of librarians are trying to do. wal-mart trying to ape myspace, seriously i just got a heads up on an advertising age story that wal-mart is trying to be myspace (and, yeah, i aped their headline, too). here’s the lead: it’s a quasi-social-networking site for teens designed to allow them to “express their individuality,” yet it screens all content, tells parents their kids have joined and forbids users to e-mail one another. oh, and it calls users “hubsters” — a twist on hipsters that proves just how painfully uncool it is to try to be cool. stage two truth arthur schopenhauer is suggested to have said: every truth passes through three stages before it is recognized. in the first it is ridiculed, in the second it is violently opposed, in the third is regarded as self-evident. if the reaction to karen calhoun‘s report to the library of congress on the changing nature of the catalog and its integration with other discovery tools is any guide, libraries are stuck firmly in the second stage. richard cheese’s lounge against the machine richard cheese‘s lounge-core renditions of pop favorites (and some not-so-favorites) have been cracking me up every time they chime into the mix on random, but i didn’t know what the guy looked like until i spied beatnikside‘s photo of the man in among his vegas people set. “cheese,” of course, is a pseudonym for la comedian mark jonathan davis, who’s been performing with a band of cheese-named musicians since . two events, two coasts matt mullenweg announced wordcamp in san francisco, then ten days later abby announced the librarything cookout in portland (maine). both are set for august . the librarything event promises free burgers and potato salad, while wordcamp attendees will enjoy both free bbq and free t-shirts. i’d like to go to both, but rather than have to make some decision about which one i’d most like to go to, i’m leaning on the fact that i’d already bought my flight to sfo when the lt event was announced. be romantic and smoke his brains out this photo from tsunaminotes appeared in ende’s photo stream and reminded me instantly of all the cool things i’d never done because i was born too late and cool stuff is what i saw in black and white photos from years past. of course, flickr says the photo was taken july th, and the photographers of the past would have burned the bright spot on his cheek during printing, but it still has a classic quality to it. pretty little thing fink‘s pretty little thing is this week’s free download at itunes, and i have to say i like it. pretty little thing is not usually what i would listen to but i found the song to be new and interesting, very “fresh”! fink‘s pretty little thing gets a . on pies listening pleasure scale. . tags, folksonomies, and whose library is it anyway? i was honored to join the conversation yesterday for the latest talis library . gang podcast, this one on folksonomies and tags. the mp is already posted and, as usual, it makes me wonder if i really sound like that. still, listen to the other participants, they had some great things to say and made it a smart discussion. i approached the conversation with the notion that what we were really talking about was whether libraries should give their patrons the opportunity to organize the resources they value in ways that make sense to them. wordcamp as noted here, i’m going to wordcamp in sfo in early august. matt describes it as a barcamp-style event (where “’barcamp-style’ is a code phrase for ‘last minute’”) with “a full day of both user and developer discussion.” i’m just going for the free t-shirt, of course, but i can imagine a number of folks will get a good value out of the sessions and discussions that will likely run, especially all the developer stuff. …it’s how you use it not a pretty librarian has kicked things off well with a first post titled “it is not a tool,” covering an argument about which has more value to a teenager: a car or a computer. on one side is the notion that “she can’t drive herself to work with a computer.” while, on the other side is the growing likelihood that she won’t drive to work at all, but instead simply work at whatever computer she has available. bsuite bug fixes (release b v ) [innerindex]update: bugfix release b v available. it’s been a while since i released a new version of bsuite, my multi-purpose wordpress plugin. i’d been hoping to finish up a series of new features, but those have been delayed and this is mostly just a collection of bugfixes. this update is recommended for all bsuite users. bsuite features tracks page loads (hits) tracks search terms used by visitors ariving at your site via search engines it’s official wpopac, a project i started on my nights and weekends, is now officially one of my day-job projects too. we’ve been using our wpopac-based catalog as a prototype since february , but the change not only allocates a portion of my work time specifically to the development of the project, but also reflects the library‘s decision to transition to wpopac as a our primary web opac. work to make a general release of the wpopac software available for download and use by any library (or anybody who wants to present structured data with faceted searching on the web) is in progress. . the music has been on random for weeks now, but . ‘s “joanne will,” from plays music played this afternoon as soundtrack to the summer rains. brent sirota may struggle to tell us how bad it is (while also giving it a . rating), but this “easier to listen to jazzy than to listen to jazz” turned out to be the perfect accompaniment for the ballet of raindrops and splashes just out of reach from my seat on the porch. beermapping.com in yet more geolocation news, beermapping.com‘s maps to breweries will make my travel planning easier, and my travels boozier. hey, it’s casual friday, take off early and go find a new brewpub for lunch. plazes updated wearing the badge “still beta,” plazes, the free, network-based geolocation service, now sports a new coat of paint. among the improvements is the flash-based badge (above) and a much improved frontpage/dashboard that combines the map of known locations with the map of active users, formerly two separate screens. on the downside, i sort of miss the old tracker. i love the icons on the new one, but there was a simplicity to the old list of recent plazes and favorite plazes that i liked. the flickr is a series of tubes it’s hard to be angry with flickr about unexpected downtime when they post funny things like this. for my part, this is more than just an excuse to link to dj ted stevens’ internet song (yeah, “the internet is a series of tubes”), it’s an excuse to point out how flickr apparently knows how to speak to their customers in language they we understand. i dare a library to do the same next time the opportunity permits. opensearch in a nutshell opensearch is a standard way of querying a database for content and returning the results. the official docs note simply: “any website that has a search feature can make their results available in opensearch format,” then adds: “publishing your search results in opensearch™ format will draw more people to your content, by exposing it to a much wider audience through aggregators such as a .com.” it’s a lot easier to understand opensearch once you’ve used it, so take a look at a . arctic monkeys while listening to my favorite radio station . fnx, i discovered my new favorite band. the arctic monkeys is a new band that comes from the uk, and their popularity is rocketing. their new album whatever people say i am, that’s what i’m not, has sold more than , copies which makes it the fastest selling debut album in uk history. having heard them months ago i was pleasantly surprised to see the arctic monkeys perform there hit single “i bet you look good on the dance floor” on mtv. nelinet it conference proposal i recently submitted my proposal for the nelinet information technology conference. it’s about wpopac, of course, but the excitement now is that the presentation would be the story of the first library outside psu to implement it. wpopac is an open source replacement for a library’s online catalog that improves the usability, findability, and remixability of the library’s collection. this presentation will detail the implementation of wpopac in the real world, including discussion of challenges and costs, as well as the improvements to service and increased access to library materials. less than a year left before paris’ retirement yup, tom reminded me recently that there’s less than a year left on the official paris hilton retirement countdown. in case you’ve forgotten, the hamburger-eating heiress announced her retirement in a june issue of newsweek (jump to page two for the relevant bits). don’t get tripped up on the postdated retirement announcement, bill gates announced his intentions to retire in last month, so one might say it’s all the rage. technology scouts at aall i’m honored to join katie bauer, of yale university library, in a program coordinated by mary jane kelsey, of yale law’s lillian goldman library. the full title of our program is technology scouts: how to keep your library and ils current in the it world (h- , pm tuesday, room ). my portion of the presentation will focus on how we’re fixing up our catalogs, with a big emphasis on how apis can be used to continuously reinvent the way we look at — and thus understand and use — the information we have. the social software over there amusing. one one side of the world is jenny levine, the original library rss bigot, pushing libraries to adopt new technologies from the bottom up, and here on the other side of the world is newsgator offering their products for top-down adoption. why are law libraries interested in newsgator? could it be that social software increases productivity? might it offer some competitive advantage? do they just make it easier to communicate (and keep track of our communications) in today’s web-driven world? inclusion or exclusion by language …the time for pedantic purism is past; if we wish to communicate with the larger audience, we must use language they understand. we do not have the luxury of defining our words, their definitions are thrust upon us by usage. i was struck by how much that sounds like something i might have said about libraries — only more compact and pointed — but it’s actually my father describing his position on an argument at the world history association annual conference a couple weeks ago. education america today i discovered (thank you ryan) kareem elnahal’s speech as valedictorian of mainland regional high school and i discovered new hope, new faith in our country’s future. when high school students can step up and speak truth to power, as elnahal did so well, i become a believer in the strength of human spirit. “we study what is, never why, never what should be. …[t]his pattern, grade for the sake of a grade, work for the sake of work, can be found everywhere,” said elnahal. rocket cars make better fireworks i pointed out this jet turbine powered toyota mr a year ago, but now i’ve discovered ron patrick’s jet powered vw beatle. the story is well told in a san francisco gate article from april (with bonus video), which describes the builder: patrick is a -year-old stanford-trained (ph.d.) engineer who owns ecm (engine control and monitoring), a sunnyvale firm that makes electronic instruments used by auto manufacturers to calibrate their engines for performance, fuel economy and emissions. antstepology french vexillographers circulate the national library, protesting flag desecration, too many windows, and cardboard sunscreens. fireworks on the fourth of july promise. celebrate independence day with a drink. tags: banana, bananas, blueberries, blueberry, flag, fourth of july, fruit, independence day, july , july th, patriotic, patriotism, raspberries, raspberry, red white and blue, stars celebrate independence day with breakfast let the vexillographers cringe, flag desecration never tasted so good. sure, it’s barbecue season, but that’s no reason not to enjoy breakfast. and what better way to break fast on the fourth of july than to dress waffles as sugary, fruity flags? do that with your hamburgers. do that with your potato salad. do that with your hot dogs. (okay, i can imagine a few ways to do that with all of those, so let’s see the pictures. today’s terms tags: , concert, music, political cr… today’s terms tags: , concert, music, political criticism, politics, show, the sun, they might be giants, tmbg echo through pine walls stretch sun along desire’s coast they might be giants they might be giants, playing at mohegan sun, drew roars of approval from the crowd when john flansburgh went off-lyric sheet during the sun (which they amusingly described as part of their venue songs series): …the heat and light of the sun are caused by nuclear reactions between a failed foreign policy, a failed domestic policy, and a failed presidency… i’ve not known tmbg to be at all political, just smart. saturday, july , : pm is there a term already for what i am about to do? ok, here goes: bad knockoffs of cheap pop oops! i did it again richard thompson strange sense of humor last.fm, despite the suggestion here stream it from npr, go buying. ****kcrwmusic toxi the chapin sisters top tune britney spears episode coverville. **** britney in wax at madame tussaud’s pretending to do hard math some fan with a brit photo on his refrigerator oops! i covered it again i don’t know why it is that i love bad knockoffs of cheap pop, but i do. that’s why, when i heard a folksy rendition of oops! i did it again playing between segments on some npr program a while ago, i had to go looking for it. as it turns out, it was richard thompson, whose strange sense of humor apparently pops up in his music regularly. you can find his version indexed in last. june : tony day in the two years joe spent researching and writing two ton: one night, one fight — tony galento v. joe louis i’ve heard a lot about this guy. tony galento was a most improbable opponent for louis, who by then had regained the world heavyweight title from max schmeling, but joe’s description tells it best: beetle-browed, nearly bald, a head that rode his collarbones like a bowling ball returning on rails, his waist size more than half his five-foot-eight height, two ton tony galento appeared nearly square, his legs two broomsticks jammed into a vertical hay bale. burning patriotism my feelings on the flag burning desecration amendment should have been clear from my flag day story. still, let me offer the t-shirts above as confirmation. sealand burning a comment from troublepup alerted me that the principality of sealand burned friday. the evening star explained: witnesses watched in amazement as a huge plume of smoke started to rise from one of the legs of sealand — and boats raced to the scene. seafront worker bruce harrison said: “it was quite spectacular. the amount of smoke was huge and people kept saying there must have been an explosion. american diplomacy i don’t collect stamps, but this set caught my eye. first there’s the irony that the usps is celebrating american diplomacy at a time when, well, there’s not much to celebrate. then i get a further chuckle when i notice the postal service can only scrounge up six examples to celebrate, but found “superlatives” to get excited about in their wonders of america collection. of course, the superlatives are relative — the bison is only the largest land mammal in the us, for example — but i don’t know enough to judge the six diplomats. the twig’s grand opening wendy sent out this invite last week: last month the monningers quite suddenly became restauranteurs. six weeks later, wendy, joe and pie are excited to announce the grand opening of “the twig”– an ever-so-cute restaurant in their hometown of warren, nh. on saturday, june th from - come to the twig for free pizza and cake. win gift certificates and enjoy the newly-opened “brook-side at the twig,” a beautiful outdoor beer garden along the bank of black brook. context, language, systems “bagged products” is little better than “cookery.” i’m gonna bet that no customer has ever asked the sales people for “bagged products,” that nobody’s ever checked the yellow pages for “bagged products,” and without context, nobody would come close to answering a question on what the heck “bagged products” are all about. but we do have context. and within that context, those two words are probably meaningful enough to the potential customers driving by. free markets, bad products, slow change rates point a: john blyberg’s ils customer bill-of-rights. point b: dan chudnov’s the problem with the “ils bill of rights” response: john blyberg’s opacs in the frying pan, vendors in the fire while there’s some disagreement between john and dan, i can’t help but see a strong concordance between their posts: both are an attempt to educate potential customers. blyberg wants customers to know what to ask/look for in evaluating products, dchud wants those customers to know how free markets work. scooter by sunset the light sunday evening was golden, so i stopped to take way too many photos of the meadow in the sunset. just before filling my memory card with all that, i got back to my scooter to find this scene with a haze settling on the field and the sun just ducking behind enough of a cloud to make the exposure work. well, okay, it was still a double exposure to get the light right across everything, but still… spark fun’s gps data logger engadget alerted me to this gps data logger from spark fun electronics. the device records up to hours of data to a mb sd card in either a simple text file or kml-compatible format that you can display in google earth. i like it, i want one (actually, i want three, and i’ll eventually post about why), but the ad copy tweaked me a bit: pull the sd card, insert it into a card reader, […] and wammo–you can see what casey did over lunch with a satellite image overlay. the pope vs. the da vinci code the above image and following text are circulating the web, tickling funny bones. this man (on the left wearing a fabulous vintage chiffon-lined dior gold lamé gown over a silk vera wang empire waist tulle cocktail dress, accessorized with a three-foot beaded peaked house of whoville hat, and the ruby slippers judy garland wore in the wizard of oz) is worried that the da vinci code might make the roman catholic church look foolish. from the memepool memepool has more than earned its place in my aggregator. where else would i learn of the monkey chow diaries (and blog), or the plot structure of fight club in legos, or this flying dude? happy bloomsday thanks to an aside in a sad/angering story at copyfight, i’m now up on bloomsday. here it is, as explained by wikipedia: bloomsday is observed annually on june to celebrate the life of irish writer james joyce and commemorate the events in his novel ulysses, all of which took place on the same day in dublin in . the day is also a secular holiday in ireland. the name derives from leopold bloom, the protagonist in ulysses, and june was the date of joyce’s first outing with his wife-to-be, nora barnacle, when they walked to the dublin village of ringsend. google geo news this post started with ryan sending me this link demonstrating a kml overlay of county borders of his bifurcated state in google maps. then i found this roundup of google’s geo developer day (btw, i so wanted to be at where . ) with tales of the new geocoding feature of the google maps api, more details about kml-in-google-maps, geotagging in picasa, and the new google earth . beta. and somewhere along the line, i ran across a link to sketchup, google’s -d modeler that seems built especially to put dimensional structures in google earth. donald norman — everyday things i was especially young and impressionable when i discovered don norman‘s the design of everyday things, but i still claim it’s required reading for anybody who’s read more than one post here at maisonbisson. that’s self selection at work, but let me put it this way: unless you’re the only consumer of the things you create, then you need to read this. now. i feel foolish to have only recently discovered norman’s website and essays. the ala/no events i’d like to see i’m not going to ala/no so i’m hoping those who are will blog it. two events i’m especially interested in: on sunday, june : catalog transformed: from traditional to emerging models of use this program, co-sponsored by the mars user access to services committee and rusa’s reference services section (rss, formerly mouss), deals with changes in library catalogs in response to the increasing googlization of electronic resources. speakers include: cindy levine (reference librarian for the humanities, north carolina state university), jill newby (english language literature and writing librarian, university of arizona), andrew k. the biblioblogger vs. the branch library steve lawson‘s a biblioblogger visits the local branch library is worth a look and quite a hoot. squashing criticism vs. improving products i wrote yesterday of nicole engard’s comment that the ils was about as open and flexible as a brick wall. today i learned that the vendor of that ils had tried to squash her public criticism. not cool. it’s pure speculation on my part, but what comes next? surely no vendor would send vinny over to bust an uppity biblioblogger’s knee-caps, but might they offer a customer a better deal if they could just help quiet down a critic within the customer’s organization? seven deadly sins seven deadly sins, the some people think seven is too many, others think it’s not enough dopa, social software, and libraries i’m more than a month late to this bandwagon, but whatever. jessamyn alerted me to dopa, the proposed deleting online predators act. what’s the point? when conservatives pit fud against free speech, reasonable people would do well to pay attention. and what’s social software? take a look at what meredith farkas has to say about it. the ils brick wall <img src="static-flickr-com- _f e b .jpg” width=" ” height=" ” alt="the great wall of “standards”” /> nicole engard last month posted about the state of our ils, describing the systems as: i’d say it’s a like the crazy cousin you have to deal with because he’s family! it doesn’t fit, we are a very open it environment, we have applications all over that need to talk to each other nicely and the [ils] is a brick wall preventing us from getting the information we need and sending the information we’d like. darn dns so, you should expect problems when you move your server to a new ip and don’t bother to update the internic registration for your nameservers. it’s an area where i don’t have much experience, so i had to go looking for the solution. paul woutrs gave some tips to get started in his short document on the subject. but the real lesson there was that i had to go back to the registrar where i’d originally registered the nameserver objects to change the registration. did adam and eve have navels? did adam and eve have navels? : discourses on reflexology, numerology, urine therapy, and other dubious subjects filed under “science — miscellanea“ ugh. “save npr and pbs (again)” my dad just forwarded the following message to me: hi, everyone expected house republicans to give up efforts to kill npr and pbs after a massive public outcry stopped them last year. but they’ve just voted to eliminate funding for npr and pbs—unbelievably, starting with programs like “sesame street.” public broadcasting would lose nearly a quarter of its federal funding this year. even worse, all funding would be eliminated in two years–threatening one of the last remaining sources of watchdog journalism. t unboxed and online my sun t is here, and with cliff‘s help it’s now patched, configured, and online. (aside: what’s a sun happy meal?) i’ll second jon‘s assessment that sun really should put some reasonable cable adapters in the box, as the the bundle of adapters necessary to make a null modem connection to the box is ridiculously out of scale (i’ll get a picture soon). i’m getting the application environment put together, which has turned out easier than expected thanks to the convenient packages from blastwave. ego soars because sometimes i feel i’m just moving my lips to the sound of babble, it’s a great delight to find a blog post that suggests i said something coherent. extra: my wife just pointed out this one with photo. nina katchadourian’s sorted books it seems common among contemporary artists that a web search might turn up a few pictures of their works, but not much about them or their works. in this case it’s nina katchadourian and the work i’m interested in is her sorted books project. a video interview from the university of colorado and researchchannel.org does offer some insight into katchadourian’s art, but why are such glimpses so rare? anyway, i was happy to find her compact, graphic poetry. thenonist how can i not appreciate thenonist‘s link dumps and other posts when they’re illustrated with works like those above? the men in suits come from may . june offers us these funny trading cards and a gallery of horror movie damsels (in distress, of course). june offers a good look at sincerity among other things. and all of this amidst a context of intelligent commentary and smart politics. i want url addressable spreadsheet cells (and cell-ranges) when i heard news that google was to release a spreadsheet companion to their freshly bought writely web-based word processing app, i got excited about all the things they could do to make it more than just a copy of numsum. let’s face it, google’s the gorilla in the room here and they’re gonna squash numsum, but wouldn’t it be cool if… well, dmitry nekrasovski get’s credit for planting the notion of url-addressable rows, columns, and cells in my mind with this commentary from months ago: solaris + amp, asap a solaris sysadmin i’m not. but now that i’ve finally got the sun t server i begged for a while back, i’ve got to ramp it up right quick. the first task is to get a, um, lamp environment up and running (samp?…oh, sun wants us to call it amps). a bit of googling turned up this forum thread that suggested blastwave.org‘s ports of php, mysql, and apache. edit: i corrected the model number. circle of gorillas thenonist brings the story of buddy/gargantua the great back with better pictures in a post subtitled “buddy, the gorilla who was scared of lightning” the urls from my portland talk following edward tufte’s advice, i’ve been wanting to offer a presentation without slides for a long time now; i finally got my chance in portland. the downside is that now i don’t have anything to offer as a takeaway memory aid for my talk. my speaking notes are too abstract to offer for public consumption, but below are the urls from them along with a tiny bit of context. foundation prime as it turns out, + , , , is not just the largest bit signed integer you’ll find most anyplace, it’s also a prime number. asian scooter gangs the members of this taiwanese scooter gang might really be cooler than me. well, they would be cooler if the scooter gangs weren’t also known to be violent: a scooter gang viciously attacked and injured teenagers — three critically — while on a violent joyride in taipei county’s tucheng city… the gang of more than scooter-riding thugs, who brandished large knives and baseball bats, went after most of their hapless victims as they were barbecuing for the mid-autumn festival. car lust i told vincent that i didn’t really care much for cars. it was my sister, i explained, that wanted to look. vincent agreed quickly and said it was rock climbing that excited him most. cars, it turned out, were just a family thing he had to play along with. still, he told me about the lotus‘ under pound dead weight, noted the tiny engine that gets nearly miles a gallon yet delivers to in better than five seconds, then opened the door and suggested i shoehorn myself inside. will google eat itself? once upon a time microsoft was the gorilla to beat. once upon a time we thought google could do it. perhaps not any more. amazon has dropped google’s search results from their a search aggregator in favor of microsoft’s live search, and while yahoo!’s on again, off again partnership talks with microsoft appear dead after y!’s announcement thursday of a partnership with ebay, microsoft still hasn’t given up on the notion. sweet portland central library in portland wasn’t open when i returned the next morning to get some snapshots, but you’ll have to take my word that they did a great job renovating it ten years ago. the outside preserves the original appearance of this historic building, and the early hour of the shot hides the hive of activity that i found the previous afternoon. i have to thank caleb and caroline for showing around town, and offer my apologies to heidi and alice, who had offered me tips and suggestions that i (again) didn’t have time to follow up on. denver sights there’s plenty of public art in denver, including a blue bear and this horse in a red chair (here and here, respectively). tourists can also sneak a peak inside the unsinkable molly brown’s house on pennsylvania st. what i didn’t get to explore, however, includes tesla’s time in colorado springs, the forney transportation museum, norad, the remains of the jewish consumptives’ relief society (apparently still findable behind a mall somewhere), and gary sweeney’s “america: why i love her” map at the airport. denver nights el chapultepec is a little jazz club on market st in lodo. the walnut room just north of everything offers live music and a sweet mile high club pizza made “kitchen sink style.” those seeking quieter times can smoke a cigar at the churchill bar at the brown palace on tremont pl. and, outstanding sunset views can be had from the peaks lounge at the hyatt on california st. presentation: designing an opac for web . iug presentation: designing an opac for web . (also available as a pdf with space for notes) web . and other “ . ” monikers have become loaded terms recently. but as we look back at the world wide web of , there can be little doubt that today’s web is better and more useful. indeed, that seems to be the conclusion millions of americans are making, as current estimates show over million users in the us, including % of youth - . and we’re discarding this? i read enough of this to get a good laugh, but not enough to understand if it was serious or not. some of it reads like satire, but other parts as are dry as, well, they’re dry (who really needs a simile anyway, they’re just dry, okay?). scooter my new scooter. it’s not much of a picture, but we’ve had two weeks of rain and this is what i could get. whiskey blanket i just bought whiskey blanket‘s it’s warmer down here ( ) on the basis of a few tracks they offered on myspace. it’s hip hop, socially critical hip hop (crit hop?), set atop a well constructed downtempo trip hop music bed (yeah, i’ll cut it with the hops already). it immediately brought to mind mc ft. jesus‘s the city sleeps and other tracks, but with better, sharper raps and without the mc’s somewhat whiny voice. flickr goes gamma just when we started wondering how much longer flickr would be beta, they announced gamma. the new design had me scratching my head for a bit, but i’m coming to like the changes. the menu/toolbar in the header has direct links to a lot more stuff, while the stuff in the footer has many fewer links. i can’t really tell if there are any links missing there, or if they’re just organized better, as i really only used one or two of them anyway. better business bureau pulls one out i gave up on hostgator a while ago, and i thought i’d cancelled my account until i noticed they were still charging me monthly (yeah, i should pay more attention to what’s on my cc bill). when i contacted them about it they claimed i never fully cancelled. here’s a copy of the form i submitted: hgsales #gsw-[[private]] october , : : pm edt subject: cancellation department: hostgator sales request details: your email: : [[private]] domain name: : maisonbisson. linkability fertilizes online communities it’s hard to know how fuzzyfruit found the wpopac catalog page for a baby sister for frances (though it is ranked fifth in a google search for the title), but what matters is that she did find it, and she was able to link to it by simply copying the url from her browser’s location bar. the link appears among her comments in the discussion about her post on an early letter she’d written to her mom. stonehill industrial history center (aka the shovel museum) most travel guides simply call it the “shovel museum,” but it’s really the stonehill industrial history center. much more than shovels, curator greg galer tells us the collection reveals interesting facts about what we were building and how we built it over the past years. located on the campus of stonehill college in easton massachusetts, the collection does boast shovels from the ames manufacturing companies. from the faq: blogging from basements my buddy cliff emailed me excited about the following quote he found on the yahoo finance message boards: sun vs dell all you need to know about dell & sun was predicted months ago by some blogger in his parent’s basement. the draft ads are cool: http://spiralbound.net/ / / /sun-talks-some-smack/ how come the big brokerage house analysts can’t figure this stuff out? cliff doesn’t really blog from his parent’s basement, but well, he was happy for the link love. pretty soon everybody will have it this isn’t as funny as it used to be. every time i read about or hear of somebody talking about autism, i recognize some many of the behaviors as my own. first it was this rather amusing comparison between “eccentric” and autistic behaviors, then it was an interview on fresh air, and just this weekend i heard kamran nazeer talking about his new book that profiles himself and four other autistic adults. amazon’s simple storage service ryan eby got me excited about s a while ago when he pointed out this post on the amazon web services blog and started talking up the notion of building library-style digital repositories. i’m interested in the notion that storage is being offered as a commodity service, where it used to be closely connected to servers and bought (and wasted) in chunks. with s , you can build a simple application that runs anywhere, store your big data in s , pay for what you use, and expand (or contract) as you need to. reputation management at applied dreams . ryan gave me the drop on this presentation by dave chiu and didier hilhorst where they do an amusingly effective job of explaining the concept of reputation management. it all went down at the conclusion of the applied dreams . project at interaction design institute ivrea in milano. the project brief begins: our identities are changing due to our constant exposure to enabling technologies. our old physical identities, fixed to a house, an address, a tax number, private, detached, individual, introvert, seem increasingly at odds with our new electronic identities, mobile, self-published, publicly exposed, extrovert, shared, accessible, communal. betty bowers first i found her harry potter review, then i found the god told me to hate you buttons and other stuff. who makes these decisions anyway? brian’s comment at remainingrelevant should resonate with many of us: something to consider about why libraries end up with bad interfaces (at least as far as catalogs go) is that it might be that the people who use the interface (and help the public use it) are not the people who decide which interface to use. when it comes to demanding better from vendors […] consortiums like mine seem to place more emphasis on “cheap and reliable” than in “useful to the patrons. george bush and cognitive dissonance: “evolution is a lie” and “bird flu will evolve to threaten humans” alpha liberal reminds me that bush somehow gets his head around the following: “the jury is still out on evolution” and “the bird flu virus could evolve to a form that can be spread easily from human to human” eh, i’ll take any excuse to point to michelle leeds’ photo and bash bush’s stupidity. used brains and black plague, on ebay he he. chuckle, chuckle. thanks to kris and brett for these pics. they ads are still there now when i search google for used brain or black plague. my question is: does ebay just submit bulk lists of terms they want to buy, or do they have a deal with google to just link ’em up like this? authority and base jumping authority has varied meanings in every context. this piece on ifilm has iiro seppanen explaining his view of the matter as it relates to jumping off the stratosphere in las vegas. view above, or click through to base concepts: authority. i don't need an excuse to drink tequila, but i'll eagerly take one ian chadwick’s in search of the blue agave begins: “tequila is mexico,” said carmelita roman, widow of the late tequila producer jesus lopez roman in an interview after her husband’s murder. “it’s the only product that identifies us as a culture.” no other drink is surrounded by as many stories, myths, legends and lore as tequila and its companion, mezcal. they transcend simple definition by reaching into the heart of mexico, past and present. q: why do some things suck? a: because we compare them to the wrong things. i’m in training today for a piece of software used in libraries. it’s the second of three days of training and things aren’t going well. some stuff doesn’t work, some things don’t work the first (second, third…ninth) time, and other things just don’t make sense. at lunch, one of the other participants mentioned to the trainer that some of the activities in the software seemed to have too many steps, too many places to go wrong, too many turns between beginning and end. wpopac gets googled a discussion on web lib last month raised the issue of google indexing our library catalogs. my answer spoke of the huge number of searches being done in search engines every day and the way that people increasingly expect that anything worth finding can be found in google. there were doubts about the effectiveness of such plans, and concerns about how frustrating it might be for a searcher in california to find books (that he or she can’t access) in new hampshire. higher ed blog con (and other things i should have posted about last month) i meant to post about this weeks ago, but highered blogcon has now come and gone. it had sections on teaching, libraries, crm, and web development. (aside: why must we call it “admissions, alumni relations, and communications & marketing” instead of the easier to swallow “crm”?) the “events” are over, but everything is online, and most of it is free. ryan did a good job of covering the first few days, and what would a blog conference be without a common tag? linkrot? we don’t have any steenking linkrot! allen asked, via the web lib list: i’m interested in how others handle linkrot in library blogs. do you fix broken links? remove them if they can’t be fixed? do nothing? michael answered: i deal with link rot on blogs as i would with any other publication, print or otherwise: do nothing. the post is dated and users should be aware that links from two years ago may no longer work. frank rich on bush’s last days frank rich’s new york times op-ed column today was full of the kind of easy one-liners that repressives conservatives usually like to use against honest people progressives. i got it from my friend joe, but because the new york times thinks their content is golden, they won’t let me link you to the full-text. eh, i looked it up in lexisnexis (also a paid service, but better (marginally)) and posted the good parts here: kobb labs joe forwarded me a link to kobb labs the other day, and i’ve got to admit that the guy has a much better introduction than anything i could have written for my site: despite what you may have been told, i am not a mad scientist. (no, no, no, that’s all slander and lies from jealous colleagues.) as you can probably tell from my website, i’m just a man curious about the universe and the order of things. moba revisited i had a good opportunity to revisit the museum of bad art in dedham mass earlier this week. above is my buddy corey, but i was amused to find that visitors appear to be leaving their own works for the collection. cupcakes? “i’ve never seen the inside of a rabbit’s brain before. what’s in there anyway?” “nobody knows yet. johnson and i are hoping it’s cupcakes.” “me too. except vegan cupcakes. because i’m a vegan. vegans don’t eat animals or animal prod–” “i know what vegan means, thomas. you’ve told us.” “well, i was just saying, because–” “i know what vegan means” thank you, tristan. twenty years and a day mark nelson’s pripyat series on flickr is full of the pictures of desolation that people seem to be looking for as we solemnly honor the twentieth anniversary of the chernobyl disaster. google added high-resolution satellite photos of the area yesterday, and pripyat.com offers both stories and photo galleries to help us remember. it is there that i learned that rimma kiselica, the woman who has guided so many of those who’ve reported from the dead-zone, died on march . chernobyl and pripyat satellite photos today, on the twentieth anniversary of the disaster, google has added high-resolution satellite photos of the chernobyl nuclear power plant and the abandoned town of pripyat. above is the plant; the damaged reactor is on the left. in pripyat, the ghostly ferris wheel was easy to find, but where’s the vehicle graveyard? update: here it is. hat tip to “di” and “pero ” for their comments. twenty years ago today twenty years ago today at : : , the chernobyl npp reactor number four exploded. five thousand tons of lead, sand, and other materials were dropped on the resulting fire in an attempt to stop the spread of the radioactive cloud. the world learned of the accident when western european nuclear facilities identified radiation anomalies and traced them to the chernobyl plant, forcing the ussr to make its first public announcement on the matter. boolean searching in wpopac wpopac takes advantage of mysql’s indexing and relevance-ranked searching (go ahead, try it), including boolean searching (on mysql versions > .x). here are some details and examples taken wholesale from the mysql manual: + a leading plus sign indicates that this word must be present in each result returned. – a leading minus sign indicates that this word must not be present in any of the resuls that are returned. shifting borders my first reaction to the notion of librarians running reading groups in second life was a question of whether this was akin to putting a reference desk in a bar. my second reaction was a question of how our systems will support these extra-library interactions. can people quickly and easily trade urls to access the library materials they’re talking about? will library systems ever be as easy to use as the game/social environments we’re trying to use them in? living the life embarrassing, stupid online without contradicting the moral weight of social software post from last week, let’s take a moment to look at three stories from arstechnica about myspace and others: online video leads to teen arrests, shooting rampage avoided due to myspace posting, and google + facebook + alcohol = trouble. these are the stories we’ve come to expect: teen does or post the results of something [stupid|illegal|dangerous] in [myspace|facebook|some other online place] and gets caught. that crazy gnarls barkley other than the notion that i heard it on a kcrw music show, i couldn’t put my finger on the tune weaving through my head. so i listened, and listened carefully, waiting to hear it again. eventually i learned the earworm was gnarls barkley‘s crazy (thanks to molly for the mp download link). the group, a collaboration between dj danger mouse (of the grey album infamy) and cee-lo, released the single on myspace and created a new instant sensation in late march. movie: airport iain anderson‘s animated film, aiport, shows even the most pedestrian of designs come to life with a bit of creativity. elsewhere, a post at copyfight, suggests that the availability of those symbols — their freedom from copyright and trademark restrictions — was a key factor in spurring their broad adoption, creating both the culture and the free imagery for artists like anderson to use in their cultural commentary. bush: “i invented the ipod” president bush, speaking in alabama at the american competitiveness initiative, made a claim that would make al gore blush: he claimed to have invented the ipod. after taking credit for the development of ultra-small hard drives, audio compression, and chemistry(?), he laid it out: “it turned out that those were the key ingredients for the development of the ipod.” tip o’ the hat to engadget. bibliochaise what book lover doesn’t look twice at this bibliochaise from nobody&co? the wealth of networks wendy seltzer gave a shout-out for yochai nenkler‘s the wealth of networks: how social production transforms markets and freedom, describing it as… …an economic history of information production. we’re moving from the age of industrial information production to one of social information production. ever-faster computers on our desks let us individually produce what would have taken a firm to organize just a decade ago. ever-further networks let us share that with the world as cheaply as storing it for ourselves. danah boyd on the moral weight of social software danah boyd posted recently at many-to-many about the future of social software. i’ve been more than a little bit gung ho on web . for a while, but i do like her caution: if myspace falters in the next - years, it will be because of this moral panic. before all of you competitors get motivated to exacerbate the moral panic, think again. if the moral panic succeeds: youth will lose (even more) freedom of speech. wordpress baseline changes to support wpopac i’ve whittled things down to the point where the only baseline change from wordpress . . is in the next_posts_link function of the wp-includes/template-functions-links.php file. the change is necessary because wpopac rewrites the sql search queries in a way that’s incompatible with a piece of this function, but necessary for performance reasons. where’d all my rewrite rules go? between wordpress .x and .x there was a big change to the way rewrite rules are handled. in the old days, everything got written out to a <a href="http://httpd.apache.org/docs/ . /mod/mod_rewrite.html#rewriterule">.htaccess</a> file. every condition, every form of permalink could be found there, and i had some comfort knowing i could see and mess with it all. i was a bit surprised to find that with . . , wp writes out a sparse file that has only one significant rule. bloody tax day april has been tax day in the us for as long as anybody can remember, but with the weekend and all, most of us have ’til monday to file and some of us in the northeast have ’til tuesday. the thing i don’t like about tax time is that it brings out the worst in me. most any other time of the year i’m a pinko liberal, but the anticipation of taxes makes me look decidedly conservative and ornery. the crucible who wouldn’t like to play with the crucible‘s “fire truck”? what’s “the crucible”? [it’s] an arts education center that fosters a collaboration of arts, industry and community. through training in the fine and industrial arts, the crucible promotes creative expression, reuse of materials and innovative design while serving as an accessible arts venue for the public. you can see the truck at the make magazine maker faire later this month, and in july at the crubible’s fire arts festival. movie combos this is strange enough on its own, but i dare you to use it as a soundtrack to this one. printer fingerprinting news came out a while ago that many of our laser printers were embedding “fingerprints” that allowed folks who knew how (like, say, the feds) to trace a printed page back to the day and time it was printed, and the serial number of the printer. or, at least that was the theory, until the eff got all csi on it. the image above is magnified x and illuminated with blue light to increase the contrast of the yellow dot pattern used by xerox docucolor printers. php ’s simplexml now passes cdata content i didn’t hear big announcement of it, but [deep in the docs][ ] (? php . . ) you’ll find a note about [additional libxml parameters][ ]. in there you’ll learn about “libxml_nocdata,” and it works like this: simplexml_load_string($xmlraw, ‘simplexmlelement’, libxml_nocdata); without that option (and with all previous versions of php/simplexml), simplexml just ignores any < ![cdata[...]]> ‘escaped’ content, such as you’ll find in most every blog feed. [ ]: http://us .php.net/manual/en/function.simplexml-load-string.php [ ]: http://us . reboot your ‘pod colin has a nifty guide to your ipod’s hidden commands, like those for rebooting or getting into the diagnostics. he’s got more ipod tips if you look. good headline don’t these mainich daily news editors think they’re the shit when they get to combine “bondage” and “rope” in the same headline. i will trademark your every word yes, as it turns out, “freedom of expression®” is a trademarked term. and, yes, as it turns out, somebody’s been cease and desisted for using it. email is for old people i happened to stumble back onto the pew internet report on teens and technology from july that report that told us “ % of [us children] between the ages of and are online.” but the part i’d missed before regarded how these teens were using communication technology: email, once the cutting edge “killer app,” is losing its privileged place among many teens as they express preferences for instant messaging (im) and text messaging [sms] as ways to connect with their friends. i m super heros gotta go too, ya know? we regret the error not all errors in news reporting are as trivial as this one: the cost of beer kegs has risen by about % since the end of . in addition, neil witte is the draught beer quality-control specialist of boulevard brewing co., and steven pauwels is the brewer’s brewmaster. a march page-one article on beer-keg theft incorrectly said that the cost of kegs has tripled in recent years and incorrectly said that mr. and he did it in a tie steve jobs demos nextstep macs vs. pcs vista delayed the delay is the latest problem for the software giant’s flagship operating system. microsoft had originally slated the software for release in late , but pushed back its target date to summer and dropped several planned features to try to guarantee delivery. the company attributed the delay to the extra time needed to insure quality and fix remaining security issues. macsimum news | apple & macintosh related news reviews & opinions bad quality i should be all down on this sneaky way of advertising nokia’s n , but…eh, they’re funny. bad quality officechairs is the latest, bad quality hydraulics (somebody tell them it’s “pneumatics”) and bad quality superglue bring up the rear. if that isn’t enough, they’ve got the bad quality blog which pulls back the curtain a bit. if you look around a bit, however, you might stumble across nokia’s lifeblog (“feed it, watch it grow”): zhang huan’s “my boston” most people may recognize zhang huan from his “my new york” work that had him dressed in a beefy muscle suit. above is “my boston,” but i have a feeling it might get repurposed elsewhere during finals this spring to represent the agony of study. ups to ryan for the pointer. drive thru crucifixion titles and typefaces ryan pointed out that the titles for thank you for smoking are pretty interesting, then he followed up with a pointer to some font spotting at typographica. dns problems things went whacky with dotster‘s hosted dns services last night. though the problem now appears to be fixed on their end (and i’ve actually move elsewhere in my attempts to get back online), it could be a while before the bad data is flushed from caches around the world. in the meantime, let me mention that ryan shared with me a useful tool i’d not seen before: dnsreport. interesting, scary ilya khrzhanovsky’s . more. identity management in social spaces (note: the following is cross-posted at identity future.) being that good software — the social software that’s nearly synonymous with web . — is stuff that gets you laid, where does that leave idm? danah boyd might not have been thinking about it in exactly those terms, but her approach is uniquely social-centered. she proposes “secureid” what is secureid? secureid is a program that helps you protect and control your digital identity by allowing you to determine who can access your private information. big iron won’t win wars anymore technology changes things, sure. the question is, how do you recognize the early signs of change before they become catastrophic? i spend most of my days working on that question in academia, but what about our armed forces? noah shachtman regularly covers that issue in defensetech: like a lot of other sage observers, naval postgraduate school professor john arquilla isn’t nuts about the idea of spending a ton on cold war-style weapons systems when we’re supposed to be fighting terrorists and insurgents. sparkline php sparklines are “intense, simple, wordlike graphics? so named by edward tufte. in lieu of a more detailed introduction, professor tufte’s site has an early release of a chapter on sparklines. cool. here’s a php library and accompanying documentation wiki. more bsuite hacking update: bugfix release b v available. some conversations with chow kah soon, who’s site is full of diversions from work , finally convinced encouraged me to solve some small problems that were giving him big trouble. chow kah soon is in the lucky, but rare, position of having over , unique daily visitors to his site, so he’s sort of my designated stress-tester. after looking at the logs he shared with me, the table structure, and the queries in bsuite, it was pretty clear that i needed to make some changes to the indexes. winter’s last breath snow and rain mixed throughout the day tuesday, but we awoke to glistening white fields and trees. above is the view due west in wentworth this morning, before the warm spring sun melted it all away. don’t think you use web . ? think again it can be hard for library folk to imagine that the web development world might be as divided about the meaning and value of “web . ” as the library world is about “library . ,” but we/they are. take jeffrey zeldman’s anti-web . , anti-ajax post, for instance. zeldman’s a smart guy, and he’s not entirely off-base, but let’s not confuse his argument. what you don’t see him suggesting is that we abandon the web. “i hate drm” and other projects to preserve the digital artistic commons people hate drm. it prevents law abiding folks from enjoying the music and movies they’ve purchased, and it does little to prevent crackers from making illegal copies. in response, somebody’s created i hate drm, “a site dedicated to reclaiming consumer digital rights.” i created this site because, as a consumer, i am fed up. i feel like all of the entertainment that i love is slowly being eroded away by overly greedy companies. number sequences think about it, at the moment this post went live, it was one hour, two minutes, and three seconds past midnight greenwich mean time. why’s that matter? it doesn’t, but it looks cool: : : - - of course, brits and most others don’t represent dates that way, so the point is really only valid in us local time. c’mon, let’s wait up. richard sambrook talks citizen journalism i’m not sure what to think of richard sambrook appearing to struggle to find a place for traditional journalism in the age of the internet, but the story’s worth a read. david weinberger […] talked about the crisis in us journalism with failing trust in the big news organisations. he pointed out that google now provided a news service with just an algorithm where there used to be a newsroom of dozens of people — and suggested algorithms were probably more reliable than journalists anyway! getting things done, and feeling okay about it how’s a guy supposed to feel when his manager gives him a copy of david allen’s getting things done? go get yer podcast on gizmodo pointed out these usb and firewire podcasting kits from alesis. the package gets you a (hopefully not sucky) microphone with desktop stand, headphones, a carrying case, podcast production software, cubase le recording and editing software, and a digital mixer that plugs directly into the computer via usb or firewire (duh). the us$ usb version does two channels of bit/ . khz audio while the us$ firewire model cranks eight channels of bit/ khz sound. information behavior it was more than a year ago that lorcan dempsey pointed out this bit from the chronicle: librarians should not assume that college students welcome their help in doing research online. the typical freshman assumes that she is already an expert user of the internet, and her daily experience leads her to believe that she can get what she wants online without having to undergo a training program. indeed, if she were to use her library’s web site, with its dozens of user interfaces, search protocols, and limitations, she might with some justification conclude that it is the library, not her, that needs help understanding the nature of electronic information retrieval. atlanta art scene, spring atlanta was a bit of a lark. i hadn’t seen my friends for a while, and they were telling me that the weather was beautiful. so why not go? anyway, chuck close is on display at the high museum. and the thing about close’s work is that it frustrates my rule of “don’t do twice what you can automate once.” many of his portraits are the result of carefully mapped and measured graph lines that allow him to create pixelated works. water feature we were excited in new hampshire to have the first week of weather warm enough to go out without our coats at midday, but atlanta was warm enough to hop in the pool and hot tub after midnight. abductions i don’t know how i feel about shilling for the california dairy industry, but this cow abduction site is pretty funny. be sure to watch the movie. want more, go look at mailorderchickens.org. the aural times thanks again to a good tip from ryan, i’ve get something new to laugh at: the aural times. did i really just put this together? huh. noah shachtman tells us that even with the wars in iraq and afghanistan raging, our military forces are spending $ billion to arm up for a new enemy. but whom? china. then over here we’re reminded that china is the us’s largest creditor. facts of life a person will do certain things for money. idm takes lessons from the microformats crowd a tip from [ryan][ ] sent me [looking][ ] at [microid][ ]: a new identity layer to the web and [microformats][ ] that allows anyone to simply claim verifiable ownership over their own pages and content hosted anywhere. the idea is to hash a user’s email address (or other identifier) with the name of the site it will be published on, giving a string that can be inserted — in true microformats style — as an element of the html on the site. …and a mechanical turk to rule them all paul bausch has concerns about amazon’s mechanical turk: i can imagine a world where my computer can organize my time in front of the screen better than i can. in fact, i bet [amazon’s mechanical turk] will eventually gather data about how many [human intelligence tasks] someone can perform at peak accuracy in a hour period. once my hit-level is known, the computer could divide all of my work into a series of decisions. involvement, inclusion, collaboration peter caputa dropped a comment on jeff nolan‘s post about zvents. the discussion was about how online event/calendar aggregators did business in a world where everything is rather thinly distributed. part of the problem is answering how do you get people to contribute content — post their events — to a site that has little traffic, and how do you build traffic without content? the suggestion is that you have editorial staff scouring for content to build the database until reader contributions can catch up, and that’s where peter comes in, suggesting that content and traffic aren’t where the value and excitement are: twenty years after chernobyl nearly years after the initial events of the chernobyl nuclear disaster of april , the story is still unfolding. this month’s national geographic magazine tells of the “long shadow of chernobyl” — grown children of the disaster now fear having their own children while some elderly residents return to their old homes inside the , square mile, still contaminated “exclusion zone.” the print article seemed to offer hope, noting that even the pines of the “red forest” — so called because they received so much radiation that it bleached the chlorophyl from them, and some say the trees actually glowed — are beginning to grow back now. germaine i found germaine across from the prudential center friday. his sound was good and i especially liked his snare drum. door of mystery i found myself wandering about boston public library for longer than i expected friday. part of it was the map exhibit and part of it was the architecture (and simply a place to relax for a bit). amusingly, stairs and stairways seem filled with drama at bpl, and if the guard hadn’t just warned me about taking flash photos, i might have tried to sneak a peak behind that door. questions are all around us these pictures are mostly foolish, but here’s a small point: none of us had ever seen a cop pull over a cab — certainly not a cab with passengers — before this, so we were all rather curious about why. in front of us stood a question, an example of the many questions we all encounter every day, and it’s the kind of question that few of us would ever suggest going to the library to answer. the things they do to students at rice i won’t say why i went looking for pictures of people getting poked with sticks (but you’ll figure it out in a later post). i will say i was happy to find these from the poke-a-spontaneous-combustion-member-with-a-stick-day at rice university. look, they even have a price list that includes: $ poke with a stick song/poem on demand two minute massage lick a sc member $ picture with [unreadable] kissing whack with a stick $ marker tattoo $ attempt hedge jumping $ human piñata shave a leg we wrestle each other $ jump into hedges nowhere on the site does it note how much the fundraiser netted for “rice’s best (only) improvisational comedy troupe. business marketing babble makes me laugh found on jeff nolan’s blog: competitive intelligence: “a large fuzzy animal may be a bear.” marketing: “sap can help you understand your fuzzy animals. with over years in the fuzzy animal industry, we know if you are looking at a bear, a guy in a coat, or a large dog.” communications: “in today’s world of increasing challenges, it’s obvious fuzzy animals are what our customers care about.” sales: “who cares what it is. tomorrow in human computer interaction my dutch skills are weak to non-existant, and without a google translator for macarena.be, i’m pretty much stuck with staring at the above video and contemplating the short description provided: a movie about the technology which apple has recently patented. it is not a movie made by apple but by some researchers. fortunately, this is an area where video is much more illustrative than words. i sometimes get accused of blue sky thinking when i speak of the role of technology in our lives, but while i go on about how access to huge volumes of instantly searchable information is changing us, this video shows a rather near future where we can manipulate it ways that seemed like science fiction just the other day. facial recognitition spytech goes social troy expressed both great amusement and trepidation in his message alerting me to riya, a new photo sharing site: i don’t know whether to say cool, or zool. the tour explains that you upload photos, riya identifies faces in your photos, then asks you to name them (or correct its guesses!). then you get all your friends to join up and we can all search for everybody by people, location, and time. speaking my language i loved this quote from dave young when i first found it, and i love it more now: talk to the customer in the language of the customer about what matters to the customer. bad advertising is about you, your company, your product or your service. good advertising is about the customer, and how your product or service will change their world. read that again, but replace the relevant bits with “user” or “patron” and “your library” or “your databases. wyoming libraries marketing campaign i have mixed feelings about the value of advertising — it’s worth pointing out that according to john battelle, google never ran an ad anywhere prior to going public — but i still enjoy seeing things like this wyoming libraries campaign. jill stover quotes wyoming libraries’ tina lackey with the news that “wyoming’s libraries are as expansive as the state, and as close as down the street.” i’m just hoping that a, the horse is real; and b, they auction it off. gates harshes poor, tells them to buy windows what’s sadder than people in burundi earning an average of only $ a year? it might be bill gates‘ criticism of mit’s efforts to bring affordable, networked computers to the poorest countries of the world in hopes of improving education (and communication and healthcare and more). the challenge is enormous: the technology needs to be durable, require low-power (and be easily rechargeable), as easy to use as an egg timer, have networking in a land without infrastructure, and be cheap, cheap, cheap. can actors sell their digital clones? alan wexelblat in copyfight poses a question from a reader about the future of entertainment: what rights do you purchase/license/contract for in creating such a reproduction of a real person? rights to the “likeness?” performance rights? do either of these cover things the actor never physically did or said? is there an exclusivity clause? there are clearly some issues around the ownership of a character, if that character has appeared before (e. pravda march headline: us to collapse on feb i regularly check the english language online edition of pravda for laughs and sometimes for their take on us domestic affairs. but today’s headline left me scratching my head. what calendar are these people using, anyway? the headlined story is offered without any context or explanation. as it turns out, author ian magnussen really did mean february th , not or later. had it appeared two months ago it might have been called speculative fiction, though more likely seen as a crazy conspiracy theory. flight of the conchords ryan sent along a link to flight of the concords‘ business time last week and i’m still laughing over it. with some exploring at a fansite, what the folk!, i dug up a trove of other amusements, including she’s so hot boom. for more info, i turned (as usual) to the wikipedia article. and if i had hbo, i could have caught a repeat of them on one night stand this past wednesday. maisonbisson cultural reporter at sxsw, can’t get tickets, brushes with owen wilson instead sxsw passes have apparently been sold out for weeks now. so what’s bob garlitz, the maisonbisson cultural affairs reporter, to do? hunt for celebrities around austin, of course. here’s how he describes his first hit: i look at him intently, he’s about six inches in front of me. a long pause as i study his face and especially note the nose. he waits, expecting, knowing, what’s next. he’s shorter than me, in a white cap, white t-shirt and maybe white jeans. everybody’s irish with a quart o’ whiskey in ‘em modern drunkard magazine suggests we chase the snakes out of our minds, for as yeats reminds us: the problem with some people is that when they’re not drunk, they’re sober. (ryan points out that you can have that quote, along with three others from quipsters dylan thomas, w.c. fields, and oscar wilde on shot glasses.) but modern drunkard and yeats (despite his fine heritage) have it wrong. saint patty’s day isn’t about getting drunk or being drunk, it’s about getting silly enough to think you can dance a jig or sing a song. native to web & the future of web apps yahoo’s tom coats was of seven star speakers at carson workshops‘ future of web apps summit last month. as usual, ryan eby was pretty quick to point out his slides to me, mostly by way of pointing out jeremy zawodny’s translation of them. if it’s not clear yet: i wasn’t there, though i very much wanted to be, especially given some of what can be found in the post-summit blog posts. office cocktails i like pretty much everything paula wirth puts up on flickr, but this afternoon i could do well with a dive like scolari’s office in san diego. but, that’s probably because it mixes “office” and “cocktails” in the sort of way that has anonymous tipsters slipping photocopies of the alcohol policy from our hr handbook under my office door. eh, here’s to happy hour. homeland security: now policing porn? the washington post reports two men in uniforms bearing “homeland security” insignia walked into a bethesda library in early february, announced that viewing of internet pornography was forbidden, and began questioning patrons. the men asked one library user to step outside just before a librarian intervened. then… the two men [and the librarian] went into the library’s work area to discuss the matter. a police officer arrived. in the end, no one had to step outside except the uniformed men. the code lib journal(s) i should’ve kept code lib was less than a month ago, but already i’ve forgotten some details. that’s why i’m glad to have notes from ed summers (day one, two, and three), art rhyno, tom hickey, karen coombs, and ryan eby. there was a lot going on, and if i missed your blog it’s because google and technorati didn’t know about it (or i was being particularly lazy with my searching). our connected students just when you thought i was done talking about how the internet really does touch everything, lichen posts some details from the most recent university of new hampshire res life student survey and it gets me going again. in order, the top three activities are: socializing ( . hours/week) studying, excluding in-class time ( . hours/week) instant messaging, ( . hours/week) lichen also points out that im activity was reported separately from “personal internet use,” which got an additional . this is what social software can do the flickrblog reports this message from gale: people have been submitting good humpback whale fluke shots to a group called humpback whale flukes. i volunteer at allied whale which holds the north atlantic humpback whale catalog and i was able to make a very exciting match with one of the whales that was posted on the group by georgek. george saw this whale in newfoundland in the summer of . willie mae rock camp for girls the willie mae rock camp for girls: just another example of why new york is cooler than new hampshire. photo by rocco kasby, performance by the pink slips. yet again, a tip of the hat to ryan eby for the pointer. bsuite feature: user contributed tags ross singer gets the prize for submitting the first reader contributed tag, the latest feature in bsuite. there are arguments about whether user-contributed tags are useful or even valid, or whether they should be stored in my site or aggregated at places like del.ici.ous. but who’s to worry about such questions? who’s to worry when you can put together the work already done to support author’s tags with wordpress’s pretty good comment system and get user contributed tag support with just a few extra lines of code? user experience map i was this close to posting soldierant‘s gobbledy gook map, but, well… i guess i wanted to make a point with his user experience map, done in collaboration with the smart folks at experience dynamics. take a careful look at the role of your competitors and a user’s expectations and goals. yeah, we’ve all got some work to do. too bad the free seminar schedule hasn’t been updated for . whisky essential to writing god bless william faulkner for pointing it out: my own experience has been that the tools i need for my trade are paper, tobacco, food, and a little whisky. nash edgerton’s lucky scott smith’s imperfect ten too slow for you? take a look at nash edgerton‘s lucky over at blue tongue films. what would you do in minutes seconds? how would you escape? zorb: another reason new zealanders are cooler than you who of us didn’t want to try it when we saw jackie chan bounce down a mountainside in one in operation condor (well, who of us who saw operation condor didn’t want to try it)? but until cool hunter gave me a pointer, i had no idea what the these strange inflatable balls (yeah, go google that) might be called or where to look for more information. as it turns out, they’re called “zorbs,” and the company even has a promo video to show them off. nuns vs. librarians in spelling bee from yahoo! news and ryan eby, there’s a funny spelling bee planned in erlanger kentucky: erlanger, ky. – after a five-year hiatus, the sisters of st. walburg monastery in villa hills are ready to show whether they are superior spellers. the sisters were champions of the annual corporate spelling bee for literacy in northern kentucky for years before giving others a chance to win. but now the nuns are back, even if they’re a little timid about challenging the reigning champions — a group of boone county librarians. scott smith’s imperfect ten the nice folks at coudal partners are hosting scott smith’s imperfect ten, “wherein one man breaks all ten commandments before breakfast.” it’s friday (march th, even), go watch. crisp green shirt between the mit show and microsoft’s vaporware, origami is back in a big way. here’s drumsnwhistles answer: a very crisp green shirt. all about opensearch and autodiscovery from davey p i’ve been meaning to point out (and steal from) dave pattern’s post on tipping off ie (and other browsers soon too, hopefully) to available opensearch targets for some time now. i haven’t had time to do the stealing, so i’ll have to settle for pointing it out while it’s still news. what’s the trick? as dave explains, you put a link in the <head> section of your pages like this: visual complexity i found the above image of a yfiles-generated site map at visualcomplexity.com. we’ve seen a lot of internet diagrams, including this one from , but what about mapping food? or disaster situations? or air routes? it’s like data porn, and there’s more in the visualcomplexity gallery. the ignorant perfection of ordinary people bob garlitz, who’s trying to decide between blogging at typepad and blogspot, wrote to offer a somewhat older phrase for the success of social software as described in the wisdom of crowds and in the definition of collabulary: “the ignorant perfection of ordinary people.” bob is at a loss to identify the source (and it pre-dates the book of the same title by a long shot), but maybe this crowd will know? mit origami competition ryan eby and make magazine alerted me to mit’s student origami exhibit, in which jason ku’s ringwraith won the best original model prize, and brian chan’s beaver — the mit mascot — got special attention from the mit news office. collabulary i found this a few days ago and realized that it embodied the difference between how i understand tag folksonomies and how others (with whom i’ve argued) may see them. that is, i see the role of the social group — the wisdom of the crowd — as essential to the success of our folksonomic efforts. as it turns out, somebody’s come up with a word that emphasizes that (uncoordinated) collaboration: collabulary. talking ‘bout library . users want a rich pool from which to search, simplicity, and satisfaction. one does not have to take a -minute instruction session to order from amazon. why should libraries continue to be so difficult for our users to master? — from page of the the university of california libraries bibliographic services task force final report. i find a new gem every time i look at it. robins at bath i heard birds chirping yesterday morning for the first time in a while, and from my office window i could see robins returned from the south. spring, it seems, has arrived in new hampshire, but nobody’s captured it better than breezin with the photo above — obviously taken from a somewhat warmer place than this in late january. tags done right flickr does tags better than any other, so far as i can tell. we love tag folksonomies for way they allow us all to organize our world, for the way they allow patterns to emerge from chaos, and for their easy flexibility. but that flexibility, if poorly implemented in our software, can interrupt the very patterns we hope to find in our tag networks. take “road trip” as an example. what one tagger thinks is two words might be just “roadtrip” to another. macbook pro reviewed jacqui cheng likes her new macbook pro and loves the performance, but gives the magsafe power adapter mixed reviews. why? she says it disconnects when it shouldn’t, and seems to stay connected when it should disconnect. well, i think i still want one. troy bennett at “ben show” ben apfelbaum died before having the chance to see it all come together, but his quirky idea seems to be a hit. here’s how jerry cullum described it for the atlanta journal constitution: “the ben show” was the brainchild of beloved spruill gallery director ben apfelbaum, who asked one day, “what’s in a name?” and proceeded to track down a host of artists named “ben.” well, actually, he asked, “is the use of a given name as a thematic device as useful as any other thematic device to create an art exhibition of interest? podbop rocks your calendar ryan eby pointed out podbop, a site that podcasts sample tracks from bands coming to your area (or any other area you select), and we both wished we’d thought of it ourselves. there’s nothing coming to warren (of course). but they’ve got coverage for denver, where i’ll be in may, so it immediately found a place in my podcast aggregator. laura fries might have covered the smart and cool factors best: oddest title of the year winner …and also rans the bookseller magazine friday announced the winner of the th annual diagram prize for oddest title. bookseller deputy editor joel rickett appeared on weekend edition saturday with the news, saying, as he did in a telegraph story on the matter: “it has been a pretty good year for strange titles.” the winner is people who don’t know they’re dead: how they attach themselves to unsuspecting bystanders and what to do about it by gary leon hill, but the list of nominees and near nominees included rock paper scissors posted on the wall in tom’s peacock bar in corvallis was a mystery: a notice of a rock paper scissors tournament. a visit to the usa rock paper scissors league‘s website proved more confusing. take the first news release as an example: rocky balboa is stepping back into the ring for his final comeback, as production has begun on “rocky vi: rocky paper scissors.” after a -year hiatus, sylvester stallone wrote the film himself, knocking out boxing from the script and replacing it with a hand sport that is more intense, more courageous and that looks even better in those dramatic slow-motion shots: rock paper scissors. “peanutty” ≠ peanut butter treehugger pointed out these p.b. slices as an example of excessive packaging. what they didn’t mention was the ingredients or processing used to make a non-sticky, peanut flavored “food product.” peanutty, but not quite peanut butter it’s worth mentioning here that i have a rule about things i find in the supermarket: if it says “food” on the label, you probably shouldn’t eat it. think about it, start first with the cat food, dog food, and fish food, then take a look at the pasteurized processed cheese food product and some of the goodies in the canned meats aisle. fun with (explosive) balloons okay, so this is certainly in the “don’t try this at home, kids” category, but we can all laugh and point at other’s stupidity. denver‘s abc channel reported last month on a foolish fellow who inflated balloons with acetalyne, the highly flamable and explosive gas used in welding, and drove off to a superbowl party. the balloons ignited, possible because of static electricity, and the explosion blew out all the windows, bent the car’s roof and doors out, and left the driver and with burst eardrums, burns, pain, and a felony explosives charges. can anybody explain this? ???????????????? morbidly curiouser zach saw my story about plane crashes and forwarded me a link to this video of an early parachutist he found on damn interesting. the connection to yesterday’s story is that the video ends with cops measuring the depth of the crater the jumper left after falling almost feet from the top of the eiffel tower. it’s the sort of thing that gets you nominated for a darwin award. morbidly curious a friend pointed me to planecrashinfo.com and i can’t help but explore. i was told to start with the pictures (which end in late , and so don’t include recent incidents like the flaming nose-wheel at lax or the overshot runway in chicago), but it was the collection of “last words” transcripts from the cockpit voice recorder (audio is available for many of them) that really trapped me. we might get a furtive chuckle over such last lines as “hey, what’s happening here” or “uh. the oregon attractions i didn’t see i’ve been back from oregon for about a week and a day now, and it’s really time to clear out my files. so here now are the attractions i had put on the list, but never got to see. i’m not complaining, afterall, i did get to see sprayfoam art, the us’s only municipal elevator, the world’s tallest barber pole, the spruce goose, mt. tabor, and the velveteria. clearly, oregon has a lot to offer wacky travelers. is sun’s t up to it? jonathan schwartz made the kind of news that makes slash dotters happy: he announced sun is (sort of) giving away free servers. it’s a promotion, a media play, of course, but one that might make a few lucky people very happy. here’s the deal: sun is really proud of their new t eight core server. each core runs at . ghz, but they’re apparently applying some distributive power of multiplication and calling it an . lego architecture the millyard museum was hosting the new england lego users’ group saturday, building lego replica’s of manchester nh‘s old victorian-era houses. it turns out they’re building a scale model of the entire millyard. love letters from your isp a friend got his own cease and desist letter the other day. his isp forwarded the notice from a copyright enforcement agency along with five pages of content intended both to stop those that know they’re sharing and help out parents (or others) who may not be aware of what all is going on with the computers attached to their cable modem. of course you’re a valued customer, and of course it wasn’t your fault, just stop it is the message. worse things a friend forwarded this, from fleur adcock: things there are worse things than having behaved foolishly in public. there are worse things than these miniature betrayals, committed or endured or suspected; there are worse things than not being able to sleep for thinking about them. it is a.m. all the worse things come stalking in and stand icily about the bed looking worse and worse and worse. as the useful becomes useless, it becomes art the story here isn’t about why i’m on the kate spade mailing list. the story is about their new line of “paper.” it’s stationary, of course. the kind of formal paper people use to send out wedding invites and thank yous and whatever other little missives that email or aim seem too uncouth for. i made this point before, in a discussion of how painting evolved from trade-craft to art after the development of the camera, but i love seeing a new example. standards cage match i prefaced my point about how the standards we choose in libraries isolate us from the larger stream of progress driving development outside libraries with the note that i was sure to get hanged for it. it’s true. i commented that there were over , registered amazon api developers and public opensearch targets (hey look, there’s another one already), but that srw/sru would always play to a smaller audience. evergreen aviation museum howard hughes‘ spruce goose now rests in mcminnville, at the evergreen aviation museum. the goose is as long as a with a wingspan a third again as broad, and for a short few seconds in , it flew. the docent was incredibly pleased to tell us that the tail almost broke off during those few seconds in the air. he claimed hughes hushed up the story and maintained the aircraft in flight-ready condition to protect himself from further attacks from government accountants. diy hoverboard my friend troy sent along a pointer to the gadget show‘s feature on diy hoverboards. they claim it all goes together with basic tools, a leaf blower, plywood, a bit of pipe, and other various parts totaling about £ . oh yeah, they also recommend “an insurance policy with good fringe benefits,” and being as british as they are, apparently “craft knives” and “scalpels” are pretty interchangeable. it all goes together in eight easy steps explained on four pages, so what’s keeping you? about my code lib presentation as with all my other presentations, the my slides tell less than half the story, but i’ve posted them anyway. i’m told the audio was recorded, and there’s a chance that will help explain all this, but until then you’ll have to piece this all together from my previous writings, what little i’m about to offer here, and the slides (which, again, without the spoken component, probably do more to misdirect interested readers than answer questions). brick i just popped in the constant gardener (trailer) and discovered the preview for brick. and even though i want to see almost every movie previewed for me, i really want to see this movie. the constant gardener, by the way, is good too. velveteria i wasn’t just surprised to find a gallery of velvet paintings, i was further surprised to learn they were hosting a show of valentines velvet works by local artist juanita and had cards advertising a show of la artist arnold pander’s oil on velvet works at the local vault martini lounge. but the fact is, carl baldwin and caren anderson’s velveteria is the place, if ever there was such a place, where such forces will collide. world’s tallest barber pole forest grove, oregon claims to have the world’s tallest barber pole, apparently presented by the portland area barbershoppers in recognition “ballad town usa’s” role in promoting and encouraging barbershop quartet singing. it stands in lincoln park (visible from sat photos!) just north of pacific university. barbershop poles and quartets they may have, but the barber i visited there did a lousy job trimming my beard. such is life, i suppose. librarians of springfield that’s my contribution to the springfield public library meme that michael casey and laura savastinuk started over the weekend. oregon city municipal elevator oregon city apparently boasts one of only four municipal elevators worldwide. one hundred thirty feet tall, with an observation deck at the top, it seemed to be worth stopping for. jason wrote in to roadside america explaining: it began as a water-powered elevator in , but was upgraded to an electric-powered elevator in . it is an example of googie architecture, which is reminiscent of the space-age housing structures in the jetson’s cartoon show. pdx’s free wifi rocks here’s a lesson the rest of the world’s airports could take from pdx: free wifi. most other aiports charge dearly for wifi, but pdx offers it free. knowing this, i arrived at the airport a couple hours early and got my dinner and caught up on my email here instead of elsewhere. the port of portland didn’t get my $ . an hour, but they did get an extra customer in their restaurants and shops. mt. hood from mt. tabor above: tonight’s sunset view of mt. hood from atop mt. tabor, an ancient volcano. roadside america claims: this is the only volcano located within a city limit in any us city. you can view the cinder cone and a few feet away from the parking lot is a kids play area. sprayfoam art in millersburg what you can’t tell about the photo above is that the eagle is huge, and made of spray foam. it stands at sprayfoam inc., just off the i at millersburg. don’t miss the cornucopia-like sign, or the completely enfoamed sprayfoam-mobile. the chuck norris meme i first caught up with all this at matt‘s blog, but on the radio out here in oregon today they kept inserting chuck norris legends between songs. here’s a bunch from chuck norris facts: when the boogeyman goes to sleep every night, he checks his closet for chuck norris. chuck norris doesn’t read books. he stares them down until he gets the information he wants. there is no theory of evolution. lessons from the microformat world i can’t help but like microformats, and part of that comes from the dogmatic principles that drive them. among those is the notion that none of us should attempt to create a format out of whole cloth. here’s how they explain it: under the title of “propose a microformat” they tell us: “actually, don’t!!!” ask yourself: “are there any well established, interoperably implemented standards we can look at which address this problem? things i learned at lunch today karaoke means “empty orchestra” in about the same way that karate means “empty hand.” the “oke” piece is actually a shortened form of “orchestra,” borrowed from western languages. ethiopians supposedly discovered coffee when they noticed goats eating the beans. no word on weather the coffee beans in their droppings are any good. you mean other businesses handle acquisitions too? art rhyno confused my by calling it erp, but he just rocked his code lib presentation and i realized he’s talking about the same thing that’s been itching me: libraries are not unique, but our software and standards are unnecessarily so. in my introduction of wpopac i made the point that i didn’t want to replace the ils — certainly not the acquisitions management functions or other business processes. art today explained that he wouldn’t want to have to develop or support those features either, but that we don’t need to. pig-n-ford races!?! so here i am looking up things to do in oregon and i come across the tillamook chamber of commerce‘s guide to local attractions and its note about the pig-n-ford races: vintage vehicles, daring drivers and squealing porkers. mixed together, the outcome can only be described as frenzied farm-style fun. most people would agree that individuals who race model-t fords must be strange to begin with. when competitors insist on carrying pigs as passengers, however, it’s a sure sign of a rare breed of driver. on flying if i didn’t like flying, or at least if i couldn’t tolerate it, i wouldn’t making my third distant trip in as many months. and though i know many others spend a whole lot more time in planes than i do, i still think vasken has a bit of a point in the following: i couldnt help thinking about the horrid dichotomy that is airline travel… on one hand, my flight from philly to manchester takes minutes, or + hours less than the trip takes in a car–on the other hand, it took me hours to get from my house to the place i was staying in pa, a savings of a mere hours. instant messenger or virtual reference? i noted aaron schmidt‘s points on im in libraries previously, but what i didn’t say then was how certain i was that popular instant messaging clients like aol instant messenger or yahoo!’s or google’s are far superior to the so-called virtual reference products. why? they’re free, our patrons are comfortable with them, and they work (three things that can’t be said about vr products). ah, heck, just take a look at what michael stephens was saying about them last week (as quoted by teresa koltzenburg at ala techsource): choose your disaster the good people at keep the faye gave me a chuckle with their series of choose you daily disaster magnets, like the hillbillies and volcano series pictures above. then they followed it up with the amusing, but somewhat less funny choose your favorite fantasy series. mysql’s slow query log zach suggested it last week, but it’s only now that i’ve gotten around to setting up mysql’s slow query log. it’s easy enough, you’ve just got to put a couple lines like this in your my.cnf (which is in /etc on my server): log-slow-queries = /var/log/mysql/mysql-slow.log<br /> long_query_time = this should get most people running, but this story in database journal offers a few more details. potentially more useful is this guide to query and index optimization (though it’s probably a little out of date). nmc’s horizon report i’d never heard of the new media consortium before, but they claim a mission to “advocate and stimulate the use of new learning and creative technologies in higher education.” anyway, their horizon report identifies the following trends among those shaping the role of technology in education: dynamic knowledge creation and social computing tools and processes are becoming more widespread and accepted. mobile and personal technology is increasingly being viewed as a delivery platform for services of all kinds. roadside attractions perhaps it’s just because i’m now scouring roadside america for tips on what to do in the hours after the end of code lib and my flight home, but i got a hoot out of this ap story about “roadside giants”: a pittsburgh-area couple find “roadside giants” historic, attractive, a boon to local economies… and silly. associated press pittsburgh – how can you find the cadet restaurant in kittanning? high-speed photography the gallery at pulse photonics has more than a few images that seem to pause time in impossible moments. they’ve got images of balloons pierced by arrows and darts, oranges exploding from [a gunshot][ ], bullets [shattering glass][ ] and [slicing through jelly][ ], and all of this [falling water][ ] and [oil][ ] in [so many][ ] [little droplets][ ]. you really oughtta go see the [whole gallery][ ]. and after that, go visit the [photron gallery][ ] of slow motion videos that [caught my eye][ ] a while ago. bicycle snowplow to go along with summer’s bicycle riding mower is this “vancouver snowplow” from joe-ks.com (yes, i feel appropriately stupid for linking to a site with an animated gif splash page). oddly, this isn’t the only such snowplow. on being busy i should be thankful to have friends who get worried about me when i don’t blog for a couple days (or at least make up stories), but let me take this moment to make it clear that i haven’t gone into boat sales. this has happened before, and it just means i’ve got a larger than usual pile of deadlines (and interesting projects like wpopac) on my plate. wpopac: an opac . testbed first things first, this thing probably needs a better name, but i’m not up to the task. got ideas? post in the comments. for the rest of this, let’s just pretend it’s an interview. what is wpopac? it’s an opac — a library catalog, for my readers outside libraries — inside the framework of wordpress, the hugely popular blog management application. why misuse wordpress that way? wordpress has a a few things we care about built-in: permalinks, comments, and trackbacks (and a good comment spam filter), just to start. performance optimization a couple notes from the past few days of tweaks and fixes: hyper-threading has a huge effect on lamp performance. from now on, i’ll have bad dreams about running mysql without query caching in the way that i used to have nightmares about going to school wearing only my underwear. the difference is that big. wordpress rocks, but it has some queries that will kill large databases. i’m playing with baseline when i fix ’em, but it’s worth it. the web is not a one-way medium anybody who questioned the pew internet and american life report about how teens use the internet and how they expect conversations and interactivity from the online services they use might do well to take a look at this comment on my chernobyl tour story: student looking for info that your not give us february rd, : you people suck. we have to do a school report and you are not giving us any info on what happened to the people, and the environmetn, we need a story from someone and about someone who lived through this inccident. faqs about those three wishes i ran across david owen’s three wishes faq in a month-old new yorker on my friend’s coffee table last night. i tore out the page thinking i’d not find it online, but lo, the new yorker posted it on their site on jan ninth! you have been granted three wishes — congratulations. if you wish wisely, your wishes may bring you great happiness. before wishing, please take a moment to read the following frequently asked questions. libraries vs. drm within minutes of each other, two friends from separate corners of the world sent me a tip about the following: slashdot pointed to this bbc news that talks about the ill effects of drm on libraries. what’s drm? it’s that “digital rights management” component of some software and media that supposedly protects against illegal copying, but more often prevents legitimate users from enjoying the stuff they’ve bought legally. now think about how this works (or doesn’t) in libraries… exxpose exxon exxonmobil’s profits of $ . billion are apparently the largest ever recorded by any corporation in america. to celebrate, the folks at saveourenvironment.org put together this funny short: exxposeexxon. the movie makes some good points, but let’s face it, high oil prices encourage conservation and research on alternative energy technologies. is j. k. rowling carolyn keene’s sister? i said previously that i drop my journalistic standards on fridays. today is no exception. background, from mysterynet: carolyn keene is a writer pen name that was used by many different people — both men and women — over the years. the company that was the creator of the nancy drew series, the stratemeyer syndicate, hired a variety of writers. for nancy drew, the writers used the pseudonym carolyn keene to assure anonymity of the creator. as if retro fashion didn’t already go far enough i guess i can see why people might be willing to throw down $ or more for these fancy northstar refrigerators, i mean, they remind rich young people of their grandma’s house, with fresh-baked cookies and a big glass of milk to dunk them in. i’ve gotta admit, i almost got suckered too. but why is it that our rosy nostalgia for the s ignores both the racial segregation (a bad thing) and the income equity (a good thing)? onion story predicted five-blade razors in gillette’s fusion five-blade razor is hitting the shelves now, but the onion predicted it in february . aim and changing modes of communication there’s a bit of discussion of aim‘s role in personal communications over at remaining relevant. i mention it here because i’ve been thinking about this lately. we’re seeing some great shifts in our modes of communication. take a look at how “webinar” technologies have changed sales forces. the promise is lower costs and faster response time, but it also challenges our expectations and the skills of the salesperson. now imagine the generation of kids who are growing up with aim entering the workforce. the future of privacy and libraries ryan eby speaks with tongue firmly in cheek in this blog post, but his point is well taken. privacy is serious to us, but we nonetheless make decisions that trade bits of our patrons’ privacy as an operational cost. while we argue about the appropriate time keep backups of our circulation records, we largely accept them — and the way they connect our patrons with the books they read — without question. zach’s couch camouflage here’s zach hidden in plain sight on a couch at a friend’s house the other day. that’s skill. where’d my go? nobody remembers how, but the bottle is empty again. we’re beginning to blame it on bandits. warren (and dog sledding) on tv tonight the folks at wmur‘s chronicle are featuring my friends joe and wendy and their dog sledding tonight. the photos above are of justin in a race a few years ago (video of the finish also online). warren hasn’t been so proud since we put the rocket up. large format scanners for document imaging the market for large-format flatbed scanners is shrinking, so products turn over slowly and development is far behind my expectations. that said, the epson gt- doesn’t look like a bad choice for tight budgets. it has a relatively low maximum resolution of only dpi, but has the highest claimed scan speed of seconds at dpi. following that is the microtek scanmaker xl, which has a much higher maximum resolution, but much slower scan speed (even at the same resolution as the epson). what does facebook matter to libraries? lichen pointed me to this librarian’s guide to etiquette post about new technologies: keep up to date with new technologies that you can co-opt for library use. so what if no one will ever listen to the pod casts of your bibliographic instruction lectures, subscribe to the rss feeds from your library’s blog, send your reference librarian instant messages, or view your library’s profile on facebook.com? at least you did your part to make all these cool technologies a little bit lamer. walking desk i used to have a stand-up desk at work. then that got replaced by a pair of standup workstations above a more normal desk. then i moved offices and switched roles from sysadmin to programmer and got the most normal desk ever. then, in january , i heard an npr story about dr. jim levine’s study that put a high value on constant movement throughout the day, and i got concerned about sitting for so long. not invented here i couldn’t say it, but alexander johannesen could: libraries are the last bastions of the “not invented here syndrome” (scroll down just a bit, you’ll find it). between alex’s post and mine, i don’t think there’s much to say except this: there may be five programmers in the world who know how to work with z . , but several thousand who can build an amazon api-based application in minutes. what technology do you want to bet on? reviews you can trust cameron moll (via ryan eby) wants “weight” customer ratings to reflect how two products of the same rating might have wildly different numbers of reviews. at first glance i agree with him, but after a moment of thought, i begin to wonder if i want the ratings weighted by the number of reviews, or the number of reviews i “trust.” amazon keeps huge amounts of data about all its customers. so how hard could it be to correlate my purchasing behavior with the purchasing behaviors of the reviewers along with the details of which reviews i’ve previously checked as “helpful. indian frankie the plan was to meet jessamyn and greg at the india queen last night, so discovering this note yesterday on slashfood about “frankies” had the added excitement of both discovering a new food i wanted to eat, and being in a position to get it that day — the sort of instant satisfaction one doesn’t expect in these parts. here’s the description: the frankie is an indian street-type food made of a thin bread similar to a tortilla that is coated with egg and fried. conceding defeat i wasn’t really in the game, but when samb posted the above picture of david brown’s typical meal, i couldn’t help but take it as a challenge. i never did get around to snapping a picture to match samb’s, and now i’ve got accept that there are others with more skill and determination than me. slashfood explains that anybody can walk in to in-n-out burger and order a sandwich of any size. to blog or not to blog a friend revealed his reticence to blogging recently by explaining that he didn’t want to create a trail of work and opinions that could limit his future career choices. fair point, perhaps. we’ve all heard stories of bloggers who’ve lost jobs as a result of the content of their posts. and if you believe the forbes story, the blogosphere is filled with teaming hordes intent on ruining established companies and destroying the economy (okay, i exaggerate). to blog or not to blog a friend decided the old pornstar name formula was good enough to use to name her blog, as she explains in her launch story. so, should this be the nick hastings blog? elsewhere, another friend is struggling with the decision to blog. when you need to talk to customer support it’s good to know hard to find numbers.com is there when you need it. here are the top five: <td width=" "> htf# </td> <td width=" "> who </td> <td width=" "> notes </td> amazon.com <td> - - <br /> <br /> - - <br /> <br /> - - <br /> - - </td> <td> cust. service<br /> <br /> seller support<br /> <br /> rebate status local or int’l </td> <td> / <br /> <br /> "<br /> " ( press to bypass menu) <br /> " </td> ebay. dawg it’s friday, a day when i drop my journalistic standards and usually publish whatever video or joke somebody forwarded me during the week. this one came from my dad: a guy is driving around and he sees a sign in front of a house: “talking dog for sale.” he rings the bell and the owner tells him the dog is in the backyard. the guy goes into the backyard and sees a labrador retriever sitting there. plesk bites i picked plesk over cpanel as my server control panel because it was cheaper, looked better, and seemed to have all the features i wanted. what i didn’t know was that it came with php and mysql at times when each was a major version ahead of that. when the good folks at my hosting provider tried to upgrade this, it conflicted with plesk and they have to back off. quickly noted: mooflex cms new ajax-happy cms: mooflex, more info at ajaxian (and in their podcast). about sherpa and their advice to digital libraries… i mentioned sherpa a while ago: sherpa is a large consortial uk project that’s attempting to build an academic archive/repository for institutions, including the british library and cambridge university. [link added] i bring this up again now because they’ve got some advice for people on the subject of digital archives. they recommend eprints, an open source project developed and maintained by the university of southampton. second to that, or for those interested in archiving a broader variety of object types, they suggest mit’s dspace. users vs. network printers in winxp it’s been a problem we’ve struggled with here for much longer than we should have, and it took a hotshot new guy in desktop support to show us the answer. but if you know the right magic, you can add a printer to windows xp and make it available to all users. see, if you add the printer using the “add printer” wizard, it’s available only to that user. but if you use the command line, then you can throw a switch to make it available to any user who logs in to that machine. jenny levine’s online library user manifesto drawing from john blyberg‘s ils customer’s bill of rights and the social customer manifesto, jenny levine offers this online library user manifesto: i want to have a say, so you need to provide mechanisms for this to happen online. i want to know when something is wrong, and what you’re going to do to fix it. i want to help shape services that i’ll find useful. i want to connect with others that share my interests. cio’s message to faculty: the internet is here as part of a larger message to faculty returning from winter break, our cio offered this summary of how he sees advancing internet use affecting higher education: are you familiar with blogs and podcasts? google them, or look them up in wikipedia. some of you may already be using these new tools. others may think these terms are the latest in a sea of techno-jargon. regardless, your millennial students — the netgens — are using these new technologies — along with the ubiquitous cell phone — more and more. the arrival of the stupendous we can be forgiven for not noticing, but the world changed not long ago. sometime after the academics gave up complaining about the apparent commercialization of the internet, and while wall street was licking it’s wounds after the first internet boom went bust, the world changed. around the time we realized that over million americans have internet access, that million americans use the internet ?on an average day, and that % of them believe the internet is a reliable source of information, we looked around and found that along with doing their banking, their taxes, and booking tickets for travel and movies, those users were making about five billion web searches each month. goodbye san antonio you won’t get your salad dressing on the side in san antonio. i don’t know what it says about a place, but in new england it’s so common i never learned to ask for it on the side, it just happens. not so in san antonio. you’ll also have trouble finding a place to eat dinner away from the riverwalk, as all the neighborhood places i found are open only for breakfast and lunch. data visualization and the opac a chat with ryan eby, also an edward tufte fan, elicited this line about another reason we continue to struggle with the design of our catalogs: data isn’t usable by itself if it was then the opac would just be marc displays and yesterday i was speaking with corey seeman about how to measure and use “popularity” information about catalog items. it got me thinking about flickr’s interestingness metric, which seems to combine the number of times a photo has been “favorited,” viewed, and commented. presentation: designing an opac for web . ala midwinter iug sig presentation: designing an opac for web . update: pdf version with space for notes web . and other “ . ” monikers have become loaded terms recently. but as we look back at the world wide web of , there can be little doubt that today’s web is better and more useful. indeed, that seems to be the conclusion millions of americans are making, as current estimates show over million users in the us, including % of youth - . fully wired and mobile in san antonio i’m in san antonio for ala midwinter and enjoying the benefits of wide-area mobile internet access via my treo and and the power of local search. this is sort of a test for me and my treo, as i passed on all the usual trip prep i do and entirely i’m depending on what i’ll find in situ or in my mobile web browser. i wandered around a bit this afternoon to get a feel for the place, but as i got hungrier, i found myself stuck in the riverwalk mall, and without any local clues about where to look for better food (steers & beers, in the mall, might have been an option if it had more activity or if those few who were sitting at tables didn’t look so miserable). educause on future of libraries take a look at this editorial by jerry d. campbell, cio and dean of university libraries at the university of southern california: academic libraries today are complex institutions with multiple roles and a host of related operations and services developed over the years. yet their fundamental purpose has remained the same: to provide access to trustworthy, authoritative knowledge. consequently, academic libraries — along with their private and governmental counterparts — have long stood unchallenged throughout the world as the primary providers of recorded knowledge and historical records. goodbye x. in recognition of the divisive and increasingly meaningless nature of x. monikers — think library . and the web . that inspired it — i’m doing away with them. when jeffrey zeldman speaks with disdain about the ajax happy nouveaux web application designers and the second internet bubble (and he’s not entirely off-base) and starts claiming he’s moving to web . , then it’s a pretty clear sign that we should give up on trying to version all this. learning: mysql optimization i have over posts here at maisonbisson, but even so, the table with all those posts is under mb. now i’ve got a project with , posts — yes, , posts! — and the table is about mb. an associated table, structured sort of like wp’s postsmeta, has over . million records and weighs in at over mb (not including the mb of indexes). up to now i’ve been a “throw more hardware at it” sort of guy — and in a server with only gb of ram, that’s probably the best solution — but i also think it’s time i learned some mysql optimization tricks. radical, militant librarian the ala’s intellectual freedom folks came up with this radical, militant librarian button (which i found in library mistress’ photostream): in recognition of the efforts of librarians to help raise awareness of the overreaching aspects of the usa patriot act, the american library association (ala) office for intellectual freedom (oif) is offering librarians an opportunity to proudly proclaim their “radical” and “militant” support for intellectual freedom, privacy, and civil liberties. wordpress plugin: add to del.icio.us i’m not running it here (only because i’m too lazy), but i was happy to find arne brachold’s del.icio.us – bookmark this! wordpress plugin. it puts a sweet bookmark on del.icio.us link whereever you call this function: <?php dbt_getlinktag(“bookmark on del.icio.us”); ?> arne also wrote the google sitemap plugin i use (though it turns out i’m a few versions behind). us census on internet access and computing rebecca lieb reports for clickz stats that, based on us census data (report), most americans have pcs and web access: sixty-two million u.s. households, or percent of american homes, had a web-connected computer in , according to just-released u.s. census data. that’s up from percent in , and more than triple ’s percent figure. home web use continues to skew toward more affluent, younger and educated demographics. how i broke my clie it’s an unseasonably warm and rainy january here in warren, where warm actually means daytime highs of about degrees and ‘seasonable weather’ would be closer to zero. the point is that it’s the worst possible winter weather: the rain ruins the regular winter activities, and it’s still too cold to take up summer activities. perhaps that’s why i take such comfort in this video of ashton, even if it is the video that killed my clie. field of trains fishfin has an interesting collection of photos from the american plains. that old train car caught my eye and fishfin replied to my comment with more detail: this old train car sits about yards from the soo line railroad in north eastern montana, it’s in comertown, an old abandoned town were they used to run whiskey from canada to the us in the early ’s. [link added] fishfin ’s old train. highways think now of the us interstate highway system. like the internet that followed, the highway system was the subject of much hype and conjecture. most notably, norman bel geddes’ -designed general motors futurama exhibit at the new york word’s fair. in it we saw magical highways connecting our cities, and whisking motorists from new york to la in hours. he predicted cities would expand their commuting radius by % by . the library vs. search engine debate, redux a while ago i reported on the pew internet project‘s november report on increased use of search engines. here’s what i had to say at the time: on an average day, about million american adults use the internet; % will use email, % will use a search engine. among all the online activities tracked, including chatting and iming, reading blogs or news, banking, and buying, not one of them includes searching a library opac. more trends in online behavior from pew internet it turns out that the pew internet and american life project sort of keeps a blog. here are some points from a november post by project director lee rainie regarding “surprising, strange, and wonderful data:” the vast majority of most internet users ( %) and many non-users (about %) expect that they will be able to find reliable information online when it comes to news, health care information, e-commerce, and government. winter’s day winter in warren can be rather picturesque. poets, justice, scotch unattributable: “poetic justice is a lie. it’s no more real than military inteligence. the entire motivation for poetry is the unjust pain of life.” separately, what’s the appropriate lc classification for scotch? my first thought was around pr , but what do i know. should it go elsewhere? what about other spirits? joel friesen’s misuse of powerpoint joel friesen‘s powerpoint-esque presentation on why his girlfriend should continue to date him didn’t win her back, but it entertained folks. yes, the diagram above shows joel’s position at the intersection of those who are graphic designers, awesome people, and people who’ve played a zombie in a low-budget horror flick, yes the other slides are as entertaining. go look: why you should continue to date me; a series of charts and graphs. presentation advice from an apple insider mike evangelist’s look behind the magic curtain of apple keynotes during his time with the company. code lib program proposal i’d be excited just to be a fly on the wall at code lib, but i’m on a bit of a mission to change the architecture of our library software — to make it more hackable, and make those hacks more sharable — so i had to propose a talk. title: what blog applications can teach us about library software architecture description: the number of programmers in the library world is growing and our individual efforts have shown great promise, but they exist largely as a spectacle that few libraries can enjoy. looking at controversy through the eyes of britannica and wikipedia the argument about wikipedia versus britannica continues to rage in libraryland. the questions are about authority and the likelihood of outright deception, of course, and a recent round brought up the limitations of peer review as exemplified in the cold fusion controversy, where two scientists claimed to have achieved a nuclear fusion reaction at room temperature. randy souther, from the university of san francisco, asked us to look more carefully: boat full of toilets my inner -year-old is cracking up over the notion of a shipwrecked load of toilets in the mediterranean. magnetic fields, earworms, fido i can’t get fido, your leash is too long, from the magnetic fields‘ love songs, out of my head. this entry is an attempt to kill this earworm by posting the lyrics. if this doesn’t work i’m checking out maim that tune. fido, your leash is too long you go where you don’t belong you’ve been digging in the rubble gettin’ bitches in trouble tag clouds… “the tag cloud is the mullet of the internet.” found at phpflickr. look closely. gallery to flickr migration tool for those people still using gallery, here’s the last straw: rasmus lerdorf got to playing with the flickr api and quickly wrote up a script to migrate his photos from gallery to flickr. he’s didn’t post a script or anything, he’s just saying it’s easy to do. a lot of things are easy to do, of course, but that doesn’t mean they get done. so it’s probably a great relief to somebody that paul baron got on the job. ddos’d my hosting provider sent along the following message: we have experienced a ddos attack today january th, which resulted in latency across the entire network. during this time your domain, email, ftp and other services may have appeared to be offline, or intermittent. our techs have been working as quickly as possible to block the attack and get the network back up to speed. i was relieved to know that the unexpected downtime was the result of something i’d done. political blogging protected by fec way back near the end of , lot reported that the federal election commission had basically ruled that bloggers are journalists: the federal election commission today issued an advisory opinion that finds the fired up network of blogs qualifies for the “press exemption” to federal campaign finance laws. the press exemption, as defined by congress, is meant to assure “the unfettered right of the newspapers, tv networks, and other media to cover and comment on political campaigns. social software works for organizations too ignore the politics for a moment. moveon‘s cto, patrick michael kane, remarked that the organization’s membership to flickr, the photo sharing site, has paid off: “flickr has got to be the best $ . we’ve ever spent.” why? micah sifry explains in a story at alternet that moveon had been soliciting photos of events from members for some time, but their ability to move those photos through the process and make them available to the public was limited. wordpress . & bsuite update: bugfix release b v available. wordpress . is out and available for download now. i don’t know how the development team did it — i mean with the holidays and all — but here it is. and now i have to admit that i haven’t even played with the betas, so i’ve got no clue how bsuite works with the big . . for all i know it works just fine, or it drops all your tables and sends your browser history to your mother, so please take caution. avenue q steve wynn could probably have had any show he wanted, but he chose avenue q, the sesame street and muppets-inspired show that has to include a disclaimer denying its roots in the program and advertising. what the show’s creators don’t have to disclaim are the three tony awards the show won in for best score, best book, and best musical. sandee bought the cast recording (also at amazon) because they’re the sort of tunes that get into your head…the sort of tunes you’ll find yourself humming days later. the eating, drinking, and dancing in vegas vegas knows liquor. vegas knows drinks. they go well with cards and dice and slot machines and such. and even though the cards and dice and slot machines and such aren’t my reasons for going to town, i do enjoy a drink. above, center you see the west wing bar’s sidecar with cognac, triple sec and lemon juice. at the left is a pinapple mojito from the wynn’s terrace pointe cafe. nevada considers atomic testing license plate, again the first license plate to remember nevada’s history as the host of the us’s nuclear testing grounds drew criticism for featuring a mushroom cloud (see the plate on the right, above). now it appears folks are at it again, this time with a plate that depicts the site’s area and includes the classic illustration of an atom’s electron cloud. all of this generated enough interest to bring the local media out to the atomic testing museum to gawk at the proposed plate, including an actual-sized rendition being shown off on a lincoln navigator. nevada desert we didn’t get to go to barstow as planned, but i couldn’t leave las vegas without a peak at the desert. fortunately, red rock canyon isn’t far from town, and the blue diamond highway does a nice loop there and back. along the way i found that the town of blue diamond has a new welcome sign, but the old text remains: “elevation: high, population: low, burros: ?” i stumbled across an upended car standing like a tombstone exclaiming “dirt man rocks. font friends you’ve got to love a friend who emails you when she finds fonts like orange whip and comic strip exclaim and say they remind her of you. on censorship regarding nudity in photographs posted to flickr, dancharvey says: honestly, i’m more concerned about all the cats and flowers. cliche is more damaging than breasts. your opinion may vary. barstow california what didn’t work out because of our problems with the hotel was our drive to barstow to see sandee’s friend joanne. i don’t know much about the town, but wikipedia told me to look out for the original del taco, rainbow basin natural area (site not loading now, try this instead), calico ghost town, and the old solar one solar energy generating experiment. along the road, however, is the the world’s tallest thermometer, in baker, california. atomic liquors i convinced sandee to join me at atomic liquors on fremont street, just beyond the western hotel casino in what the las vegas sun calls the “gritty underbelly of las vegas.” owner joe sobchick and his wife stella started business in with a cafe called virginia’s. they converted it into a bar in , and changed the name to recognize their proximity to the nuclear tests just miles away. welcome to fabulous las vegas…with your host, casey the wind along las vegas boulevard was blowing hard, so it hides the fact that i’m currently sporting one of the worst haircuts of all time. i’ve been meaning to take a picture of this damn sign for years — and more so after seeing beatnickside‘s collection of vegas photos. what you lose in the whirligig… nobody’s saying what caused it, but things didn’t go as planned at the mgm grand sunday night. we were told our room wasn’t ready when we tried to check in a little before midnight, so we ambled over to the cafe for a midnight breakfast on the house. then at am, when our rooms still weren’t ready, we were sent to the bellagio with a voucher for a free room and cab fare. the real king kong here’s another story from my friend joe monninger. this time it’s a piece he cut from a book he’s working on, but i’m happy to take his tailings. the text that follows is his: with the mega-release of king kong swarming the country this week, it might be interesting to hear a true big ape story. i came across this story while doing research for a project, and i pass it along as it came to me. happy holidays from las vegas! the bellagio is all done up for the holidays, vegas-style (which means it’ll give you a headache). happy holidays from warren snow, thick and heavy because of the thaw these past few days, covers warren. our rocket stands tall for all seasons. shuffling ipods i couldn’t help but want one when they were released. i still wanted one after reading the reviews. and i couldn’t help but think about buying one when i finally got to play with it in the store. my wife, loving me and knowing me as she does, got me one. yes, i got a video ipod for christmas. thing is, presents like this create a crisis. how do i extract the gigabytes of music i’ve accumulated on the old ipod? last minute gift idea my friend joe loved his chickens, though a fox did them in this last fall. he’d planned to leave the coop empty for the winter and start fresh in the spring, but his surfing lead him to mail order chickens (adoption card pictured above). so…what better a gift for a friend than a chicken by mail? and what better a gift to the world than trade justice? santa vs. cops i always get a laugh out of cops, and an even bigger laugh out parodies of the show. so i have to thank cliff for finding this animated video of santa getting pulled over. the war on christmas i like christmas as much as anybody (well, anybody who likes christmas), but i’m a “happy holidays” guy. why? because christmas and the holidays aren’t about me, they’re about the way we spread happiness and joy to others, no matter how they celebrate. so while i quietly hope for my own merry christmas, i resist the urge to wish everybody else a happy festivus and opt for “happy holidays.” blogging the office party (mostly because they suggested it) i don’t work for central it anymore, but they still invite me to their holiday party. and no office holiday party would be complete without a yankee swap. i brought a sort of crappy battery operated screwdriver that seemed to be popular (but keep in mind that we have really low standards for these things), but i was pretty happy to unwrap a martini set with four glasses and pitcher for myself. serena collage customer sites zach got a call from the serena collage rep who rattled off this list of customers in new england: boston college northeastern bristol community college umass lowell the sungard/sct luminis content management suite demo we got the demo yesterday of sungard/sct‘s luminis content management suite (sales video). i mentioned previously that the sales rep thinks pima community college and edison college show it off well. here’s what we learned in the demo: it started with the explanation that data is stored as xml, processed by jsp, and rendered to the browser as xhtml according to templates, layouts, and “web views.” it was later explained that the product was “web server agnostic” and could run under apache, iis, sunone, or others. electric aerobic color me amused to learn that somebody (don’t worry, amazon will never tell me who) bought carmen electra’s aerobic striptease after following one of my amazon affiliate links. book flower institutional and academic repositories mit has dspace, their solution to save, share, and search the collected work of their faculty and students (in use by public sites). now royce just shared with me this presentation by bill hubbard, the sherpa project manager at university of nottingham. what’s sherpa? the name is an acronym for securing a hybrid environment for research preservation and access, but it’s a project intended to archive the pre and post publication papers and other research products. kim’s cms shortlist with , cms vendors in the marketplace, we’re mining what we know or know-of as a way to shorten the list. kim named the following four: joomla, a derivative of mambo collage appears to have good content reuse features omniupdate has a good list of higher ed clients drupal: open source and turning heads ryan eby’s pursuit of live-search ryan eby gets excited over livesearch. and who can blame him? i mention the preceding because it explains the following: two links leading to some good examples of livesearch in the wild. inquisitor is a livesearch plugin for os x’s safari web browser. it gives the top few hits, spelling suggestions where appropriate, and links to jump to other search engines. garrett murray’s maniacalrage is an interesting blog on its own, but he’s also doing some good ajax on his search interfaces. simon mahler audioproduktion simon mahler did the audio for benjamin stephan and lutz vogel‘s trusted computing movie. the movie is good, but i realized i was letting it play in the background just to hear the soundtrack, so i finally looked up mahler’s fotone.net and found the three free song downloads. it’s good stuff, but i’m wondering where the album is… cop tasers cop two cops: he wanted a soda, she didn’t. she had the wheel, he had a taser. details from this associated press story: hamtramck, mich. — a police officer has been charged with using a taser on his partner during an argument over whether they should stop for a soft drink. ronald dupuis, , was charged wednesday with assault and could face up to three months in jail if convicted. the six-year veteran was fired after the nov. they might be giants podcast thanks go to jenny for the link to the they might be giants podcast! and all that brings up something i was too lazy to figure out before. interestingly, it became an issue now only because i was also too lazy to look for the tmbg podcast in the itunes podcast directory. it turned out to be easy enough to subscribe directly, but here are the directions from apple: if you can’t find a podcast on the itunes music store, never fear. free palm/treo aim client my treo rocks. part of my love for the new gadget is how i can now aim on the run without sms. sure, i risk frostbitten fingers as i walk across campus and i’d probably be a lot better off if i just called the person, but…but… anyway, everything treo was near the top of my google query with a roundup of three commercial im apps for palm. but none of the reviewed apps seemed all that great, and i sort of expected to find a free client. two things to know about library . you don’t like the “ . ” moniker? so what. john blyberg reminds us that “if we’re arguing over semantics, we’ve been derailed.” and stephen abram is said to have cautioned us: “when librarians study something to death, we forget that death was not the original goal.” bsuite bug fixes (release b v ) [innerindex]i’ve fixed another bug in bsuite b , my multi-purpose plugin. this update is recommended for all bsuite users. fixed previous versions would throw errors at the bottom of the page when the http referrer info included search words from a recognized search engine. installation follow the directions for the bsuite b release. the download link there will always fetch the current version. upgrades from earlier versions of bsuite are easy, just replace the old bsuite. improving wordpress search results simplesearch – a full-text solution | beau collins nature concludes wikipedia not bad fresh from nature: a peer reveiw comparison of wikipedia’s science coverage against encyclopaedia britannica: one of the extraordinary stories of the internet age is that of wikipedia, a free online encyclopaedia that anyone can edit. this radical and rapidly growing publication, which includes close to million entries, is now a much-used resource. but it is also controversial: if anyone can edit entries, how do users know if wikipedia is as accurate as established sources such as encyclopaedia britannica? yahoo! rocks the web no, i don’t mean that they’re disrupting it, i mean they’re getting it. and in saying that, i don’t mean they’re figured it our first, but they they’re making some damn good acquisitions to get it right. mostly, i’m speaking of they’re purchase of flickr last year and their acquisition of del.icio.us friday. but in a somewhat lesser way i’m also speaking of their announcement monday that they’ll be offering blogs as well. yahoo! buys del.icio.us nial kennedy threw down some of the first coverage of yahoo!’s acquisition of del.icio.us last week. del.icio.us will most likely be integrated with existing yahoo! search property my web. my web allows yahoo! members to tag search results for discovery through a defined social network (y! ) or all yahoo! users. yahoo! will use del.icio.us bookmarks to better inform personalized search results throughout its services. its ability to combine signals of relevance from search result click-throughs to a listing of sites bookmarked and classified will lead to increased use of yahoo! opensearch spec updated i just received this email from the a opensearch team: we have just released opensearch . draft . we hope to declare it the final version shortly, and it is already supported by a .com. uprading from a previous version should only take a few minutes… opensearch . allows you to specify search results in html, atom, or any other format (or multiple formats) in addition to just rss. in addition, opensearch . a patron’s perspective on library . my friend joe monninger is perhaps a library’s favorite patron. he’s an avid reader who depends on his public library for books and audiobooks and dvds, and as a writer and professor he depends on the services of the university library. but he doesn’t work in libraries, and though he listens patiently to my work stories, he doesn’t really care about the politics or internal struggles we face. that said, i’m reprinting here the full text of his recent column for the valley news, a paper serving hanover new hampshire and other upper connecticut river valley communities. bush joke i wish i could admit the provenance of the following, but i’ve been sworn to secrecy. here goes: donald rumsfeld is briefing president bush: “yesterday, brazilian soldiers were killed.” “oh no!” exclaims the president, “that’s terrible!” his staff is stunned at this unprecedented display of emotion, watching as bush sits, head in hands. finally, he looks up and asks, “how many is a brazillion?” identity management podcast josh porter and alex barnett got dick hardt and kim cameron on the line to talk about identity management. the result is available as a podcast. i should add that josh and alex are big on the attention economy and social software, so they’re asking questions about how idm works in those contexts. most people thinking about idm today seem to be thinking about its uses in the enterprise or in education, but when i say identity management is the next big thing, i mean it in the social context that josh and alex are rooted in. sungard/sct luminis content management suite we’re looking at the sungard/sct luminis content management suite (sales video). the real demo comes later, but the sales rep thinks pima community college and edison college show it off well. hmm. four million dominos, a sparrow, an exterminator people like to topple dominos, and some people like to topple great long snaking lines of them. so tv crews get involved, people spend a month or more lining the damn things up, and domino day becomes an annual event. enter sparrow. sparrow menaces dominos, topples , of them. enter exterminator. exterminator shoots sparrow. enter news media. enter public outcry. enter death threats. result: a record million dominos, the sparrow incident is being investigated by a reported seven agencies, and the martyr sparrow has been preserved for display in . free fonts zone erogene has ten fonts available for free download, including migraine serif and the faux-cyrillic perestroika. tip for mac os x users: rename the font to remove the “.txt” extension that will get added to the filename, then double-click it. the dial up isp wasteland yes, there are some parts of the continental us not yet served by dsl or cable modems. that’s why i’m looking for a dial up isp. nationally we’ve got aol and earthlink, followed by budget operators netzero, peoplepc, and netscape online. but here’s the thing, and forgive my ignorance, why do all these services suggest you need to download and install software just to dial in? i mean, hasn’t dial up networking been a standard feature of various releases of mac os and windows since or so? treo for me i’ve been talking up the pepper pad and nokia a lot, and i’ve mentioned a moment of lust for the lifedrive (despite my complaints against pdas), but today i bought a treo (even though i had doubts). my decision surprised me, but the following factors all weighed in its favor: my cell phone contract expired. verizon was dangling their standard $ discount (on top of other discounts) on a new phone if i renewed. the bathroom reader somebody at gizmodo found this agence france-presse story about the intersection of american surfing and bathroom habits in the hindustan times. it’s based on a report by the usc annenberg school‘s center for the digital future. for five years running now, the center has tracked internet use (and non-use) in a , household representative sample of america (choosing a new sample each year). this year, researchers found: “over half of those who used wi-fi had used it in the bathroom. gao report confirms election fraud this should be no surprise — especially to those who’ve been appropriately concerned about electronic voting machines: lyn davis lear is reporting on a gao report that concluded the election was fraudulent and a diebold insider is blowing the whistle (via engadget). what does the report confirm? bob fitrakis & harvey wasserman summarize: some electronic voting machines “did not encrypt cast ballots or system audit logs, and it was possible to alter both without being detected. supamonks video al sent this video along via email, and it seems perfect for friday afternoon. it’s all about super-monks (supramoine in french?), a kind of european shaolin, maybe. warning label humor amadana‘s new headphones come with an amusing warning label: can’t climb wall. can’t listen to the voice in your heart. can’t open the coffer (safe). sure, the above looks fake, but lichen pointed out this other engrishism: “fits well and stable…with movable ear hangers.” want more? go visit galleries of oddness. astro dog press jon link is among the smartest and coolest people i know, so when he decides to start up a press, and then decides to fund his startup with t-shirt sales, i get in line. nokia in the wild gizmodo‘s reporting the nokia is in customer’s hands and getting some buttons pushed. now we’ve got nokia and pepper exploring this space. where to next? frontrow for everybody via an im from ryan eby: a pointer to andrew escobar‘s directions on how to install apple’s front row. digitize vinyl easy engadget and gizmodo both have the skinny on a usb turntable. microformats oliver brown introduced me to microformats a while ago, the ryan eby got excited about them, then coins-pmh showed how useful they could be for libraries, but i still haven’t done anything with them myself (other than beg peter binkley to release his coins-pmh wordpress plugin). what are microformats? garrett dimon explains the theory: when writing markup against deadlines and priorities, it’s easy to forget that somebody else will eventually have to maintain it. macos x . = built-in vnc server macminicolo.net explains how to use it. queen mashups are all the rage michael sauers pointed out q-unit, a mashup of queen and cent. they’re sure to have disney (the rights owner for queen’s catalog) on their back soon. at least, it didn’t take disney long to shut down the kleptones, whose “a night at the hip-hopera” has a spot on my ipod. and that’s where the story comes around, are we at the point where we can say queen’s music has taken on the status of a modern fairy tale? oclc report: libraries vs. search engines so, the report was released monday, and it’s actually titled perceptions of libraries and information resources ( ), but the part i’m highlighting here is the results of the question that asked users to compare their experiences with search engines against their experiences with libraries. here’s the quesiton: satisfaction with the librarian and the search engine — by total respondents based on the most recent search you conducted through [search engine used most recently],how satisfied were you in each of the following areas? all conversations in warren revolve around heat a friend of mine jokes that every conversation in warren revolves around heat. but, it wouldn’t be funny if it wasn’t at least a little bit true. as it turns out, most of the rest of the country is talking about heat too. pellet stoves have been all the rage this fall. i feel lucky to have gotten one before the rush, but i’m also a little dismayed about the selection. jabber as inter-process communication standard? open-ils blog » blog archive » opensrf jabber: a technical review oss in lib ryan eby tells me that the current issue of library hi tech includes some discussion of open source software’s uses in libraries. my cultural go-to guy most of my reading is non-fiction, so i depend on bob garlitz to keep me current with the rest of the literary world and a bit of the art world. raging arguments about the future of the ils i feel a little misrepresented by a post from talis’ richard wallis claiming you don’t need technology for library . – but it helps, but the company blog doesn’t allow embedded urls, so i’m posting my comment here: richard, please don’t misunderstand me. technology is the essential infrastructure for library . . my point was that technology alone doesn’t make a library. it would be better to read my post in the context of meredith farkas‘ and jenny levine‘s recent posts crying out for more programmers in libraries. who’s afraid of wikipedia? arguments about wikipedia‘s value and authority will rage for quite a while, but it’s interesting to see where the lines are being drawn. on the one had we’ve got a year-old pointing out errors in encyclopaedia britannica (via many many) and now on the other side we’ve got john seigenthaler, a former editorial page editor at usa today, piping mad about some libelous content in his wikipedia biography page. now, i have to agree with seigenthaler in as much as i would never want anybody to make such claims against me, and i’d probably consider my legal options in such a matter, but i’m sure i’m not the only one who gets a chuckle over the matter. understanding airport codes dave english explains why airport codes can be so darn confusing (even while some of them are stupid obvious). criticism of modern movies we’ve all heard it before, but we just can’t get it out of our heads. today’s movies make us feel dumb. paulina borsook joins the chorus and condemns contemporary cinema by praising movies of the s and s: they were movies made for adults, even if they had been mainstream movies and/or nominally rated pg. they made presumptions about the intelligence of their audience, didn’t need things to be boldly spelled out, and they were predicated on the assumption that their audience was capable of making inferences. $ laptop details i’ve been doing a lot of talking about the coming information age and how it depends on access technology that is as cheap and easy to use as our cell phones (and applications of it that are as appealing as people find their cell phones). but i’ve been slow to mention the mit media lab‘s one laptop per child $ laptop plan. the truth is that i just don’t know that much about it. humanoid robots are eerie my friend troy pointed out a while ago that the more “realistic” our -d models of humans get, the scarier they look. apparently it applies to robots to, at least judging by the “actroid” above. maybe i better put how to survive a robot uprising closer to the top of my reading list. more at akihabara news, found via gizmodo. understanding wp_rewrite and related hooks the docs are in the codex, this tag plugin offers quite a few examples, as does jerome’s keywords plugin. wp geo mashup plugin i don’t know how i missed cyberhobo‘s geo-mashup-plugin (also at wp-plugins.org) until now. it’s been ahah all this time? i might be reading this wrong, but it looks like i’ve been using ahah when i’ve thought i was using ajax. hmm… bsuite bug fixes (release b b) [innerindex]i’ve fixed a couple bugs in bsuite b , released last week. fixes a bug with search word highlighting that caused it to litter the display in some cases. a silly mistake of mine that cause a mysql error for some users. installation follow the directions for the bsuite b release. the download link there will always fetch the current version. upgrades from bsuite b are easy, just replace the old bsuite. safe: design takes on risk i’ve been sitting on this story since october, hoping i’d be able to get to the show, but it’s increasingly clear that i’m not getting to nyc for a while. so, anyway… moma is showing safe: design takes on risk wired magazine described it: just in time for the wave of catastrophes plaguing our fragile planet, some top designers unveil a series of aesthetically pleasing objects that could be handy in dangerous situations, from the banal to the apocalyptic. library . ? rochelle worries that all this library . talk is lost on her library. ross tells us why he hates the library . meme and dan reminds us it’s not about buzzwords. but michael is getting closest to a point that’s been troubling me for a while: library . isn’t about software, it’s about libraries. it’s about the evolution of all of our services to meet the needs of our users. bar hosts burglaries in years yahoo! news tells me that brigitte hoffmann’s tages-bar in berlin gets robbed a lot. edward gorey’s “elephant” house edward gorey is known for having created the gashlycrumb tinies, an alphabet of ways young children can meet an early end. that, and the bumper animations for public television’s mystery! (here, have some games). gorey is dead now, but his house in yarmouth is open to the public. admission is $ for adults (http://edwardgoreyhouse.org/, phone - - ). i found out about the house at odd new england. , cms vendors! cms market watch tells us that there are , cms vendors, and some of them are getting a little feisty. a library for all peoples in a washington post column last week, librarian of congress james h. billington proposed a library for the new world: [t]he time may be right for our country’s delegation to consider introducing to the [unesco] a proposal for the cooperative building of a world digital library. this would offer the promise of bringing people closer together by celebrating the depth and uniqueness of different cultures in a single global undertaking. bsuite features: the photo spread bsuite highlights the search words used to find blog posts in google and other search engines, and uses those search terms to recommend other related posts at your wordpress site. — – — bsuite uses the tags of one post to recommend related posts in your wordpress blog. — – — bsuite includes an easy to use statistics engine that tracks the daily hits to every post and page. opportunity knocks message from jenny levine: opportunity knocks. some people hear it, others claim it’s just squirrels on the roof. opac web services should be like amazon web services no, i’m not talking about the interface our users see in the web browser — there’s enough argument about that — i’m talking about web services, the technologies that form much of the infrastructure for web . . once upon a time, the technology that displayed a set of data, let’s say catalog records, was inextricably linked to the technology that stored that set of data. as we started to fill our data repositories, we found it usefull to import (and export) the data so that we could benefit from the work others had done and share our contributions with others. talk big if i lived in seattle, i’d look to beatnickside’s photos for clues about where the fun is. here’s his photo of the “iron composer” competition at the crocodile cafe. dance dance revolution, nyc i caught the following story on npr’s all things considered (realaudio stream) last night: new york is known for its vibrant nightlife, yet in many bars and restaurants it’s illegal to dance. now, a law professor is challenging the “cabaret laws,” claiming they violate a dancer’s right of free expression. the city says dancing by patrons is not a protected right — and can prove it. (link added) this was a big surprise to me, and a bigger surprise to learn that it’s not just some blue law. bsuite wordpress plugin (b release) [innerindex]the first thing we all have to agree on is that bsuite is the replacement for bstat. the name change reflects that fact that the plugin is doing a lot more than simply track page loads. the most exciting new feature is a module i can’t help but call bsuggestive. it uses the tags of the current post to suggest related posts to your readers. and when readers arrive at your site via a search engine, it not only highlights the search words they used, but offers a list of other posts matching their search criteria. cms pitfalls everybody wants a content management system, but there’s little agreement about what a cms is or what it should do. even knowledgeable people often find themselves struggling for an answer before giving up and defining a cms by example. the problem is that we know we want better websites, and we know technology should help, but how. jeffery veen offers some sage advice to those who would ignore the non-technical facets of the problem: theories of information behavior via librarian way i found the lis radio webcast of a conversation between sandra erdelez and karen fischer, two of three editors of theories of information behavior from asis&t and information today. unfortunately, the interview focuses on how the book came to be more than the content, but the description reads: overviews of more than conceptual frameworks for understanding how people seek, manage, share, and use information in different contexts. bsuggestive and bsuite tag support bsuite, the follow-up to bstat, now includes a module called “bsuggestive” that recommends related posts based on the current post’s tags or alternate posts based on your search words when you arrive from a recognized search engine. that is, bsuggestive does two neat things: first, visitors will see a section in each post with links to other posts on your site that have similar content. the “similarity” is judged by comparing the current posts tags against the content and titles of all other posts in the database. bsuite is coming i’m about to release a public beta of my wordpress plugin over at maisonbisson. information about my favorite new feature, bsuggestive, online now. bsuite started out as bstat, and continues to offer rich stats tracking features. update: bsuite b is out! wayfaring.com wayfaring: with wayfaring.com you can explore maps created by others, or create your own personalized map. share them with friends or the whole world. now imagine it with earthcomber integration. wouldn’t that be neat. raging arguments about the future of the ils i hadn’t seen ryan eby’s post at libdev that connected ilss with wordpress before i posted that library catalogs should be like wordpress here. it connects with a my comment on a post at meredith farkas’ information wants to be free. my comment there goes in two directions, but i’d like to focus on the technology side now. our vendors will inevitably bend to our demands and add small features here and there, but even after that, we’ll still be stuck paying enormous amounts of money for systems that remain fundamentally flawed. rollyo metasearch rollyo: roll your own search engine. create personal search engines using only the sources you trust. relevant. reliable. rollyo. they call them “searchrolls” library catalogs should be like wordpress library catalogs should be be like wordpress. that is, every entry should support comments, trackbacks, and pingbacks. every record should have a permalink. content should be tag-able. the look should be easily customizable with themes. everything should be available via rss or atom. it should be extendable with a rich plugin api. and when that fails, it would be nice if it were all written in a convenient language like php so we can hack it ourselves. infrared photos among the infrared photos at pbase.com is this plantation infrared collection by joseph levy. above: part of the collection by richard higgs. blog value the sale of weblogs inc. to aol last month for $ + million got a lot of bloggers excited. tristan louis did the math and put the sale value into perspective against the number of incoming links the the weblogs inc. properties. it’s an interesting assertion of the value of the google economy, no? the various properties have a total of almost , incoming links, which work out to being worth between about $ and $ each, depending on the actual sale price, which everybody’s mum about. karen kills in karts karen has the smart-sexy-funny thing going on, but that doesn’t stop her from eating donut after donut or beating will and me in every white-knuckled kart race we ran last weekend. drivers sit only an inch or two off the ground in karts that are said to go miles an hour. eight minute races may seem short, but at between and seconds per lap (my best time was -some-odd seconds, karen’s was at least a second faster), you’ll get plenty of chances to skid out at every turn. thanksgiving there is, supposedly, some historical meaning to our thanksgiving holiday, but all i can figure out is that i wasn’t there and it probably didn’t go as i’ve been told. thing is, thanksgiving isn’t so much about what we were, but who we are. thanksgiving celebrates the two most important things in life: food and family. almost unique among us holidays, retailers haven’t yet found a way to commercialize it. international readers may wonder how a us holiday can exist without commercial involvement, but they should know that we make up for it in the way we eat. my wife the technology dependent anti-geek my wife sandee cringes at the suggestion that she’s a geek. she writes poetry and teaches english, she cooks fabulous meals and dances all night long. surely you’re mistaken she’ll say. but she does have a laptop, a digital camera, and an ipod. and she immediately saw the value of having a computer in the living room when mp s replaced cds many years ago. so you’ll point to all of this and ask for a clarification and she’ll explain that her use of technology does not make her a technophile any more than her use of a car makes her a nascar fan. pew internet report: search engines gain ground according to the recently released pew internet report on online activities: on an average day, about million american adults use the internet; % will use email, % will use a search engine. among all the online activities tracked, including chatting and iming, reading blogs or news, banking, and buying, not one of them includes searching a library opac. november snow we’ve had snow on the mountains for a while now, but this is the first accumulation in my yard. when you hit bottom and need design help stock.xchng has nothing on flickr for searching, finding, sharing photos, except that they’re uploaded with the express intention of offering them for re-use. some are available free, others free for non-commercial use, others with their own license terms. but stock photos aren’t really the bottom of the barrel. no, for that you have to look at pixellogo. it’s there that you’ll see the sorts of things you can do to put some pop in a limp design. using xml in php everybody likes documentation. the zend folks posted this overview and simplexml introduction the o’reilly folks at onlamp offered this guide to using simplexml. of course, there’s always the simplexml docs at php.net. two problems: i haven’t encountered cdata in my xml yet, but i do hope to develop a better solution than offered here when i do. the other is that simplexml chokes on illegal characters, a unfortunately common occurrence in documents coming from iii’s xml server. akismet spam catcher i’ve been getting spam, a lot of spam; spam comments and trackbacks in the last two months or so. so it was a relief to find akismet, a networked spam blocking plugin for wordpress. they claim to have blocked , spams since its release, and i’ve been pretty happy with it. instant messaging in libraries: ten points from aaron schmidt aaron schmidt’s points about im in libraries include: instant messaging is free (minus staff time) millions of our patrons use im every day. for some, not being available via im is like not having a telephone number. there are three major im networks (aim, y!m, msn) y!m and msn will be interoperable at some point. trillian is a multi-network im client, meebo is a web-based multi-network client. use them. retro gaming for the holidays it’s amusing how retailers will try to capture a trend. so retro gaming fans have been building their own arcade cabinets for years now, but i just saw that target is offering a midway arcade machine for the holidays. the -pound machine is described as “full-size” and offers joust, defender i and ii, robotron, rampage, splat, satan’s hollow, root beer tapper, bubbles, wizard of war, timber and sinistar. thermometer museum dick porter, of onset ma, has been building his collection of over thermometers since the mid- s, though the collection has nearly doubled since when it was just over . he calls it the world’s largest and only thermometer museum. he’s certainly passionate about them, and he’s been an invited speaker at more than a few thermometer and weather related events, like the christening of the world’s largest thermometer in baker california. harmon’s lunch i learned of harmon’s lunch from a mention on the splendid table a few weeks ago. i wrote down the following quote from the show from memory, so it may not be entirely accurate: they have two things on the menu, and nobody ever orders the other one. they serve hamburgers, and the only option is with or without onions. as it turns out, the menu is a little richer than suggested. collective intelligence: wisdom of the crowds i’m here at neasis&t’s “social software, libraries, and the communities that (could) sustain them” event, presented by steven cohen. he’s suggesting we read james surowiecki’s the wisdom of crowds. surowiecki first developed his ideas for wisdom of crowds in his “financial page” column of the new yorker. many critics found his premise to be an interesting twist on the long held notion that americans generally question the masses and eschew groupthink. more neasis&t buy hack or build followup first, josh porter, the first speaker of the day has a blog where he’s posted his presentation notes and some key points. josh spoke about web . , and ended with the conclusion that successful online technologies are those that best model user behavior. “i think web . is about modeling something that already exists in our offline worlds, mostly in the spoken words and minds of humankind.” interestingly, in findability terms, it was josh’s post that clued me in that the event podcast was online because he linked to my blog in his post. nelinet bibliographic services conference i’m here at the nelinet bibliographic services conference at the college of the holy cross today. the conference is titled “google vs. the opac: the challenge is on!” and there’s quite a lineup of speakers. my presentation is on “the social life of metadata.” my slides are online, and below is some background. **the library catalog… ** the catalog is among a library’s most important assets. an unread book offers little value, but the catalog offers the promise that the library’s resources will be found and used, and a well constructed catalog makes the finding easier by offering rich details and easy navigation. neasis&t buy, hack or build followup i was tempted to speak without slides yesterday, and i must offer my apologies to anybody trying to read them now, as i’m not sure how the slides make sense without the context of my speech. on that point, it’s worth knowing that lichen did an outstanding job liveblogging the event, despite struggling with a blown tire earlier that morning. it’s probably well understood by anybody reading this that most library services are at the web . neasis&t buy, hack or build i’m here at the neasis&t buy, hack or build event today at mit’s media lab. on the list are joshua porter, director of web development for user interface engineering, pete bell [corrected], co-founder of endeca solutions, and me. i’m posting my slides here now, but i’m told we’ll see a podcast of the proceedings soon after the conclusion. be aware that the slides are full of links. i won’t be able to explore them all during the presentation, but they might add value later. zimbra rocks zach made me take another look at zimbra, the web-based, web . -smart, very social and ajaxed up collaboration, email, and calendar suite (plus some other goodies). go ahead, watch the flash-based demo or kick the tires with their hosted demo. i think you’ll agree that it looks better than anything else we’ve seen yet. part of the success of the project is that the developers appear to understand the problem. here’s the list of [how broken email is][ ] from the white paper: ars on video ipod it’s old news now, but arstechnica did a really thorough review of the video ipod. i especially appreciated reviewer clint ecker’s opinion of the video playback capabilities. now i’m curious about what this does to enable more video podcasts. virtual economies i’m not much of a gamer, but matt got me following video game law with curious interest. and now, via arstechnica, i’ve learned of crazy things going on in role playing game economies. to some, the only surprise in jon jacobs’s us$ , purchase of in-game real estate is that nobody thought of it sooner. the first thing to know is that unlike most other mmorpgs, project entropia mixes its virtual economy with the real world. second annual west texas beautiful burro and mule contest held today the text of what appears to be the press release (online at alpine avalanche): the fort davis merchants association and the jeff davis county -h club encourage everyone to come join the fun as they host the second annual west texas beautiful burro and mule contest saturday, nov. . the contest will be held on the west side of the jeff davis county courthouse, and begins at a. slot car camera i got a slot car set for christmas when i was about eight years old. i ran the cars until the contact pads wore out, then i pretty much gave up on them. but simon jansen is just getting into the action, and he’s doing it at a time when compact and cheap electronics afford (potentially) more interactivity. see, jansen taped his cellie on one of his cars and started recording the action with the built-in camera. wolfram’s tones wolframtones mixes hard science with social software in the form of a ringtone generator. each click on any of the style buttons yields a “unique [note: not random] composition.” why not random? the faqs note: once wolfram_tones_ has picked a rule to use, all the notes it will generate are in principle determined. but that doesn’t mean there’s an easy way to predict them; in fact, wolfram’s phenomenon of computational irreducibility shows that in general there can’t be. tech tuesdays: spam management john martin was kind enough to lead a session on spam management tuesday (november th). here was the description: spam is annoying and often offensive, but it’s a fact of life for all of us. john martin will lead a discussion about how we can limit the amount of spam we see using tools running on our campus mail server and in outlook. he’ll also discuss what we can do to keep our email addresses out of spam lists in the first place and spam related issues such as phishing. six weapons of influence ken forwarded me this podcast of robert cialdini speaking on his six weapons of influence, which he lists as reciprocation commitment and consistency social proof authority liking scarcity cialdini’s book is in its fourth edition, and has apparently been adopted as a text for more than a few classes and the concepts have worked their way into everybody’s marketing seminars. motivation speaker and marketing yakyak patricia fripp summarizes those six weapons like this: library integration stuff i’d meant to point out these two articles from library journal ages ago, but now that i’m putting together my presentations for next week (neasis&t & nelinet), i realized i hadn’t. roy tennant writes in doing data differently that “our rich collections of metadata are underused.” while roland dietz & carl grant, in the same issue, bemoan the dis-integrated world of library systems. how to survive a robot uprising so there i am trying to read things i can’t possible read and i stumble across a link to daniel h. wilson’s how to survive a robot uprising: tips on defending yourself against the coming rebellion. from th amazon book description: how do you spot a robot mimicking a human? how do you recognize and then deactivate a rebel servant robot? how do you escape a murderous “smart” house, or evade a swarm of marauding robotic flies? digital library systems group shows wares i was in cambridge today attending the digital library systems group presentation on their fancy scanners and imaging workflow software. we have no digital collections program going yet, but we’re part of a university system plan to acquire either ex libris’s digitool or encompass for digital collections (sample sites). but getting the collection management software just creates another problem: we don’t have any imaging resources to use to fill the new digital archive. ikea comes to new england hey, doesn’t the ikea near boston open today? sure does. the company has stores worldwide. according to a story in the pheonix: oddly enough, ikea flopped when it opened its first us store in . but by making concessions to american expectations (softer couches, american bed sizes, good thread counts) it gradually won over low-budget consumers attracted to its upmarket design, with its subtle implications of class mobility. that they were willing to bruise their toes lifting those deceptively heavy boxes speaks to the brand’s participatory appeal […]. internet, interactivity, & youth jenny levine alerted me to the pew internet & american life project report on teens as both content creators and consumers. it turns out that teens, and teen girls especially, are highly active online iming, sharing photos, blogging, reading and commenting on other’s blogs, and gaming. an especially strong trend in this group is the use of web technologies for collaboration. interactivity, increasingly, is being defined by the teen’s ability to ask questions, comment, or contribute. reva “electricity car” how crazy is it that we can get neither flying cars nor (affordable) fuel efficient cars today? anyway, the reva (shown above) is a tiny little electric that seats two adults, can go miles on a charge, and fully charges in five hours (two hours gets an % charge). it’s an indian company, but they export to europe and the website has some mention of test-marketing the cars in the us. pen-based computing loses the tablet via engadget i found mention of the leapfrog fly, a pen with embedded computer that reads your handwriting. need a calculator? just write out “ + = ” and hear a response from the pen computer’s synthesized voice. need to schedule something? write out the date. it’s targeted at kids, and the company has released it with a variety of tutoring applications and games (you guessed it: flyware) appropriate for kids in rd to th grade. this car climbed hubbert peak this car climbed hubbert peak bumper stickers from hubbertpeak.net. devil’s horn on npr’s weekend edition today: an interview with michael segel, author of the devil’s horn, subtitled “the story of the saxophone, from noisy novelty to king of cool.” adolph sax’s instrument seems to have been controversial from the start. other manufacturers tried to assassinate him, the pope declared the church’s opposition to the instrument, ladies home journal explained that it “rendered listeners unable to distinguish right and wrong.” i get love letters (about bill bennett’s racist remarks) “john b,” from omaha, ne writes regarding my post about conservatives, freakonomics, and bill bennett’s racism: [i]f you had actually listened when bill bennett made the comment you quote, you would see it was not intentionally racist. you’ve taken the quote completely out of context. i’m willing to bet that you know you’ve taken the quote out of context, but really don’t care. you’ll do anything to make anyone conservative or republican look bad. the codex series this, from chris anderson: the codex is a episode series of machinimas made on xboxes running halo . the result caught the attention of his six- and eight-year-old children, and then him. machinimas are computer animated in real-time, using video games to create the environment, and human “puppeteers” to drive the action. the action is capture, edited, and voice-overs added. because they remove many of the economic and technical barriers to film production, they hold the promise of emphasizing story and plot, and exposing talent among those who create them. gnarly trees gnarly trees: “this group is for trees with oddly-formed limbs, strange bulges or growths, braided roots, or otherwise abnormal looking parts.” this car climbed hubbert peak this is probably the perfect bumper sticker for your neighbor’s suv, at least until your neighbor comes over with the perfect chainsaw for your front door (yeah, try to run from that in birkenstocks). but seriously, shouldn’t somebody tell these people that the world is running out of oil? venkman javascript debugger how did i miss this before? the venkman javascript debugger; available here, with user’s guide and faq. ostankino tower & world federation of great towers i don’t remember exactly why i found myself looking up moscow‘s ostankino tower, a ft ( m) tall radio-television tower. compared to the world’s tallest buildings, it’s taller than all the greats: the taipei , the sears tower, empire state building, though some people keep towers — even those with observation platforms — in a category separate from skyscrapers. so what’s a tower enthusiast to do? go take a look at the world federation of great towers (also at wikipedia). decision death spiral scott berkun, the author of the art of project management just blogged about the data death spiral: whenever data is misused as the only means for making decisions, a death spiral begins. the lust for data overwhelms all sensibilities. cowardly decision makers howl in glee at reams of unnecessary data, while bright people sit handcuffed to ugly slidedecks and mediocre ideas. decision makers forget their brains and wait for numbers, fueling an organizational addiction to unnecessary and distracting data. the livermore centennial bulb treehugger alerted me to the rather surprising story of this light bulb, burning continuously since . yeah, at least that’s the story here, at the centennial light bulb committee’s website (a partnership of the livermore-pleasanton fire department, livermore heritage guild, lawrence livermore national laboratories, and sandia national laboratories). the bulb is said to have been made by the shelby electric company of shelby, ohio, and given to the fire department by dennis bernal, owner the livermore power and light co. russian navy likes it big (and heavy) maybe the meaning is simply lost in translation, but take a look at the captions for this photo essay of the russian navy titled “baltops military exercise: russia is showing its muscles.” here, have two big ships, some big anti-aircraft ships, a big landing ship, a big anti-submarine ship, even a big atomic missile cruiser, and add this heavy atomic cruiser. now how would you feel about captaining the one small landing ship? what’s in a web search? sometimes the answer isn’t as interesting as the question. consider this note from yahoo buzz: on sunday, the day before the nomination became official, [searches for] alito sprang up a sudden %. did searches for alito spike on tips white house staffers, or were white house staffers vetting their nominee via the search engines? seattle via the programmableweb: seattle .com. it’s another mashup with google maps, but who knew anybody could get data in real time? sure, it’s only for seattle, and only their fire/ems servers (no police), but technology wise, it’s cool. kudos to seattle, i guess. what’s my reticence? i don’t know if i should have this data…and putting it together like this hits my privacy funny bone a bit. but then, this data exists…it’s a matter of public record. uc irvine’s hiperwall putting together ″ apple cinema hd displays with power mac g s gets you million pixels of screen real estate spread over x feet. call it uc irvine’s hiperwall. paper house a visit to the paper house will run $ . and takes you out to a beautiful corner of the massachusetts coast, pigeon hill street rockport, ma , just up the hill from pigeon cove. call ( ) - if you’ve got questions. more info at odd new england. pictures tell quite a story, so take a look at the photoset showing details of the fireplace, curtains, and exterior walls. missiles explode in south korea one or more trucks carrying disassembled nike-hercules missiles exploded in a tunnel near the cities of taegu and masan in south korea today. reuters reports no deaths, the korea times criticizes lack of safety. the new imacs… i live quite a distance from any apple stores, so it’s only now that i’ve been able to see the new stuff. the photo booth application bundled with the new imacs is actually more fun than i expected. that’s me above with the “comic book” effect applied. but front row is every bit as sweet as it looks in the demos. yes, i want it on my current machine. and, yes, i would pay $ , or maybe $ , i might even be convinced to pay $ for the remote and software. i will crush you or, er, my server will be crushed. i guess i should admit that my stuff could do with some optimization, maybe. perhaps what i really need is something faster than celeron with mb ram. maybe. is search rank group-think? way back in april , jakob nielsen tried to educate us on zipf distributions and the power law, and their relationship to the web. this is where discussions of the chris anderson’s long tail start, but the emphasis is on the whole picture, not just the many economic opportunities at the end of the tail. here’s how it works with hits to websites: a few sites become popular and form the “big head” at the left a few more sites form the slope a huge number of websites score very low and form the “long tail” nielsen adds these examples: + ways good html can go bad via brad neuberg: rsnake’s xss (cross site scripting) cheatsheet: esp: for filter evasion. limitations on cross site scripting (xss hereafter) have been troubling me as i try to write enhancements to our library catalog, but the reasons for the prohibition are sound. without them i could snort your browser cookies (rsnake lists: “cookie/credential stealing/replay/session riding” among the threats, but a well-planned attack could also fetch resources from internal webservers and deliver them to external data thieves). ipod linux tutorial how to install ipod linux on & g mini, g, photo attack of the blogs (yeah)! online reaction to the forbes cover story attack of the blogs has been quick and strong, and given the doom and gloom language, it’s not surprising: blogs started a few years ago as a simple way for people to keep online diaries. suddenly they are the ultimate vehicle for brand-bashing, personal attacks, political extremism and smear campaigns. it’s not easy to fight back: often a bashing victim can’t even figure out who his attacker is. swarmsketch via information nation, i found swarmsketch. here’s the description: swarmsketch: collective sketching of the collective consciousness. swarmsketch is an ongoing online canvas that explores the possibilities of distributed design by the masses. each week it randomly chooses a popular search term which becomes the sketch subject for the week. in this way, the collective is sketching what the collective thought was important each week. (due to increased traffic sketches are currently being updated after about lines) learn japanese online tutoring in japanese at udanstraight.com. here, have some trial lessons. new social web apps ross mayfield’s new social software list discusses ning, flock, wink, memeorandum, sphere, and rollyo. the fight over massport wifi i do a lot of flying in and out of boston’s logan airport, so i’ve been following the controversy about wifi there with some interest. the story is that massport, the government agency that runs the airport, is trying to tell tennents — like the airlines — that they can’t operate their own wifi networks. but the fcc previously ruled that landowners had no authority can control use of the wifi spectrum on their premises. public broadcasting sms to construction sign (at engadget and textually), and sms to megaphone — for the armchair protester (at textually and engadget). gen h- personal helicopter it’s nearing the end of and we still don’t have any flying cars like we were promised, but the gen h- personal helicopter looks promising (and dangerous). here it is in the air, and i might be crazy, but it looks to be controlled by weight-shift (even more photos). ohgizmo says it sells for about $ , . gizmodo claims it drives its counter-rotating rotors with an eight-horsepower, cc engine. and odd things from japan wonders if “this is the nearest thing on earth to ‘takekoputa. goats show i can’t really pass as an undergrad anymore, but they still let me in to friday night rock to see the mountain goats. mp s: this year commandante going to bridlington homer simpson nuclear safety simulator here: have at it with a swedish nuclear power plant simulator. raise and lower the control rods, turn pumps on and off, open and close valves, just make sure you don’t blowup anything. go look at the chernobyl tour to see what happens when you mess up. the original page includes this context: the control-room operators of the kärnobyl nuclear power plant are telecommuting and are running the plant through the web. minutes of attention i won’t link to the new york times anymore, but when ross mayfield quotes them, i don’t have to. the story is that life is full of interruptions. the typical office environment today apparently allows workers “only minutes on any given project before being interrupted and whisked off to do something else.” worse, “each -minute project was itself fragmented into even shorter three-minute tasks, like answering e-mail messages, reading a web page or working on a spreadsheet. ubicomp goes spray-on via gizmodo, we make money not art, and the engineer: spray-on computers. the idea is to develop computers about the size of a grain of sand (though they say a cubic millimeter here), give them sensors and networking capabilities, and completely change our notion of “computer.” from the engineer: each speck will be autonomous, with its own captive, renewable energy source. thousands of specks, scattered or sprayed on a person or surfaces, will collaborate in programmable computational networks called specknets. dick hardt ‘s identity . presentation i said “identity management is the next big thing” back in september. that was before i’d seen sxip founder dick hardt’s presentation on identity . . zach peeped me the link and told me i wouldn’t regret watching the presentation. he was right. everybody, especially the people who don’t yet care about identity management, should take a look. the language of your website lynne puckett on the web lib list pointed me to web pages that suck and highlighted this quote from the site: nobody cares about you or your site. really. what visitors care about is getting their problems solved. most people visit a web site to solve one or more of the following three problems. they want/need information they want/need to make a purchase / donation. they want/need to be entertained. what are blogs? tech tuesdays: blogs and blogging tech tuesdays: blogs and blogging note: these are my presentation notes for a brown bag discussion with library faculty and university it staff today. this may become a series…[[pageindex]] more: my presentation slides and the daily show video. introduction public awareness of blogs seems to begin during the years of campaigning leading up to the election, but many people credit bloggers for swaying news coverage of senator trent lott‘s comments at senator strom thurmond‘s th birthday celebration in december . mike walter’s mellotron before gadgeteers could get affordable (or any) electronics for polyphonic sound synthesis or sample playback, they dallied with tape playback devices that would link each key to its own tape mechanism that played a pre-recorded tape loop at the keyed pitch. they called it a mellotron, and yes, an -key piano would require tape mechanisms. mike walters’ home-made melloman uses walkman-style cassette players wired to a two-octave keyboard in that snazzy-cool case. flock out the flock preview is out and i love it. the good folks at wordpress.com are saying “it’s like firefox with goodies.” i’m saying it’s a browser built for web . . somebody somewhere is starting the gamer’s rights movement annalee newitz tells me that video game developers are looking for cheaters by installing spyware with their games. blizzard, developer of world of warcraft, starcraft, and diablo is among the biggest names doing this. greg hoglund, quoted at copyfight, notes: i watched the [software] warden sniff down the email addresses of people i was communicating with on msn, the url of several websites that i had open at the time, and the names of all my running programs, including those that were minimized or in the toolbar. engadget caption contest caption contest: what large honkers you have! genuine fractals resolution on demand onone software‘s genuine fractals putting your video on a new ipod [how-to: automatically download and convert tv for your ipod – hackaday.com ] understanding web . ross mayfield says web . is “made of people.” tim o’reilly tells us it’s about participation. and to marc canter, it’s the connectivity. more to come… mmm. spelunking in sewers international urban glow – europe underground mt. moriah: summit denied will and i didn’t summit mt. moriah yesterday. we’d started late and the weather was turning against us, but i did get this shot of mt. washington and the presidential range. email . from ross mayfield in many many: this email is: [ ] bloggable [x] ask first [ ] private whale watching on lake michigan? whale watching on lake michigan? false: whale watching on lake michigan way back in , classroomhelp.com published a story on whale watching in lake michigan. as it turns out, the info was based on content on a geocities.com member page that suggests they book trips to see and swim with marine fauna in the great lakes. unfortunately, classroomhelp.com later posted a retraction saying “we thought it was true …it looked so real. it looked like a legitimate web site.” jim wenzloff notes web pages that suck “web pages that suck: learn usability and good web design by looking at bad web design.” where are the mit weblog survey results? where are the mit weblog survey results? they were supposed to be out september first, but they’re still missing… all i can find is this older page from fernanda viegas. bad covers: oops! i did it again memepool.com points out that the folks at supermasterpiece are claiming priority over britney spears’ oops ! i did it again. their story is: “oops ! i did it again” was recorded in april, in a chicago studio, most likely nearlie’s or west and fourth. cut for the decca label by louis armstrong and elemends of zilner randolph’s touring group, “oops!” failed to make the chart impact of “all of me,” another side recorded in the same session, and soon fell out of print. now search lamson library at a .com a , the search engine from amazon.com, does some pretty interesting things that libraries should be aware of. first, any library considering a metasearch product should look at what can be done for free, and second, libraries should take a look at the opensearch technology that drives it. so now, when searching for harry potter, you’ll also find relevant results from plymouth state university‘s lamson library. we’re not the first library — i think seattle public was — and my work mostly follows the cookbook written up by ryan eby, of michigan state university libraries. camera tossing memepool introduced me to camera tossing at flickr, where there’s even a group for those who are willing to risk their camera for a chance at a shot of streaky lights. but not everybody tosses in the dark, it’s turned out to be a a new fad in self-portraiture. click through for credits and more info on the photos above. php + xml = love the zend overview of the new xml features in php has re-energized me for building xml server applications at my library. hello wordpress.com! cliff invited me to wordpress.com earlier this week and i’ve just gotten a chance to get things up and running over there. i’m planning (though plans are never certain) to move my link blogging (think “blinks”) over there and (perhaps) re-publish them here in some aggregated form. we’ll see how that works out over time. dan grossman’s list of top ten ajax apps top ajax applications at a venture forth. fuel efficient vehicles people looking for oversized pickups, ridiculously large russian army trucks, even jet powered speedsters have it easy. but what about people who have some understanding of the hubbert peak and don’t want suvs? + mpg cars have been available in japan for years now, and can be bought used in canada for under$ . but us law forbids importing them to the us! heck, the smart, the super-efficient line from daimlerchrysler, has been available in europe (and now canada) for about ten years now, but it too can only be imported with a lot of restrictions. affordable fuel efficient vehicles (not in the us) i’m a fan of the smart, the fuel efficient european roadster that’s smaller than a mini (see above). it’s coming to america, but indirectly and not without some complexity. oddly, considering the current energy crisis and that buyers appear to be looking for more efficient vehicles now, there’s a lot of red tape involved with bringing efficient vehicles new or old to the us. take these japanese k-cars that get around miles per gallon and can be imported and bought used for under $ , but only in canada. manhattan user’s guide manhattan user’s guide caught my attention when i followed a link to their hump day list of funnies. social geography: common census commoncensus map project: the commoncensus map project is redrawing the map of the united states based on your voting, to show how the country is organized culturally, as opposed to traditional political boundaries. it shows how the country is divided into ‘spheres of influence’ between different cities at the national, regional, and local levels. movie night: save the green planet i’m at a loss for words of my own to describe save the green planet (imdb page), so i’ll have to crib from others. amazon’s description: a sensitive, blue collar sad sack hopped up on conspiracy theories and sci-fi is convinced that aliens have infiltrated human society and are planning to destroy the planet at the next lunar eclipse. he sets out to kidnap his boss to torture him until he confesses to his alien identity and stops the invasion. the conservatives vs. freakonomics conservatives hate freakonomics, that book by economist steven d. levitt and journalist stephen j. dubner that takes on more than a few sticky issues that most people don’t normally consider to be within the purview of economics. (see also the freakonomics blog). publisher’s weekly notes: there isn’t really a grand theory of everything here, except perhaps the suggestion that self-styled experts have a vested interest in promoting conventional wisdom even when it’s wrong. weird travel it started with the plastics museum and museum of bad art, progressed with a visit to the international bowling museum and hall of fame and continued with a tour of donut shops in lowell, ma. now i can report that the maisonbisson weird travel archives include the thermometer museum, the edward gorey house, and the paper house. click the links to see photosets at flickr, and watch maisonbisson for full reports later. cubesat kickstarts new space race cubesat is cal poly’s plan to make space accessible to the rest of us. that is, they want to make it easy and cheap enough to launch satellites that even high schools can get a chance at it. engadget says they call it “the apple ii of space exploration” (link added). here, read this: the cubesat project is a international collaboration of over universities, high schools, and private firms developing picosatellites containing scientific, private, and government payloads. group portrait at pigeon cove an unconventional panorama in rockport‘s pigeon cove. from left to right stand will and corey. of course, it looks better bigger. note: this was just a sideshow on our weird travel tour. the jumping in rockport it was raining today in rockport, but that didn’t stop corey (top) or will (bottom) from doing a little jumping on the seawall. note: this was just a sideshow on our weird travel tour. getting a passport my old passport is expired and my wife has never had a passport, so i had to look this up. fortunately, the us state department has a pretty good website for it. there are rules of course, especially for first-timers or expired passport holders. you’ll have to fill out a ds application form and bring to one of the facilities — mostly post offices — around the country. a [photographer’s guide][ ] is worth looking at for those considering taking their own photos, as the state department cares greatly for the [lighting][ ], [composition][ ], and [quality][ ] of those photos. balloon museum i was browsing the npr archives the other day and found this report on the international balloon museum in albuquerque, n.m.. of course i want to go there. pepper pad as multipurpose voip device i’m quite taken with my new bluetooth headset, despite the little hiccup i encountered. so, naturally, i’m thinking about how it would work with the voip softphone that’s promised for the pepper pad soon. i’ve become a super-fan of gizmo project on my powerbook, but that loaner pepper pad was a capable enough and more than portable enough machine that it has me wondering if i’d rather have a desktop mac and a pepper pad when upgrade times comes. monkey business if that proverbial room full of monkeys at typewriters ever really did randomly pound out the complete works of shakespeare, would they be as good? what if they randomly pounded out something better? james torio’s blogging thesis james torio has been working on his masters in marketing and took a strong look at blogs for his thesis. i looked at how blogs have impacted business and communication, how some blogs create revenue, how some companies are using blogs, how blogs greatly boost the spread of information, how blogs add richness to the media landscape, how blogs work in the long tail, how some companies are tracking the blogosphere and what the future of blogging may be. pravda and mccarthyism don’t worry. i’m right on top of whatever happens in pravda, the leading newspaper of the russian federation. or, at least, i’m right on top of whatever they report in their english language version. the thing that had me choking on my onion and boursin cheese bagel this morning was the story headlined fbi arrests another spy in the white house, ‘prevents’ philippine revolution. the whole philippine thing is entertaining and laughable on its own, but further down in the story the reader will find so many layers of irony and amusement as to spray their breakfast cereal about the room. findability, the google economy, and libraries peter morville, author of ambient findability, stirred up the web lib email list with a message about authority and findability. his message is about how services like wikipedia and google are changing our global information architecture and the meaning of “authority.” the reaction was quick, and largely critical, but good argument tests our thinking and weeds the gardens of our mind. argument is good. here’s my side. it’s important that we understand how modern search engines work. what bloggers need to know about cahill v. doe wendy seltzer alerts us to the delaware supreme court’s ruling last week in cahill v. doe, a case that tested our rights to anonymity online, as well as the standard for judging defamation. as it turns out, the court decided against the plaintiff, a city councilman, and protected the identity of “proud citizen,” who the councilman accused of posting defamatory remarks in an online forum. further, it also decided that the context of the remarks “a chatroom filled with invective and personal opinion” are “not a source of facts or data upon which a reasonable person would rely. bluetooth headset problems i’m still excited about that bluetooth headset i got last week, but i did encounter a little problem with it. rather, i encountered a problem with mac os x and the bluetooth headset. i don’t remember all the precipitating details, but the obvious threshold event was when gizmo project complained that it couldn’t find the headset. i tried deleting the configuration and re-pairing, but aside from some momentary linkages, it was all for nada. fried ravioli of course i like my new camera. if you don’t think these fried ravioli have enough detail, take a look at the full-size version ( x ). priorities so long as i’m talking about change i want to bring attention to some commentaries by chris farrell in marketplace money. on september th he noted that hurricane katrina (rita hadn’t hit yet) “ripped the veil off poverty in america” and wondered aloud weather the voting public would continue to support the republican obsession with tax breaks in the face of this new empathy for those struggling to hold on to the bottom rung of that same economic ladder. changethis worth looking at: changethis, started by seth godin and “a sharp team of change agents.” the quote comes from ben mcconnell at church of the customer, who also reminds us of the ways that conservatives in every field favor traditional views and values and oppose change: stay the course don’t fix what isn’t broken ignore all critics we don’t have time keep out anything foreign to us (actual or metaphorical) destroy anyone who opposes us or our way of thinking who cares that godin and mcconnell are marketers. …and the floods moved north the rains this weekend swelled the rivers to flood stage in south-western new hampshire. as much as half of keene is said to be under water. further north, the small and historic downtown of alstead has been washed away. this picture comes from the portsmouth herald, and reports in the washington post from keene and alstead add detail. the current death count is five, according to nhpr news, and nh governor john lynch has declared a state of emergency and activated the national guard. switched servers i switched to lunarpages last week after the fiasco with my old hosting provider. now, because of bandwidth and cpu usage, i’m moving to a new server at lunarpages. i wasn’t surprised about what they said when i got a message from the sysadmins about excessive cpu usage on my shared hosting account, but i was surprised with their proactive and customer friendly approach. anyway, i’ll be figuring out my new server and control panel (it’s plesk, and i’d been using cpanel for a while). bluetooth headset as i was contemplating making angry calls to my hosting provider last week when they shut down maisonbisson for a couple days, it occurred to me that i would rather make those calls via skypeout or some similar service that didn’t reveal my home phone number. after all, i wouldn’t want an angry sysop to take revenge by having a spare modem call me up every minutes between the hours of midnight and seven am. ear shrapnel noise grenade engadget calls it “skull-shattering fun” and gizmodo labeled it “ear shrapnel.” it’s available at paladone.com and boy’s stuff, though nobody seems to have yet found a domestic supplier. from the catalog page: the sonic grenade features three different levels of the most noxious sound since the last westlife album. to launch, pull the pin and throw it towards your target. after seconds, the sonic explosion occurs, giving even the deepest sleeper a wake-up call like they’ve never had before. library feel-good a flash animation about why libraries matter. rules for writing bad poetry tips from a friend: center justify the text and write things like “kill me daddy, the robins chirped.” compact, modular, and lego-like housing compact, modular, and lego-like housing is nothing new. buckminster fuller‘s dymaxion house (now at the henry ford museum), designed in the s, was probably the first. but the lustron house was actually sold commercially in the years after world war two. though it didn’t turn out to be a commercial success, the house did show the promise of pre-fabrication and mass-manufacture for house. they even have have an enduring fan base, with websites like the lustron connection and lustron luxury, and a documentary. cladonia exchanger xml editor interesting: cladonia exchanger xml editor, a java-based app that makes reading raw xml easy. much easier than in a regular text editor, even with syntax highlighting. stone face fables note: the following comes without attribution from an acquaintance of my father’s. once upon a time there were people who lived in a valley near a mountain. on the mountain there appeared a large rock formation which resembled a face. you could almost see the nose and eyes and mouth. some people claimed that it was the face of a god and they claimed that if you looked closely you would see that for yourself and once you did you would be able to live a happy and comfortable life. bye bye pepper pad my week with the pepper pad is over, and the ups van just drove off with it, but i’ve still got a lot to report. my testing ran into problems when it turned out that the wifi network in the library was on the fritz. i did some netstumbling today and found that only two aps were broadcasting at anything close to full-power and all the others were whispering like they were gonna get shushed by an old-time librarian. who knew transit maps were copyrighted? the mta, the folks who run new york’s subways and busses and such, weren’t the only ones to smack a cease and desist down on ipod subway maps last week, but they’re the first to tell they can pay $ for the privilege of distributing those maps in an ipod-readable format — but only for non-commercial distribution. cluetrain moment: doesn’t the mta understand that services like this serve potential tourists like me? five days left to apply to be chivas life editor chivas, the folks who bring us chivas regal scotch whisky and virtual tours of the playboy mansion, is looking for a pair of ambassador editors for thisisthelife.com. the deal pays $ , to the lucky pair to tour the world making good press and pictures for the brand. you’ve got six more days to put together the three-minute application video, so get on it. thanks to gadling for the link. library-related geekery ryan beat me to reporting on the interesting new services at the ockham network (noted in this web lib post). the easiest one to grok is this spelling service, but there are others that are cooler. he also alerted me to a perl script to proxy z . to rss. though for those more into php (like me), i’d like to point out the yaz extension from the folks at index data. distracted by my shiny new camera the olympus c , one of the best digital cameras ever, can be had for under $ , refurbished, from some sellers on amazon. that’s about where the price/features ratio against the c i was excited about last week tips strongly in favor of the c . i might get into why i’m not excited about dslrs in a later post, but i won’t deny that price is part of it. still, i think even the most die-hard dslr aficionado will agree the c has a lot to love. open content alliance the news is that yahoo! announced they’ve formed the open content alliance. though that certainly fits the google versus yahoo! story that newsmen want to report on now, it’s somewhat disingenuous to the internet archive, which has been beating the open content drum for a while. but brewster kahle, the founder of the internet archive doesn’t seem to care. he was talking about it on the yahoo! search blog yesterday: mac wireless card compatibility in case you’re looking: metaphyzx’s mac os wireless adapter compatibility list. introducing bsuite_speedcache i wrote bsuite_speedcache to reduce the number of database queries i was executing per page load. by implementing it on some of the content in my sidebar, i dropped queries for each cache hit. that might not seem like much, but it should average about queries per minute that that my host server won’t need to process. now that i’m looking seriously at optimizing my queries, i’ve also cut the monthly archives links from the sidebar. meltdown sometime around pm friday the mysql server at my hosting provider took a walk. the hosting sysop blamed it on my site and disabled the database that serves it by making the directory the mysql files are in unreadable. mysql didn’t seem to handle that condition well, and since maisonbisson was still piling up queries looking for the content in the db, things continued to go downhill. my involvement started around pm friday night (yes, i’m that dorky). pepper links pepper computer buying a pepper pad at amazon pepper hacks victor rehorst has been blogging about his pepper since he got it (a few days ago) pepper pad stories at teleread other pepper pad stories here at maisonbisson open test sites i guess not everybody in nevada loves the test site as much as this postcard might suggest, but hey, what do tourists know? the image comes from _roberta‘s flickr photostream, and she doesn’t seem too critical. about miles southeast today, the trinity site — where the world’s first atomic weapon was detonated in a test on july , at : : a.m. — is open to the public. pepper pad — first impressions the pepper pad (available at amazon) has a very clean out of box experience. there’s nothing to assemble and no questions about what order to do things in. just open, unwrap, plug in, startup. i attempted running through the configuration in my office, but the wifi propagation is very weak there and pepper pad couldn’t catch a signal. the requirements listed on the box say only two things: “broadband” and “wifi,” so it’s no surprise that the configuration application requires wifi — or perhaps a bluetooth phone it can connect through? those crazy k-fee ads it turns out that k-fee, the company that pushes its energy drink with the scary tv ads, has a english-language website. it also turns out they’ve got the scary car ad and eight others online. here they are: angler car buddha golf beach meadow yoga soothing waves ocean path pepper pad — arrival the pepper pad‘s technical details — a lightweight linux powered device with an . -inch svga touchscreen, wi-fi auto-configuration, bluetooth device support, multi-gigabyte disk, full qwerty thumb-keypad, stereo speakers, and more — are already well reported. but i’ve been arguing that attention to such details runs counter to the purpose and intended use of the device. many computer users can name (and point to) the cpu in their computer, but who of those can tell me what cpu or chipset drives their cellphone? must read: ambient findability peter morville‘s ambient findability sold out at amazon today on the first day of release. there’s a reason: it’s good. morville’s work is the most appropriate follow-on to the usability concepts so well promoted by steven krug in his don’t make me think and jakob nielsen in designing web usability. findability, morville argues, is a necessary component in the success and propagation of an idea or detail or fact. business and non-profits alike will benefit from understanding the value of findability. mt. moosilauke will and i climbed moosilauke in early august, but it was only now that i got around to stitching the panorama. the view is considerably wider than degrees, composited from photos. the “full-size” version on flickr contains gigapixels of data. the real full-size version is a over gigapixels. bsuite_innerindex wordpress plugin [[pageindex]] about “blogging” typically connotes short-form writing that needs little internal structure, but that’s no reason to cramp your style. as people start to explore wordpress‘s pages feature, it seems likely that we’ll need a way to structure content within posts or pages sooner or later. that’s why i’m working on bsuite_innerindex. it’s a wordpress plugin that puts named anchors on all of the <h >, <h >, <h*>-tagged content, and builds a list of links to those anchors that can be inserted anywhere on the page. game law redux matt says my attempts to analogize online roleplaying games to more familiar contests like chess or automobile racing are “just silly.” but his response appears to reinforce my point rather than refute it. it is the responsibility of the gamers and gaming organizations to create and enforce rules. people violating those rules are subject to sanctions by the gaming organization first, but it’s hard to imagine how any contestant who follows the rules of a (legal) game can be subject to legal sanction. teachers get paid crap from alternet: teaching in america: the impossible dream. tagline: many public school teachers today must work two jobs to survive, and can’t afford to buy homes or raise families. why do we treat our teachers so poorly? open source gis here’s an interesting geoplace.com article on open source gis tools, including gis extensions to posgresql and mysql. via the map room. distracted by my shiny new camera my olympus c is hard to beat. steve’s digicams reviewed it well, and many friends with newer cameras find features or capabilities in it they miss on theirs. so, despite my schoolboy giddiness at the arrival of new gadgets, i’m waiting to be convinced that my new c will replace it. it too was well reviewed, and already i can see that it addresses some of my few complaints about the c , but transitions like this take time. bsuite_geocode plugin for wordpress i’m a big fan of the wp geo plugin, but i want more. my biggest complaint is that i want to insert coordinates using google maps or multimap urls, rather than insert them in the modified story editor. so i wrote a bit of code that reads through the urls in a post, finds the “maps.google” or “multimap.com” urls, fishes the latitude and longitude out of them, and adds some geocoding tags to the body of the post. home theater remote control i have a sort of guilt complex about looking at home theater issues. nonetheless, i’ve been building one piecemeal ever since i found an incredible deal on a video projector. now i’m working on assembling a video jukebox of sorts and i need to face the remote control stumbling block. that’s why i like the logitech harmony {# ,contentid= }, available at amazon. credit due: i got the tip from a post at engadget some time ago. helpful pages in the wordpress codex the following pages from the wordpress codex were surprisingly helpful recently: creating a static front page « wordpress codex creating tables with plugins « wordpress codex alphabetizing posts « wordpress codex the potential of political campaigning in online games matt and i have been talking about online role playing games lately. he’s more than interested in the new challenges they pose to our legal system, the new media opportunities they offer, the ways they’re altering culture. we got into a conversation about how companies are taking advantage of them in marketing campaigns, so i asked him, “in what presidential election year will we see the first in-game campaigning?” he seemed to think it might be as late as before that happened, but immediately embraced the concept. what’s zimbra? they say “zimbra is a community for building and maintaining next generation collaboration technology.” what i’d like to know, however, is whether zmbra is a community driven, social software answer to the problems of groupware — typically driven by management’s needs. a motivated team member is a productive team member i think this is dave. apparently they keep him in a cell at the server farm. doubletake stitches panoramic photos cheap i actually like the look of a broken panorama, where the borders of each photo are clearly visible — even emphasized. but last night i got the notion of doing a seamless pano and found doubletake, a $ shareware app that makes the process pretty darn easy. the sunrise shot above (larger sizes) was my first crack at it, but i was so sure i’d use it again (and again) that i’ve already registered it. ambient findability and the google economy i’m only just getting into peter morville‘s ambient findability, but i’m eating it up. in trying to prep the reader to understand his thesis — summed up on the front cover as “what we find changes who we become” — morville relates his difficulty in finding authoritative, non-marketing information about his daughter’s newly diagnosed peanut allergy: i can tell you from personal experience that google does not perform well when it comes to health. editing wordpress “pages” via xml-rpc wordpress‘s pages open the door to using wp as a content management system. unfortunately, pages can’t be edited via xml-rpc blogging apps like ecto. this might be a good thing, but i’m foolhardy enough to try working around it. here’s how: find a text editor you like and open up the wp-includes/functions-post.php file. in the wp_get_recent_posts() function, change this: $sql = “select * from $wpdb->posts where post_status in ('publish', 'draft', 'private') order by post_date desc $limit”; recycling tips from our physical plant along with the energy saving and water saving tips previously, our physical plant folks have sent out these recycling tips: recycling of aluminum cans — saves % of the energy required to make the same amount of aluminum from its virgin source. one ton of recycled aluminum saves , kwh of energy, barrels of oil, million btus of energy. one ton of recycled aluminum saves cubic yards of landfill space. suv sales slump earnings reports from car makers seemed to suggest suv sales were down last spring, and with gas prices near $ per gallon in some parts of the country still, nobody should be surprised that yahoo! is saying interest in suvs is down — way down — now: if the buzz is any indication, then yes. searches on “hybrids” outrank “suvs” by a tremendous margin, and it’s the same story with individual models. satellite broadband macsimum news did a story on satellite internet options a few weeks ago, but reader reports focused on fixed base station solutions for domestic use. what about mobile data solutions for international use? that’s where companies like outfitter satellite come in. they’ve got inmarsat solutions that can do kbps (or bonded to kbps) almost anywhere in the world. and, for customers in the mid-east or asia, they’ve got a kbps rbgan solution that seems to offer much better throughput at far lower prices. plan c: signed javascripts the mozilla docs on javascript security give a hint of hope that signed scripts will work around the cross-domain script exclusions that all good browsers enforce. but an item at devarticles.com throws water on the idea: signed scripts are primarily useful in an intranet environment; they’re not so useful on the web in general. to see why this is, consider that even though you can authenticate the origin of a signed script on the web, there’s still no reason to trust the creator. pc world pepper pad reviewer doesn’t get it david rothman pointed me to michael lasky’s pc world review of the pepper pad. lasky bangs on pepper, saying he can’t recommend it. too often, i think, technology reviewers approach a new product without understanding it. lasky tells us how the pepper performs when playing music or videos before comparing it to “notebook computers available for the same or a lower price.” we wouldn’t let an automotive reviewer conclude a review of a prius hybrid to a chevy truck by saying the truck is the better deal because it has a bigger engine for the same money, so why let technology reviewers off so easy? bstat japan! it looks like bstat has been localized for japan! with that in mind, i’d love to hear from international users about what i can do to make localization easier. there will be some big changes in the transition to bsuite, and it might be a good time to make sure i’m properly supporting wp‘s translation tables and localization features. plan b: remote scripting with iframes i have plans to apply ajax to our library catalog but i’m running into a problem where i can’t do xmlhttprequest events to servers other than the one i loaded the main webpage from. mozilla calls it the “same origin policy,” everyone else calls it a cross-domain script exclusion, or something like that. some mozilla folks are working on a standard to address the problem, but it could be quite a while before browser support is common enough to build for it. water saving tips our physical plant folks sent out this list of water saving tips to followup on the energy savings tips they sent previously. again, i think they should be blogging them, but what do i know? (it’s a rhetorical question, please don’t answer.) limit the use of domestic hot water — use cold water whenever it will do. turn off the water while you are brushing your teeth or washing your face. atlanta scene my friend troy keeps a studio at saltworks, a combined gallery and studio space in atlanta where prema murthy just opened her destructures show. i was in atlanta to see troy and family, so the opening was added sugar, and quite a pleasure. the image above comes from troy’s above and below series. next big thing: identity management i might be overstating it, but identity management is the next big thing for the open source community to tackle. that’s why i like sxip, even though i know so little about it. there are a number of other solutions stewing, but most of those that i’m aware of are targeted at academic and enterprise users. wouldn’t it be nice to have some federated system of identity management among blogs? linotype fontexplorer i was never a very good graphic designer, but the part of me that thought i was still pays attention when i see software like linotype’s free fontexplorer, described somewhere as “the itunes for fonts.” that’s excitement… “oooh… i want a number ten.” — a man stepping into line at the airport mcdonalds. the number ten meal, by the way, is a ten piece chicken mcnuggets meal. absinthe roderick sent me a link to this reason article on absinthe that claims: the u.s. food and drug administration considers true absinthe “adulterated” because of the wormwood. production, sale, and importation are banned, but mere possession is not, and customs agents typically ignore a bottle or two in your suitcase. it’s a legal situation that seems designed to keep absinthe cool.“ the wikipedia article on absinthe pretty much confirms that point, so who’s going to test it? improvised anti-telemarketing device the telecrapper is an improvised, homemade system that identifies telemarketing calls and leads the marketer through an artificial conversation that wastes the company’s time and money. the idea is to drive down productivity, and like so many other productivity sapping things, it can be quite funny. check this flash-animated recording: my hip hurts (mirror) rather less funny, though interesting nonetheless, is egbg’s counterscript. tc k hint via engadget. fixing position: fixed in ie it turns out the internet explorer doesn’t properly support css’s position: fixed. google led me to the following: how to create – making internet explorer use position: fixed; doxdesk.com: software: fixed.js fixed positioning for windows internet explorer the doxdesk solution looks promising and simple, but i think bugs elsewhere in my layout are preventing it from working. it’s time to start again from scratch. powerpoint. killer app? ruth marcus at the washington post wonders if powerpoint is a killing app. she’s not the first to note that nasa administrators make decisions — sometimes fatal decisions — on the basis of powerpoint presentations that mask or misrepresent details. i wrote about edward tufte’s cognitive style of powerpoint essay in a previous post. marcus doesn’t add to many new points, but the column is a sign that an anti-powerpoint movement may be growing. [fwd:] katrina eyewitness report (about the photo) the following report comes from cosmobaker.com, which includes this preamble: edit: the following is an email that was sent to my mother from one of her colleagues. although i cannot substantiate the contents, after all the horror stories that i’ve heard so far, i though that this one was important to tell. stand up and be counted. spread truth. stay awake. c —–original message—– wifi in public spaces a message came acrross the web lib list a few weeks ago with the following request: i want to hear from libraries who are currently implementing, or who already have implemented, wireless access for staff and/or patrons. i want your ‘stories’–good, bad and ugly. issues and/or triumphs with it staff, vendors, library staff, library boards, faculty committees, etc. i’m looking for all aspects of the process-finding hardware, implementation, policy (!), training staff, marketing the service to your patron base, troubleshooting and maintenance issues. search, findability, the google economy: how it shapes us just when i was beginning to feel a little on my own with my talk about the google economy here, i see two related new books are coming out. the first is peter morville’s ambient findability. the second is john battelle’s the search. findability appears to ask the big question that i’ve been pushing toward. from the description at amazon: are we truly at a critical point in our evolution where the quality of our digital networks will dictate how we behave as a species? trusted computing: the movie benjamin stephan and lutz vogel at lafkon bring us this wonderfully engaging animated story of trusted computing. there’s lots more to the story at againsttcpa.com, and i need to thank david rothman at teleread for alerting me to both the video and the site. i haven’t had much to say about tcpa, but i think of it like technology politics…politics where i have no say, no vote, no power. wide world of video games matt started talking up the weird issues developing around multiplayer online games a few weeks ago. then soon after he blogged it, a story appeared in on the media (listen, transcript) short story: online gaming is huge — one developer claims four million paying customers. more significantly, the interplay between real and virtual worlds might create new challenges for this real world legal system. “theft” of in-game money and equipment among players in the online world is possible, but it’s lead to the real-world arrest of at least one person and the murder of another when authorities refused to act. energy saving tips our physical plant folks sent out a message with tips on how to conserve energy. perhaps they oughtta blog this stuff? here it is: computer power management — a typical computer monitor uses to watts of electrical power, depending upon screen size. do not use screensavers as energy savers as they continue to use the monitor at full power and do not conserve energy. configure your monitor to turn off after minutes of inactivity, your hard drive to turn off after minutes of inactivity, and your desktop computer or laptop to go into a standby or sleep mode after minutes of inactivity. osceola weekend i climbed the osceolas with will and adam this weekend. it was my first overnight in a long, long time, and their first mountaintop sunrise. i used to do sunrises on mt. monadnock, but i’d lost the habit. more pictures of the osceola adventure at flickr. what counts will reminds us: “flasks are like people, it’s what’s on the inside that counts.” from the top of mt. osceola. the quotable john scott john scott reminds the naive: “don’t believe everything you find in google.” be a leader! manage your staff with ralph wiggum quotes! “i eated the purpleberries” (groaning). “how are they ralph…. good?” “they taste like…burning.” more goodness at the ralph wiggum soundboard, via informationnation. more quotes, like “oh boy, sleep! that’s where i’m a viking!,” at thedotdotdot. if i close my eyes, does it go away? can bush censor his shame away? reuters: fema accused of censorship: “it’s impossible for me to imagine how you report a story whose subject is death without allowing the public to see images of the subject of the story,” said larry siems of the pen american center, an authors’ group that defends free expression. brian williams’ msnbc nightly news blog: while we were attempting to take pictures of the national guard (a unit from oklahoma) taking up positions outside a brooks brothers on the edge of the quarter, the sergeant ordered us to the other side of the boulevard. axe gang security bumbles again we laugh at the single minded foolishness of the axe gang in kung fu hustle jackie chan’s the legend of drunken master, but do we laugh when we see it in our own security policies? to intelligence staffers and border guards working under a policy of hammers, all the world is a nail. here’s an example: in august , us customs agents stopped and searched ahmad el maati, a kuwaiti-born canadian and a truck driver crossing the us-canadian border at buffalo, ny. marketing and search engine optimization i don’t want to admit to being interested in marketing, but i am. here’s a few links… blogs: church of the customer seth godin aaron wall’s seo book.com threadwatch.org randomness: writing, briefly google’s search result quality evaluation guidelines definition of the google economy at wikipedia the fall of advertising and the rise of pr simple bookmarklet demo bookmarklets are interesting little bits of javascript stored as bookmarks. they’ve been around since about (earlier?), but i’ve never bothered to write one. here are a few examples: this sort of creates a bookmark alexa snapshot wayback la femme’s poison browsing flickr the other day i found la_femme‘s poison. other good photos in her photostream. energy crisis mike whelan posted the above photo to his flickr photostream recently. back in april, when gas prices were still well below the $ -per-gallon mark, it looked like sales of suvs were starting to slow. interestingly, we’ve crossed the threshold keith bradsher quotes in high and mighty, his book detailing how the us auto industry became so dependent on suvs and how common sense has been powerless against them. the threshold was the point at which gas prices would begin having the same effect on current car purchases as the s oil crisis did. doing relevance ranked full-text searches in mysql i’m going out on a limb to say mysql’s full-text indexing and searching features are underused. they appeared in mysql . . (most people are using .x, and is in development), but it’s been news to most of the people i know. here’s the deal, the match() function can search a full-text index for a string of text (one or more words) and return relevance-ranked results. it’s at the core of the list of related links at the bottom of every post here. la tomatina from a reuters story in chinadaily: at noon [wednesday], municipal trucks dumped about tons of ripe, juicy plum tomatoes at the feet of adrenaline-charged crowds in town’s main square. within minutes the area was covered in red slime, and clouds of tomato sauce filled the air. it all takes place in buñol, in spain’s valencia region along the mediterranean coast. canada.com{#db d - e- fa -a b -be ad } describes the origins: local lore says it began in the mid- s with a food battle that broke out between youngsters near a vegetable stand on the town square in buñol, kilometres southeast of madrid. signals tells google a thing or two signals takes on google and suggests some improvements. ucla takes on google scholar via jay bhatt at lisnews: ucla libraries‘ discussion of google scholar, search engines, databases, and the research process. time-picayune in exile times-picayune editor jim amoss answered questions for on the media‘s brooke gladstone. amoss and his staff have been covering the catastrophe in new orleans as only locals can. some of the best reporting i’ve seen on this has come from the times-picayune, and i was quite amazed when i discovered the electronic edition wednesday. despite the damage, they appear to have start releasing a print version again and are distributing it in the city and in communities where refugees have fled. sneaky is there a sneaky surprise hidden in your hotel room? see if you can recognize anything in these photos (tip: mouse-over them). back to school video kate says: “life is good. and i’ve got a sleeping bag from the future.” tim explains, a bit. none of that matters nearly as much as the video kate is quoting from, and that matters now because back to school time means play dates and sleepovers. tim guarantees it will kill a few braincells, but nothing ridicules us the way we once were (and often still are) better than saturday night live. things go to hell defensetech’s noah shachtman writes: organizing thousands and thousands of people, in hellish conditions and in a hurry, is tough work. let’s take that as a given. but still: we’re now a work week into a natural disaster that had been forecast for years, and new orleans “is being run by thugs,” the city’s emergency preparedness director tells the times. “some people there have not eaten or drunk water for three or four days, which is inexcusable. rollerblading via pya{# }. policing by cellphone though we imagine the dutch to be a rather unexcitable lot, i did anyway, it turns out they have a history of getting rowdy at football games (yes, if this all happened back in the states i be calling it “soccer”). so it can’t be so much of a surprise that fans rioted again in april. what is surprising is that mobile phone companies got involved in the investigation. this ap report tells the story: the water down there i don’t watch tv, so i haven’t seen many images of the flooding in new orleans until i found these. amazingly, the times picayune is publishing pdf editions during disaster. the hurricane and flood damage are truly scary, but the worst news is on page five, which tells of widespread looting: law enforcement efforts to contain the emergency left by katrina slipped into chaos in parts of new orleans tuesday. the google economy will beat you with a stick call it a law, or dictum, or just a big stick, but it goes like this: the value and influence of an idea or piece of information is limited by the extent that the information provider has embraced the google economy; unavailable or unfindable information buried on the second or tenth page of search results might as well be hidden in a cave. the ultraviolet sun from the nasa website: eit (extreme ultraviolet imaging telescope) images the solar atmosphere at several wavelengths, and therefore, shows solar material at different temperatures. in the images taken at angstroms the bright material is at , to , degrees kelvin. in those taken at , at million degrees. angstrom images correspond to about . million kelvin. angstrom, to million degrees. the hotter the temperature, the higher you look in the solar atmosphere. enabling .htaccess on mac os x i do a lot of web development on my laptop. i’ve got apache and php there, so it’s really convenient, but i usually move projects off to other server before i get around to wanting to mess with mod_rewrite. not so, recently, but i ran into a big stumbling block when i discovered os x’s apache comes pre-configured to ignore .htaccess files. a couple points. first, apache’s own mod_rewrite docs include the following quote: coconut battery coconutbattery: coconutbattery is a tool that reads out the data of your notebook-battery (ibook/powerbook). it shows the current charge of your battery as well as the current maximum capacity related to its original. via o’grady’s powerpage{# } awstats as much as i like the bstat functionality of bsuite, i never intended it to be a replacement for a full server log-based stats application. that’s why i’m happy my hosting provider offers awstats. the reports suggested ways to optimize my pages so that i could control my bandwidth consumption — up to . gb/day before optimization, now . gb/day. but today i found an awstat feature that got me excited enough to email the university sysadmin about it: email stats. the google economy — the wikipedia entry i’m rather passionate about the google economy, so it shouldn’t be too much of a surprise to learn that i just wrote about it in my first ever wikipedia entry. here it is: http://en.wikipedia.org/wiki/google_economy “google economy” identifies the concept that the value of a resource can be determined by the way that resource is linked to other resources. it is more complex than search ranking, and broader than interlinked web pages, though it draws meaning from both. bsuite development bstat has become bsuite. the name change reflects the fact that i want the plugin to do a lot more than track usage stats. one of the first features to enter testing here is the “related” section below. i’m calling it “bsuggestive,” but that may turn out to be too cute a name to tolerate for long. the results are based on the tags for the post, so it doesn’t work with old posts that haven’t been tagged, and it sometimes returns some weird matches, but it’s still alpha, so what can we ask for. beloit college’s list of things that make us look old to incoming students we’ve seen lists like this before. beloit college in beloit wisconsin releases their “mindeset list” for their incoming class every year around now. the point is to remind us how cultural touchstones change over time. it does that, but it also give us (me, anyway) a good chuckle. it’s worth reading all the way down to number , at least, where libraries get a good mention. video bulb and zakka shop nyc the video bulb is a “lipstick-sized tube” that plugs in to your tv’s rca jack and plays bitman videos. gadgetmadness explains what bitman is: bitman is the creation of japanese art performer “meiwa denki” and was an -bit electronic stick figure who would dance, pose, etc. the videobulb sounds interesting enough, but i think i could get into the reseller as much as the gadgetmadness writer did: i went to zakka shop & space the last time i was in nyc, and literally wanted everything in the store. changing modes of communication i talk a lot about the google economy here, and how that and other ideas are driving changing modes of communication. today i learned of arxiv. henry farrell describes it at crookedtimber: [i]t’s effectively replaced journal publication as the primary means for physicists to communicate with each other. journal publication is still important – but as an imprimatur, a proof of quality, rather than a way to disseminate findings to a wider audience. wordpress as cms a friend and i have been talking about what it would take to turn wordpress into a cms. we both have our doubts, but today i found this job ad that suggests we’re not alone in at least thinking of the possibility. needed: web designer/programmer for our sites we’re growing very fast, and have outgrown our current cms and design. we’re looking for a designer and/or programmer to redesign our rapidly growing network and implement a cms that ties it all together. kingcosmonaut & wp themes i stumbled across the sometimes funny how to live your life and got curious about the theme. turns out it’s by sebastian schmieg, who keeps things real at kingcosmonaut. the theme is blix, but the kingcosmonaut site is much cooler. flock the developers describe flock as [t]he world’s most innovative social browsing experience. we call it the two-way web. which is a good enough sales pitch to make me try the free demo, but it’s all still a private beta. perhaps they’re trying to prove the point that nothing builds buzz better than unavailability. osakasteve gushes: a browser that is designed around social software like blogs and flickr itunes music store api? i can’t explain why, at least not yet, but i’m looking for a way to search the itunes music store{#xffsogqwv s&offerid= . &type= &subid= } catalog outside of itunes. rumors of an itunes-google partnership{# } have been flying lately, but what i really want is a webservice/api i can use. yes, apple offers an affiliate program that supports direct links, but again, they don’t offer an amazon-style api to search their catalog. all of this has me thinking about reverse-engineering the itms to build the webservice i’m looking for. a list apart updated a list apart, has been revamped and they’re proud of it. they should be, it’s beautiful and functional. it’s one of the few early web development resources that’s still with us, and there’s a reason. copyright and academic libraries back when i was looking things up for my digital preservation and copyright story i found a bunch of info the university of texas system had gathered on issues related to copyright, libraries, and education. in among the pages on copying copyrighted works, a/v reserves, and electronic reserves i found a document titled: educational fair use guidelines for digital images. it’s some interesting stuff — if you get excited about copyright law. re-shelving orwell’s via jon gordon‘s future tense: re-shelving george orwell. smart people everywhere are taking it upon themselves to re-shelve george orwell’s from fiction to more appropriate sections in non-fiction, like “current events”, “politics”, “history”, “true crime”, or “new non-fiction.” instructions and photos on flickr. laura quilter defends google print with all the talk about google scanning or not scanning copyrighted books, i was happy to see laura quilter talking about google as a library. the internet archive is certainly a library. […] libraries may be private, semi-private, public; for- or not-for-profit; paper or digital. why is google not a library? more interestingly, she casts a critical eye on the texaco decision that everybody points to as the guiding law on fair use. wikipedia api? i want wikipedia to have an api, but it doesn’t. some web searching turned up gina trapani’s wikipedizetext, but that still wasn’t exactly what i wanted. a note in the source code, however, put me back on the trail to the wikipedia database downloads, and while that’s not what i want, i did learn that they’ve got a table of just the article titles (over . million of them) in their downloads. drug side effects drive patients to gamble, eat, drink, and … …people with parkinson’s disease temporarily became compulsive gamblers after taking […] drugs designed to control movement problems caused by the illness… that’s the lead in this forbes story on the matter, and that’s not all. a variety of ‘interesting’ side effects popped up among a relatively small number of study participants: pathological gambling compulsive eating increased alcohol consumption obsession with sex. the drugs in question are “dopamine agonists” and are part of the standard treatment of parkinson’s disease. segway easy rider movie trailer remember those guys who rode a segway cross-country last year? well, they’ve got a movie coming out. yup, there’s even a trailer. possibly more interesting: the photo gallery (from which the photo above came). thanks to engadget for the link. steelers fan never misses a game day in remembering james henry smith, a zealous pittsburgh steelers fan who died of prostate cancer in early july, his family asked the samuel e. coston funeral home to do things “as he would have wanted them to be.” for the viewing, the funeral home arranged a living room ensemble with the tv and recliner just as smith liked it on game day. an ap article describes it: smith’s body was on the recliner, his feet crossed and a remote in his hand. alt browser shiira project, an apple webkit-based browser with some interesting features. sadly, it also brings page transitions to the mac. let’s hope these don’t become the new . chasing clicks al asked how low i will go to chase traffic. truth is, i can’t answer. maisonbisson has had moments of popularity, but it’s hard to know why. alexa tells us there are million unique sites on the web, but… if you take alexa’s top , sites you’ll find that almost out every clicks are spoken for. in other words, almost % of all the traffic on the web goes to the sites in the top k list, leaving the remaining million or so sites to fight over the scraps. neutron bomb boing boing has an exclusive profile of neutron bomb inventor samuel t. cohen by charles platt. all the reports so far are that it’s a , word “must read.” the article, profits of fear, is available in pdf, plain text, and palm doc versions at boing boing. thanks to david rothman for the heads up. extra: rothman asks what it all says about mainstream media when respected authors eschew traditional media for blogs. another limitation of lc classification right up front in the prologue of ruth wajnryb’s expletive deleted she quotes the following from richard dooling on the difficulty in researching “bad language”: the library of congress classification system does not provide a selection of books … on swearing or dirty words. a researcher … must travel to the bf of psychoanalysis, the pe of slang, the gt of anthropology, the p of literature and literary theory, the n of art, the rc of medical psychiatry, and back to the b of religion and philosphy. network effects on violence some time ago i pointed to john robb’s discussion of the potential for the network to amplify the threat of violence from otherwise un-connected and un-organized individuals. now noah shachtman at defensetech is writing about “open source insurgents.” it used to be that a small group of ideological-driven guerilla leaders would spread information, tactics, training, and cash to their followers. no more. internet-enabled insurgents with only the loosest of real-world connections can now share all of that freely online. grizzly man david edelstein’s review of werner herzog’s documentary, grizzly man, describes timothy treadwell as …a manic but lovable whack-job who doggedly filmed and obsessively idealized the bears that would ultimately eat him… the film is made up largely of the bits of the hundreds of hours of video that treadwell himself shot during his years with the bears. later, however, edelstein — probably restraining laughter — calls treadwell “histrionic” and a “drama-queen” (isn’t that sort of redundant? php developer resources somebody asked for some links to get started with php. of course i lead them to the php.net official site, where the documentation is some of the best i’ve seen for any product. i also suggested phpdeveloper.org and phpfreaks.com, though the truth is i usually google any questions i have that the official docs don’t answer. still, i’ve found some good info at both of those. finally, the php cheat sheet at ilovejackdaniels. drm = customer lock-in donna wentworth is now saying what i’ve been saying for over a year now. digital rights management (drm) isn’t about preventing copyright violations by ne’er-do-wells, it’s about eliminating legal me me fair use and locking in customers. in your pc == a toaster, wentworth quotes don marti saying: isn’t it time to drop the polite fiction that msft and other incumbent it and ce [ce = consumer electronics — casey] vendors are only doing drm because of big, bad hollywood? digital preservation and copyright we’re struggling with the question of what to do with our collection of vinyl recordings. they’re deteriorating, and we’re finding it increasingly difficult to keep the playback equipment in working order — the record needles seem to disappear. we’re re-purchased much of our collection on cd, but some items — this one might be one of them — are impossible to find on cd. so we’re considering digital preservation, capturing the audio of the records and scanning the dust jackets. the part where speakeasy cons me into shilling for them the speakeasy speed test is an okay way to waste some time, but the most amusing thing is how easy they make it to promote them. the speakeasy badge here looks like any web ad, but they’re not paying for it. all they did was post a link saying add speakeasy speed test to your site. i guess we all ought to take this marketing tip from them: make sure your readers know how to link to you. maisonbisson top seven the most recent version of my wordpress stats tracking plugin makes it very easy to see and track my top stories. i don’t know whether i should be proud or ashamed by them, but here they are: big bear photos that story gets a lot of morbid interest, and i’m sure the movie grizzly man will too. k-fee energy drink tv ad for a while, though, people looking for that story were finding my zygo energy vodka story instead. atomic while looking for a picture for my memorial to the bomb, i found a number of related links. this blog is sometimes nothing more than an annotated bookmark list, and this is why…. the bomb project describes itself as: a comprehensive on-line compendium of nuclear-related links, imagery and documentation. it is intended specifically as a resource for artists, and encourages those working in all media, from net.art, film and video, eco-intervention and site-specific installation to more traditional forms of agitprop, to use this site to search for raw material. linking bias danah boyd posted about the biases of links over at many many the other day. she looked for patterns in a random set of blogs tracked by technorati as well as the top blogs tracked by technorati. she found patterns in who keeps blogrolls and who is in them, as well as patterns about how bloggers link in context and who they link to. the patterns boyd points to would certainly effect the google economy, our way of creating and identifying value based on linking structures. annoises via gizmodo: a cd of annoying sounds at gadgets.co.uk. twenty “ear splitting” sound effects and a pair of earplugs “for your sanity and protection” for £ . . what sound effects? drill party (at least people) orgasm (outstanding) train drum (played by a child) inhuman screams walking (high heels) domestic squabble doors banging bowling unhappy dog practicing a violin traffic jam garbage truck a screaming newborn baby phone ringing ball game pigeons spring house cleaning cock-a-doodle-do! grizzly man within the last wild lands of north america dwells an animal that inspires respect and fear around the world. it is the grizzly bear, a living legend of the wilderness. grizzlies can sprint thirty five plus miles an hour, smell carrion at nine or more miles, and drag a thousand-pund animal up steep mountains. the grizzly bear is one of a very few animals remaining on earth that can kill a human in physical combat. point ‘n shoot defensetech reported on the firefly, a disposable camera that can be shot from the m grenade launchers used by us land forces. the cameras fly meters in eight seconds, wirelessly sending pictures back to the soldier’s pda. now they’ll know what’s over that hill or around that corner. not that soldiers don’t need this sort of thing, but one wonders when hasbro will release a plastic version in bright colors. movie night: open water joe recommended open water whole heartedly, but others, like some of these one-star reviewers at amazon, had equally strong reactions against it. i first learned of the events the movie is based on in bill bryson’s in a sunburned country, where he described the events of thomas and eileen lonergan’s disappearance during a dive in the australian pacific. the similarity between these true events and the movie’s events likely ends there. jimmy wales’ free culture manifesto jimmy wales, the founder of wikipedia and director of the wikimedia foundation, is working on his keynote for the wikimania conference in frankfurt. ross mayfield at many many posted a preview and gives some background. what should we expect? wales’ speech touches on ten things necessary for free culture: free the encyclopedia! free the dictionary! free the curriculum! free the music! free the art! free the file formats! free the maps! years later in what was to be the final act of world war ii in the pacific, the united states made the first and only use of nuclear power as a weapon in the bombing of hiroshima and nagasaki on august th and th (us dates), . george weller of the chicago daily news snuck in to nagasaki in early september and became the first american journalist to see the destruction. his stories were censored, and official sources maintained control of news about the bombings and the aftermath for many years. reminisce: my first ebook the first ebook i ever read was bruce sterling’s hacker crackdown on my newton message pad . it had a big and bright screen — “the best screen for reading ebooks on the (non-)market” says dj vollkasko — but it could get a bit little heavy at times. crackdown is available for free, along with perhaps , others, at matthew mcclintock’s manybooks.net. downloads are available in different formats, or you can read online. information is sexy it used to be you could identify the librarian by the sensible shoes, but times they are a changing. witness this ad from library bar. sure their “librarians” are bartenders, but what cultural shift changed to thrust librarians up the sex appeal scale? yeah, this is old. after all, it was the spring of bust magazine that asked if librarians might be the new “it” girls, but it’s still amusing. drm: bad for customers, bad for publishers the news came out last week that the biggest music consumers — the ones throwing down cash for music — are also the biggest music sharers. alan wexblat at copyfight says simply: “those who share, care” (bbc link via teleread). rather than taking legal action against downloaders, the music industry needs to entice them to use legal alternatives, the report said. lawsuits against customers go hand in hand with drm in limiting community buzz for a particular artist or song. gizmos for geeks colin pointed out spark fun electronics as a source for all manner of geeky components, like component level gpss and accelerometers. thing is, they also sell the components in kits with custom pc boards, some with usb interfaces. the coming information age that headline might seem a little late among the folks reading this. but we’re all geeks, and if not geeks, then at least regular computer users. regular computer users, however, are a minority. worldwide, only around million people have internet access, and fewer than million people in the us have internet access at home. with populations of over billion and million respectively, there’s clearly a lot of growth potential. faces i stumbled upon captnkurt’s information nation where he popped a link over to eric myer’s stereotypes. the gimic — and it’s a fun one — is that you can mix and match bits of faces. i don’t know why i like the combo above so much, but, anyway. the thing about this is that it reminds me of troy bennett’s human-intoface, reported here back in and . separately, i need to go back and take another look at captnkurt’s story about couchsurfing. nokia i’ve been babbling like a stoolie for pepper here for the past couple weeks, but after some prodding by roger sperberg i’ve started to take a serious look at the nokia linux-based internet tablet. to get me started is mike cane’s hands on report from some time spent with it at linuxworld expo. nokia is pushing maemo.org to support the developer/hacker community, and there’s already some interesting work being done. more bluetooth hacks as if bluejacking wasn’t fun enough, now a few folks have now taken it a little further and figured out how to connect to the growing number of bluetooth handsfree sets all around us. gizmodo fed me the link to what they’re calling “the car whisperer.” nothing against these guys, but it’s not like they did anything amazingly complex. their story explains that they’re simply taking advantage of poor security like default passwords. movie night: house of flying daggers i’ve been a fan of zhang yimou’s films since, well, for a while now. but i’m also a huge kung fu fan — jackie chan especially — so house of flying daggers was quite a treat. it’s not that i didn’t like hero, or that daggers was particularly funny. to the contrary, it’s tale of complex characters who don’t end well. that might be story enough, but every scene is richly photographed and styled — a hallmark of so many of yimou’s films, but wonderfully so in daggers. sweet cheat sheets colin over at command-tab alerted me to some great cheat sheets, including this one for javascript at ilovejackdaniels.com. apple releases multi-button mouse apple this morning released the mighty mouse. with a scrollball, left and right click, and side buttons, it’s a big departure from apple’s old opposition to multi-button mice. apple didn’t invent the mouse, but they were probably the first to put mice through usability testing. one, two, and three button mice of a great many different shapes and sizes were tested before they settled on a one-button mouse for the original macintosh in . hands on the pepper pad the most amazing thing about the pepper pad is how easy it is to pick up and use, how easy it is to walk around with, and how it’s available when you want it and gone when you don’t. the pepper pad‘s portability goes far beyond that of laptops. i mentioned previously that laptops move from desk to desk and bill gates tells us how poorly laptops work in elevators. netflix expands queues this is old news, but netflix{#xffsogqwv s&offerid= . &type= &subid= } now offers multiple queues for each account. queues, of course, are the movie wish lists each netflix customer keeps; when you return a movie, they send out the next movie in your queue. in the old days, each subscriber got just one queue, no matter how many members of the household had an interest in the movies. two people, one queue? marital drama ensued in my home and others. movie night: the underneath steven soderbergh has done a number of good films, but the underneath isn’t among them. it’s interesting to see the director working out his moves, but more entertaining to see them in a more mature form, as in out of site. eh, i’m ready to give the guy a break. my real complaint has nothing to with this film. instead, it’s about kafka, one of his best works. it was released in , and though they’ve still got a few vhs copies in a warehouse somewhere, it deserves a dvd release space shuttle tracking (and other good uses of the google maps api) tom mangan has put the google maps api to interesting use with his space shuttle tracking page. also worth checking out: his blackbird spotting site and tlable, a little extension to make pinning/annotating maps even better. politics and the google economy while i’m anxiously working to better fit libraries into the google economy, a few paragraphs of barry glassner’s the culture of fear, got me thinking about its role in politics. glassner was telling of how a article in usa today quoted the national assocation of scholars{# } saying that georgetown university had dumbed down its curriculum and dropped shakespeare{# } requirements. of course, nothing could have been farther from the truth, a point confirmed by the georgetown’s dean. japanoid k-cars gizmodo reported it a while ago, but a canadian company called japanoid is importing these and other tiny japanese cars. how tiny? at or under . meters (under feet!) wide with engines cc or under. they’re called kei jidousha, or keicars, or just k-cars (though not to be confused with chrysler’s k-cars). japanoid has vehicles listed, but my favorites are those four above and this funny looking truck. movie night: entropy phil joanou’s entropy isn’t available in the us on dvd, but i found it at amazon uk. imdb has this to say: stephen dorff narrates this tale about how his life goes astray as his character attempts to strike a balance between the demands of directing his first film and the pressures of his new romance with a model. u ’s bono plays a role in this film as both himself and dorff’s character’s concience. the problem with pdas today when i finally get around to writing up my impressions of the pepper pad, i’ll be pointing to roger sperberg’s recent posts at teleread about non-pda handhelds and computers for stand up use. at the moment, however, some of his points remind my of a few i’ve got to make about pdas here. i’ve got a sony clie th- , the top of the line of the last series they imported to north american shores. gizmo project, voip, asterisk jason o’grady{# } introduced me to the skype-like gizmo project by the folks over at sipphone. i’ve been a vonage customer for a couple years now, so i’ve had a chance to get familiar with voip, and i’m looking for a good bluetooth headset so i can try gizmo and skype (and others), but i got to wondering what more i could do. asterisk is an open source pbx application that runs on linux, macos x, and others. marriage alternet has a story by monica mehta titled the myth of marriage with this synopsis: a radical new book debunks the concept of marriage as a time-honored institution, and argues that we need to loosen up about it. the book is stephanie coontz’s marriage, a history. related previous story: the “sanctity” of marriage. put a pepper in your library libraries are known for books. and despite the constant march of technology, despite the fact that we can put a bazillion songs in our pocket, despite the availability of the new york times and so many other newspapers and thousands of journals online, books are a big part of what libraries are. books, dead tree books with that rotting paper smell. and though i dare not prognosticate, i expect they’ll be an emblematic feature of libraries for a while now. elements of murder john emsley, author of elements of murder: a history of poisons appeared in an interview on npr’s fresh air’{# } earlier today. those who were fascinated by the morbid details of devil in the white city should give it a listen. i plan on checking out the book too, though it sounds like emsley offers more chemical formulae than outright suspense. ils: inventory or search and retrieval system? there’s an interesting discussion going at libdev about what our ilss are. it all started with a discussion of what role xml and webservices could/should play with ils/catalogs, but a comment reminded us that vendor’s decisions about adding new features to products that have been around for or years sometimes edge towards lock-in. i replied offering flickr as an example of a vendor that’s been successful in part because of their open apis. nuclear family vacation via defense tech: slate did a series last week titled a nuclear family vacation that visited the nevada test site; los alamos, lawrence livermore, and sandia national labs; and trinity. extra: a slideshow accompanies the text and the authors interviewed{# } on npr’s day to day{# }. related: previous nuclear stories at maisonbisson. karl rove’s leak-and-covergate two items from the blogosphere about rove’s leak-and-covergate at tikun olam and alternet. life magazine covers i get a kick out of these and life magazine covers. take a look and i think you’ll agree that no magazine puts photos like this on their covers today. screen real estate at x pixels, apple’s cinema hd display{#xffsogqwv s&offerid= . &type= &subid= } is big enough for three people’s egos. xml/php/swf charts flash app dynamically generates charts based on xml formated data or values in a php array. xml/swf charts is a simple, yet powerful tool to create attractive web charts and graphs from dynamic xml data. create an xml source to describe a chart, then pass it to this tool’s flash file to generate the chart. the same tool also accepts php sources. xml/swf charts makes the best of both the xml and swf worlds. pepper i’m off visiting the good folks at pepper today. i’ll update this post with photos as soon as they’re available, then look for a pair of posts about how the hardware/software works and what i’d like to do with it later. until then, here are some related posts: ultra portable computing, pepper pad , and portable computing. update: the picture above is blurry because of my poor photography skills. better pictures can be found at the pepper site. tags tags tags david weinberger at many-to-many pointed me to tom coates’ post about different schools of thought regarding tags. coates has been thinking about tags as keywords, annotations. thats how i’ve been using and thinking about tags too, but some people have different ideas. …at the end of the argument i said to joshua that it was almost like he was treating tags as folders. and he replied, exasperated, that this was exactly what they were. what’s a mirt? mirts turn red lights green, but merely having one will probably get you in a pile of trouble. more info at i-hacked.com{# }. peerflix ross rubin at engadget just alerted me to peerflix …which can be described on a basic level as ebay meets netflix. peerflix resembles many online dvd stores, but it neither rents nor sells dvds. rather, it depends on a community of users willing to trade dvds they have for dvds they want. there are no subscription fees. peerflix charges a -cent transaction fee and senders are responsible for the postage charge of cents for the mailers that the company distributes. john barlycorn must die in a popular antebellum arkansas story, a backwoodsman bought a -gallon barrel of whiskey, only to return a week later for another. “surely you haven’t drank that whiskey already?” inquired the astonished merchant. “it ain’t so much,” replied the backwoodsman. “there are six of us, counting the kids, and we have no cow.” it’s not quite as detailed as some of the stories in the foxfire books, but it’s a good treat. the failures of permission culture donna wentworth, over at copyfight pointed out a jd lasica piece detailing the responses from seven studios to his requests to use short ( - seconds) clips of their films in a non-commercial project he was working on with his child. …four of the studios refused outright, two refused to respond, and the seventh wobbled. this is the quandary millions of us face today: the hollywood studios demand that we ask for permission to borrow from their works — and then they deny our requests as a matter of course. google moon rocks google engineers have got the moon on their minds lately. we all got a laugh at their april fools day lunar hosting and research center job opening, but they’ve done themselves one better and several points more serious with google moon. sure, it’s in celebration of the first lunar landing years ago today, but if they’re so fixated on the moon, why not sponsor a space competition? google maps gets all the attention it would reasonably appear that here in the us, there’s only one map site: good ol’ google. but until google adds maps for countries other than the us, canada, and uk, the rest of the world will have to look elsewhere. enter the uk competitor: multimap.com has been serving the world outside the bubble since . from their self description: key features include street-level maps of the united kingdom, europe, and the us; road maps of the world; door-to-door travel directions; aerial photographs; and local information. jenny’s drm scourge jenny levine, over at the shifted librarian, is telling the latest chapter in her long-running struggle with drm. now, i’ve installed a lot of windows software in my day, so i feel pretty confident in my ability to double-click on an installation file. however, when i try to install [yahoo music engine], i get three screens into the installer (oh the joy of accepting the license agreement over and over) before i get an error message that says, “the file c:\downloads\ could not be opened. bstat beta release [[pageindex]]update: shout outs to zach, cliff, justin, and thomas who’ve submitted bug reports. their feedback has been rolled in to the b july release, available now (look for the link below). this is likely the last release before the code gets bundled into bsuite (more details on that later). changes this documentation supersedes any previous documentation. more changes to the bstat_pulse() function; bstat_pulse_style() is no longer used. it’s been replaced by a flag in the call. see the usage example to understand. want to customize the style? start with that example, look at the xhtml it outputs, work from there. update thanks to zach, these parameters are all optional. you can call it with nothing more than “bstat_pulse()”, if that’s your thing. still, i’d recommend using the full example below. there are a lot of improvements to the management console. the number of lines to display for each category and the date range (past day, week, month, etc.) are now configurable. quick start installation download and unzip bstat.zip place bstat.php in you wp-content/plugins directory place spacer.gif in your wp-content directory log in to your wordpress admin panel and activate the plugin, then visit the new bstat submenu of the options tab. this will allow bstat to create its database tables. add the bstat_hitit function to the footer.php of your theme (or in some other place where it will be called once for each page load). this starts the counting; you can see the results in the bstat submenu of the manage tab of the wordpress admin panel. in order to view the bstat results on your public pages, you’ll need to the bstat display functions to your pages. it’s funny ‘cause it’s true first lady laura bush speaking at the white house correspondents association gala noted: george’s answer to any problem at the ranch is to cut it down with a chain saw. which, i think, is why he and cheney and rumsfeld get along so well. the quote is all over the net now, but i found it in the august issue of vanity fair. australia’s rum jungle alan moorhead, in his rum jungle — a sort of casual ethnography or serious travelogue — explains the uses and attitudes towards alcohol in his native australia: […] i took it for granted that for all social occasions, at any time of the day or night, beer was the drink. you did not take it with your meals, but before or afterwards and in considerable quantities. beer was the solace of life and the white man’s true vision of bliss. full-text searching inside books search engine watch did a story about how to use google and amazon’s tools to search full-text content inside books. the gist? when you can get to the tools and where they’ve got content, it does a lot to make books as accessible and open as electronic content. sort of related: i’ve spoken of google print before and there’s more in the libraries and networked information category. organizational/institutional blogging done right jenny levine is talking about an example of the perfect library blog over at the shifted librarian. the posts are written in the first person and in a conversational tone, with the author’s first name to help stress the people in the library. the staff isn’t afraid to note problems with the new catalog, the web site, or anything else. full transparency — nice. you can feel the level of trust building online. hackable snackable gumstix the make: podcast pointed me to gumstix — really small computers built for hacking. cool. google hacks from o’grady’s powerpage{# }: i have no interest in true hacking (i.e. rummaging through people’s private junk) although viewing random unprotected ip cameras around the world in public places and controlling their panning and zoom functions is kind of mind-blowing. there are a ton of fun ghacks out there – like spelling out words in pictures using google image search, and the google poetry generator, or the news map generator etc. skyhook wifi geolocation old news from gizmodo and wi-fi networking news (quoting wifi nn): skyhook has assembled a database of information about . million access points across major cities in the u.s. by driving every street in every city. their software records multiple data points per sample for directionality. fire up their software on a laptop, and it compares the wi-fi information it sees with what’s in the skyhook database, popping out a latitude and longitude within to meters. coolest watch ever, today anyway the nixon rotolog{# &menu_id= }. ike dwight eisenhower’s eight years as president were about a lot more than i like ike buttons and interstate highways. from wikipedia: after his many wartime successes, general eisenhower returned to the united states a great hero. it would not be long before many supporters were pressuring him to run for public office. eisenhower was generally considered a political moderate, and it was not immediately clear which party he would choose to join. jet turbine powered toyota mr on ebay yup, it’s up on ebay now (closing in a day or so) with the following description: everybody needs one of these, cleaning out the garage, this little car is so much fun, it is thrust powered by ge t- turbines, has fuel tanks, power steering, power brakes, fire detection, fire suppression, roll over protection, self starting and quick. i have taken this car to the salt flats twice, the first time it wanted to fly @ mph, but after adding the spoilers and air dam it stayed solid thru mph with a lot more room to go. the google economy i’ve been talking about it a lot lately, most recently in a comment at libdev. in the old world, information companies could create value by limiting access to their content. most of us have so internalized this scarcity = value theory that we do little more than grumble about the new york times’ authwall or similar limitations to the free-flow and linking of information. jenny levine wrote recently about oclc/lj’s short-run (though not yet ended) experiment with authwalls. what’s a “blink” ? stealing from corante/copyfight: it’s a short, one-sentence blog post + a link, à la kottke remainders. [it’s] to share links to articles, resources, and websites of interest that do not necessarily require paragraphs of context or analysis. enjoy! solar backpacks & chargers solar charging backpacks: juice bags (news), voltaic solar backpack (news). and, solar ipod charger: solio (news, news). personalizing the preservation problem i went looking for an old file the other day. as it turns out, the file was from years ago, but that doesn’t seem so long ago now. anyway, i was amused to find how most of my images from that time were tiffs instead of jpegs. thankfully, tiffs are well supported now, but my old pagemaker files are largely useless to me. and while i was looking at these files from so long ago i found my really bad music from the day. is blogging career suicide? ken (i wish he had a blog to link to) pointed out bloggers need not apply in the chronicle of higher ed over the weekend. the story is to some a highly cautionary tale: a candidate’s blog is more accessible to the search committee than most forms of scholarly output. it can be hard to lay your hands on an obscure journal or book chapter, but the applicant’s blog comes up on any computer. the big switch other than a bit of head scratching after the announcement in june, i’ve been quiet about apple’s switch to intel processors. now, arstechnica‘s jon “hannibal” stokes has written some of the most intelligent material i’ve seen since. how’s it work? hannibal thinks apple’s relationship with ibm soured to the point where they refused to play the game. and apple is imagining a world of devices macs, ipods, and as yet unannounced portable, personal lifestyle devices. napster’s hard road napster — the legal, reincarnated music download site — essentially invented the concept of incumbent campus download services. they loudly{# } touted deals with schools “anxious” to stop the p p music sharing problem. trouble is, according to this story at the reg, it’s not working well. a survey at one client university paints a sad picture: not a single university of rochester student admitted to buying a song via napster during the fall semester. the high cost of metasearch for libraries i’ve been looking seriously at metasearch/federated search products for libraries recently. after a lot of reading and a few demos i’ve got some complaints. i’m surprised how vendors, even now, devote so much time demonstrating patron features that are neither used nor appreciated by any patrons without an mls. recent lessons (one, two, three) should have made it clear that libraries need to conform to patron expectations of how online resources should work. bstat features update: bstat has been updated. bstat is a hit and search term stats tracking plugin for wordpress. in addition to reporting lists of popular stories and popular search terms, it will report recent comments and a unique “pulse” graph showing the activity for a story or the entire blog over time. the documentation for the current release (b , as of july , ) explains the public functions and their use. i believe they reveal themselves in their names, so here’s a list of most of them: make my xb a low rider team pneumatik’s faq addresses the question “why do i need air suspension” simply: “because you wanna be cool!” and now, with pneumatik’s forthcoming kit, scion xb owners can be cool too. thing is, based on the photos it just doesn’t have the same effect on an xb as it does on, say, a caddy. braving home jake halpern’s braving home (also in softcover) easily took my interest. here’s how john moe described it for amazon.com: as a cub reporter at the new republic, jake halpern earned the unofficial job title of bad homes correspondent. braving home tells his stories of places where people really ought not live and the people who live there anyway. halpern traveled to such inadvisable destinations as a bed and breakfast at the foot of an active hawaiian volcano, a north carolina town trying to recover from being completely submerged, an indoor alaskan city, and an island in the gulf of mexico located directly in the cross hairs of numerous hurricanes. bstat beta release update: bstat has been updated. beta never went public. this is beta . changes this documentation supersedes any previous documentation. the bstat_pulse() function has been improved and now uses your css for appearance. call bstat_pulse_style() to add my default styles inline if you don’t want to modify your css. also, bstat_pulse() now has two switches to control what it displays. please take a look at the usage guide below for how to call this function now. libdev launched libdev launched today. from the welcome message there: libdev is a site for those interested in libraries and networked information. want to find a way to apply tags or social bookmarking to library content? interested in how wikipedia can serve libraries? want to find a better way to do patron loads or talk about what identity management means to libraries? looking for single sign-on solutions so patrons can move seamlessly from the campus portal to your opac without re-authenticating? idaho politics earlier this year the idaho legislature passed a bill recognizing the success of napoleon dynamite, a film about idaho life by idahoan native sons. legislature of the state of idaho first regular session – house concurrent resolution no. stating legislative findings and commending jared and jerusha hess and the city of preston for the production of the movie “napoleon dynamite.” be it resolved by the legislature of the state of idaho: the struggle to protect democracy in florida my dad, who’s called florida home for quite a while now, emailed me the following about goings on there: the big news here is the struggle to prevent volusia county adopting the the diebold touch screen ballot machines. they are bad news, because these diebold machines do not leave a paper trail and so a manual recount of a disputed election is impossible. the republican leaders of florida, who take pride in their deviousness, are trying to require the adoption of these machines under the guise of providing an accessible voting system for the handicapped, especially the visually impaired. happy birthday, popsicle npr’s food essayist bonny wolf reported yesterday on the th birthday of the popsicle{# } for weekend edition sunday{# } (listen in realaudio). like so many brilliant inventions, it happened by accident in . and through a century of change, it remains a consistent american icon, stick and all. it all started, apparently, with a forgotten bottle of soda pop with a stick in it and an unusually cold night. when is principality of sealand’s independence day? principality of sealand is a wwii-era gunnery platform — called roughs tower — in the north sea, outside britain’s pre- three nautical mile claim of sovereign waters. founded by roy and joan bates in , over time, roy wrote a constitution and named himself and joan as prince and princess. the wikipedia article on sealand tells the story of the world’s smallest micronation about as well and evenly as might be possible, but sean hastings’ website offers a more gripping tale. cannon aerial tramway it’s hot in new hampshire, but on top of cannon mountain, feet about sea level, it’s a little cooler. it’s an easy enough hike, but the aerial tram will save you the sweat. the current tram was built in and replaced the tram. the foot climb from the base takes a mile of cable each way, and the two cars make a trip every fifteen minutes. google maps rock, the google maps api rocks more we don’t need to hack google maps anymore. now that google has released a public maps api, we can make more reliable map-dependent apps (which will now have better browser compatibility, thank you). within a few minutes of signing up for a maps api key i had put together the following of the nevada test site tour. yeah, click the satellite button, scroll, zoom… it’s real. the api is all javascript, but i use a bit of php to iterate through an array of points and generate the code that puts the lines and pins on the map. photron makes my favorite video camera photron’s apx-rs video camera{#kingofhighspeedvideo} can capture , frames per second at top speed, and it can get megapixel+ resolution at , frames per second. it’s one of a dozen or so cameras in photron‘s lineup that can shoot very, very fast video. how fast is a thousand frames a second? how fast is several thousand frames a second? numbers alone do a bad job of telling that story. that’s why they did up this set of sample vids… color picking i needed to pick some colors for a new website recently. i’m color blind, so that complicates things. thing is, color relationships can be defined mathematically and “good” or “bad” color combos can be selected by a formula, so it possible to pick colors that go together without actually being able to see them. i’ve done this color math manually for years, but i went looking for a piece of software to make it easier. wordpress’ is_x() function an entry at the wordpress support forums{# } gave me the list i needed. how do they work? “you can use [these] in a conditional to display certain stuff only on [certain] page[s], or to omit certain stuff on [those] page[s].” here’s the list: is_ () is_archive() is_author() is_category() is_date() is_day() is_feed() is_home() is_month() is_new_day() is_page() is_search() is_single() is_time() is_year() so there you go. freight elevator quartet jazzmusique (rss, stream) treated me to freight elevator quartet‘s so fragile (from their becoming transparent album) not long ago and i liked it enough to take a note to look them up later. the band released five albums between and , but seems to have disappeared since. their site is still alive, and most entertainingly, has fan remixes of svengali (also from becoming transparent) available for download. my favorite is the version by absinthe & shiftless. alcohol knowledge test i just love tests (previously: psychotic, leadership style in movies and famous people in history, and eccentric or autistic), so i was quick to try myself at this one when al emailed me. it’s about alcohol, and like most tests, it’s not about getting the answer right, but giving the answer that the test writer wants. so it’s flawed, but it’s a bit of fun. here are my results: score: bacardi sending sms messages my friend will was in meetings all day friday, and there are few better times to have sms messaging than in meetings. thing is, i didn’t want to type on my phone’s numeric keypad when i had my computer in front of me, so i went looking for the details of this old hint that describes how to send sms messages with ichat (would also work with any aim client). regex reference regular expressions are a pain. jan goyvaerts’ regex reference helps. in a related tip, the following will eliminate any non-numeric components in a string: ereg_replace(“[^ - ]”, “”, $string) . i guess i’ll have to admit that i’d not used the exclusion operator before (the carrot immediately following a square bracket). now i know. geotagging gets a new meaning who doesn’t love tagging? no, tagging as in annotating, not graffiti. anyway, rixome is the latest among a bunch of plans/projects to enable tagging of geographic spaces/real-life environments. the good people at we make money not art had this in their post: rixome is a network and a tool that turns mobile screens into windows that show the virtual and public dimensions of our point of view. a walker (a rixome user) can see on his/her mobile phone/pda/laptop screen the virtual interventions that have been added to the location where s/he now stands. art deco hair daniela turudich knows vintage fashion. her books include not only hair, but how to recreate a vintage wedding, vintage recipes and candy making, and beauty secrets of history’s most notorious courtesans. here’s the description from art deco hair: art deco has long been associated with uncompromising style and sophistication, and this guide to recreating the sassy, controversial styles of the ’ s and ’ s offers a glimpse back at the hairstyles of this era. oooms design ist sehr gut guido ooms has some pretty neat ideas. engadget got high on his anti gravity machine (you must watch the video), but there’s a lot more to see. i wish i could link to examples of his furniture, bottle holders, personal transportation devices, or dohickies, but his flash-based site won’t let me. his glassbulbs are pictured here, but go visit the oooms site and click on the “products” link to see more. how to measure the tallest building zach likes tall buildings. perhaps it relates to his superhero obsession (leap giant buildings in a single bound and all), but it’s undeniable that he likes them. here, he gushes about the details of what makes a tall building and how it is measured. judging can be to the top of the highest occupied floor, top of the roof, architectural top (including spires), and top of mast or antenna. of course, the building must be freestanding and habitable too. culture of entertainment i don’t remember how i found this tip to baitcar.com‘s collection of police videos of car thefts. they’re good for a few laughs, but things like this — and about half of the programming on spike{# } — make me wonder how far we are from from the worlds depicted in running man and so many other stories. eh, at least we’ve got bravo. that’s some good tv. least wanted i’m entirely captivated by mark michaelson‘s collection of mug shots on flickr. it’s titled “least wanted” and he notes with little fanfare that they’re “nobody famous.” some of the photos contain little histories, like this set from the s and s that includes conviction details — “ days w. h.” for “selling obscene literature.” another image shows rapid aging over a three year period starting in . it’s part of a small collection of recidivist women of the s. overheard in the library “i want all the books that i’m interested in on one shelf.” making zip files on mac os x everybody else may know this, but macos x includes the command-line utility to make windows-compatible zip files. it works a lot like tar, but without needing any switches. > zip {target file} {source files} big brother gets more eyes engadget yesterday had a story about the mobile plate hunter , a device that mounts on police cars and scans to license plates an hour. more details are in the wired news story, where la county police commander sid heal notes that the system is hands-off: “it doesn’t require the [officer] to do anything.” the plates are automatically checked against a database of stole cars, and the patrolling officer when the system finds a match. switching hosting providers i’ll be switching hosting providers this week. at some point i’ll have to turn off the comments here so that i can synchronize the database and prevent loss of comments as the dns changes propagate. **update: ** the switch seems to have gone well and the dns changes have propagated to the networks i’m using. comments are on again. that’s the way it’s supposed to work. bstat beta release update: bstat has been updated. i’ve finally added a clean admin interface to my bstat wordpress stats tracking plugin and cleaned up the code for release as a public beta. quick start installation download and unzip bstat.zip place bstat.php in you wp-content/plugins directory place spacer.gif in your wp-content directory log in to your wordpress admin panel and activate the plugin, then visit the new bstat submenu of the options tab. what makes ohio red it’s a story that won’t die, and yet it can’t get any attention. since november rd, reasonable people have been wondering what happened. on election night, exit polls predicted a million vote win for kerry, but the official election results declared bush the winner by million votes. we’re all suspicious of polls, but an million vote discrepancy is big and exit polls are considered the most accurate of all. north-country drive-ins the fairlee drive-in theatre is open with double features on weekends details: route , fairlee vt (one mile north of town) - - notes from driveinmovie.com: the usual hotel/motel concept of in-room movies is cable tv, this is one of only two drive-ins in america that have a motel on the premises with a view of a drive-in movie. all rooms have a picture window and speaker, allowing motel guests to watch the movies. squirrel decanter and other dead animal art the strange folks over at custom creature taxidermy arts have come out with a squirrel liquor decanter that’s making the rounds. jon said simply “words cannot describe.” but the good folks at gizmodo assure us that “anyone who sees you sucking on the desiccated neck of an ex-squirrel will know you are a man of class and style.” other items in their novelty selection include flying squirrels and punk rock squirrels. american reporter’s nagasaki story emerges after years of censorship george weller won a pulitzer prize, a polk award, and was named a neimann fellow during his fifty-some-odd year career during which he covered much of europe and asia for the new york times and chicago daily news. weller died in at age , leaving behind a body of work that tells much of the th century’s events. his story about an appendectomy performed by navy pharmacist’s mate wheeler lipes in a submarine feet below pacific waters amid the concussive blasts of depth charges is legendary. the difference between progressive and conservative bloggers david rothman points to a daily kos story that points to a mydd story titled “aristocratic right wing blogosphere stagnating.” what’s the point? of the top political blogs, more than half are ‘liberal,’ and more importantly, they support community involvement — including basic features like comments — that the conservative blogs shun. of the five most trafficked conservative blogs (over , page views per week), only one […] even allows comments… google print: reports from michigan & oxford i’m listening and watching along with the educause online presentation from the universities of michigan and oxford and their participation in google print. presenters: john p. wilkin associate university librarian library information technology and technical and access services university of michigan reginald carr director of university library services and bodley’s librarian university of oxford google print is old news by now, but it’s interesting to get their reports on it. geolocating the news last week i got excited about the as-yet unreleased geolocation api for bbc backstage. now larry d. larsen of the poynter institute is excited too. in a post titled the future of news (… hint: gps){# &aid= } he talks about putting news in geographic context with geolocation tags. eventually, clicking an article in a news/google map hybrid might zoom in to a d model of the area where an automatic pop-up starts playing a slideshow with pictures of the scene or streaming video along with the text news content. blogger’s legal guide copyfight is pointing to the eff‘s new legal guide for bloggers. most of the content is about liability, but it also addresses issues of access and privilege that are generally granted to journalists, election law, and labor law. from the introduction: whether you’re a newly minted blogger or a relative old-timer, you’ve been seeing more and more stories pop up every day about bloggers getting in trouble for what they post. when you don’t have a gps… geolocation by gps my be the most straightforward approach, but we mustn’t forget the other ways to get lat/lon coordinates. all current cell phones support agps positioning to comply with federal e- mandates, but not all phones make it easy for the user to get that information out of them. still, some do and gps-enabled moblogging is becoming common in asia and europe, and there’s at least a public proof of concept going in the us. the mystifying aroma of rot i love libraries, and i love books, but there the needs of our students and limitations of our budgets have no room for misplaced romantic attachments. that’s why i’ve found myself paraphrasing something from ibiblio’s paul jones (via teleread): that smell of an old book, that smell of old libraries? that’s the smell of the books rotting. we must remember that libraries catalog and share information and knowledge, not books. pinball wizard gets his due the laconia citizen{#/ /citizen / /- /citizen} reported today that ron mowry’s year quest for recognition as the real pinball wizard of has finally achieved some success. the twin galaxies official video game & pinball book of world records will credit mowry’s hour minute marathon pinball session as a record. mowry set his record at a sandwich shop in hallandale beach, florida, but he was raised in plymouth, nh, where he now works for the university. bstat pulse i imported the content of my old referrer tracking database as hits in my new bstat stats datatabase so i could have more data to work with. i mixed this with a fairly simple graphing routine and now we can see the “pulse” of the whole site and each story. take a look at the bottom of the main page and between the body and comments in the single story pages to see what i mean. bstat progress i’ve been hard at work on my bstat stats tracking plugin for wordpress and you can see the results in the sidebar and in the story views here. the work has been made especially easy because of the great documentation, including writing a plugin, plugin api, and related pages at the wordpress codex. i’m testing the plugin with a limited group now (thank you sandee and cliff). but with a few more tweaks and a little more time to prove itself, i think it will be ready for an open beta. professionals don’t use ofoto or wal mart photo services at least that’s the only thing a person can conclude from the stories at copyfight earlier this week. this post reports on two stories where the photo services concluded that the photos to be printed were too good to have come from an average customer. upon trying to order prints of her child, one ofoto user found the following: your order has been cancelled because it appears your order contains one of the following… . bbc backstage is gonna rock (once they release the apis) the apis aren’t yet out, but the bbc has already won me over with their backstage bbc concept. of course, i’m a fan of anything with an api, but the real deal here is that it appears they’re planning on releasing a “query by geo-location data” api — and i’m all a gaga about about geolocation. i’ll definitely be looking to see what takes shape across the pond. damn pngs in internet explorer i don’t know why ie has never displayed my transparent pngs correctly, but i know now that i’m not the only one with this complaint. bob osola (name?) shares my frustration, and better, he sat down and coded a solution, shared the code, and posted a wonderfully informative guide to the problem. not sure if your browser can display transparent pngs properly? follow that link for examples. the google economy vs. libraries roger over at electric forest is making some arguments about the value of open access to information. hopefully he’ll forgive me for my edit of his comment (though readers check the original to make sure i preserved the original meaning): …keep the [information] under heavy protection and you will find that people ignore this sheltered content in favor of the sources that embrace the web and make everything accessible… [open and accessible resources] will become the influential authorities, not because they are more trustworthy, or more authoritative, or better written, but because they are more accessible. what? i’m not sure what to think about steve j’s wwdc announcement (video stream) of apple’s switch to x processors. coverage at macnn, mac rumors, ars technica, etc. i’m not sure, but it would be easier to take if i wasn’t the only one who saw conspiracy in it. does this relate to intel’s recent shoehorning of drm onto the cpu? it wasn’t long ago that i was praising apple for making devices that served the remix world that exists in the void between fair use and copyright infringement, but moves since then have concerned me. on the media does copyright issue i had just sat down to post a note about an interview with j.d. lasica in on the media (listen to mp ) this week when i found david rothman beat me to it. the interview was one of the better treatments of copyright issues that’s i’ve heard/seen in the (relatively-) popular media. here’s the summary from the otm site: for every move that media industries have taken to protect their copyrights, there has been an equal and opposite countermove by consumers. doggy and you: mark schutte’s dog powered scooter engadget has a link to mark schutte’s dog powered scooter. this catches my eye because my friend joe is always looking for ways to exercise his sled dogs in the summer. the developer, of course, is very serious about its befits and usefulness of this contraption. here’s the sales pitch: focus your dogs energy and enjoy the new sport of urban dog mushing. engadget has some complaints, but this looks like the best solution i’ve seen yet for running sled dogs in the summer. remixing reality: good or bad? we’ve all seen the ads they digitally insert on the field during football games and we’ve heard talk about inserting new product placements as old tv shows play in syndication. ernie miller has been thinking about this recently. last week he noted that folks are creating ipod-able, independent audio tours of museums. “…hack the gallery experience, […] remix moma!” commands artmobs, one of the groups producing these unauthorized audio tours. ohara fireflies i don’t consider myself a japonophile, but i do find myself reading mainichi daily news each day, and when they put up a picture like this, of fireflies near the yamada river in ohara, (chiba prefecture) i can’t help but notice. teleread spends morning on portable computing stories …well, not entirely, but i couldn’t help but read the posts on the pepperpad and history of the newton. i’m a fan of computing devices that don’t fit the mold, so i eat up stuff like this. i noted the pepper pad previously, and written a few posts about the newton and ultra-portable computing. update: engadget is getting in on the excitement too. they’re pointing to this osopinion article that’s at the center of it all. wikipedia and libraries wikipedia seems to get mixed reviews in the academic world, but i don’t fully understand why. there are those that complain that they can’t trust the untamed masses with such an important task as writing and editing an encyclopedia, then there are others that say you can’t trust the experts with it either. for my part, i’ve come to love wikipedia, despite having access to eb and other, more traditional sources. disobey gary wolf wrote in the june issue of wired about how smart mobs in new york’s world trade center outbrained the “authorities” and enjoyed higher survival rates because of it. wolf is talking about the nist report on occupant behavior, egress, and emergency communications (warning: pdfs). there’s also this executive summary and this looks like a mind numbing powerpoint presentation (also pdf). so, what about it? for nearly four years – steadily, seriously, and with the unsentimental rigor for which we love them – civil engineers have been studying the destruction of the world trade center towers, sifting the tragedy for its lessons. japanese government employees extremely troubled by summer casual dress code today is the first day of summer, according to japan’s environmental ministry, and that means it’s time to take off the ties and suit jackets and put on “casual” clothes. the ministry has been leading a charge to reduce energy consumption and ease global warming by asking all government employees to leave their neckties at home so they feel cooler with less air conditioning. but despite endorsements from prime minister junichiro koizumi it might not be going as well as planned. take a picture, get hassled by the man alan wexelblat at copyfight pointed out this story that talks about increasing limits on public photography. if you’re standing on public property, you can shoot anything the naked eye can see, explains ken kobre, professor of photojournalism at san francisco state university and author of one of the seminal textbooks on the subject. …but that apparently doesn’t stop security guards, cops, and others from intimidating and sometimes arresting those who try it. theme change… theme change not yet complete, but looking good. it’s a widened version of clemens orth’s relaxation_ column, itself a derivitive of john wrana‘s two columned relaxation theme. i found it on the wordpress codex, and though it was among the first group i looked at, i dutifully clicked through to every other three-columned theme listed there. anyway, expect the banner to change, and i’m working on how i want to handle the width on smaller monitors (where “smaller” actually equals anything narrower than px). bad movie, verboten subject? i’m embarrassed to be in the middle of fantasy mission force, a kung fu movie that demonstrates a brand of asian humor that i haven’t yet learned to appreciate. i’m watching it because i’m a sucker for jackie chan flicks and netflix makes it too easy to queue up bad movies. david chute wrote the amazon editorial review: jackie chan makes a brief guest appearance in this surreally goofy action comedy, a high-spirited shambles from that hovers awkwardly somewhere between monty python and the three stooges. global threats, as seen through eyes of movie producers and insurers jonathan crowe points out this risks in global filmmaking map by aon, the entertainment industry insurance company. go view the pdf or a full-size png{# &size=o} for all the details. lunch at burdick’s treated mom to lunch at l.a. burdick’s in walpole today. the food at burdick’s is always remarkable, but this time i got a decent photo of it. i’m calling the plate in front a real tuna salad. yes, those are strips of medium-rare tuna, but it’s the pickled onions that delighted me. in the middle is my rare steak with a dollop of stilton butter. for desert, we enjoyed a frappes and shared a piece of hazelnut-orange cake while thunder and large hailstones menaced the street outside. wordpress stats goodness work on my bstats plugin continues. i’ve added recently commented posts tracking, begun work on a usage graph, as requested by richard akerman, and put together an interesting way to track usage of the google ads. i’m using the google ads to figure out how to best use them on another project later. i think they look a little too commercial here too. i’ve done nothing yet to created a list of related posts, and i’m still researching how i want to do referrer tracking. of wordpress tags, keywords, xml-rpc, and the movabletype api wordpress’s xml-rpc support looks pretty good. heck, it supports a half dozen apis and works well with ecto … except for tag support, which is my only complaint with it so far. the movable type api supports a “keywords” field that i’m thinking can be hijacked as a “tags” field instead, but while ecto sends the goods — i can see them in the xml-rpc data that gets sent out, wordpress seems to ignore them upon receipt. bstats plugin i’m more than surprised that there’s no (decent) stats plugin for wordpress, but that hasn’t stopped me from writing me own. it’s called “bstats,” and i’ll release a beta soon. in the meantime. the “today’s most popular” list comes directly from this new plugin. one step forward… i thought i was real smart when i modified the tags plugin to support integration with technorati. the code was simple, just look in the tags.php plugin file for the foreach statements that run through the tags names and turn them into links on the page and change the $tags[] = statement to look something like this: $tags[] = “tag_name).”\“ target=\”$target\“ rel=\”tag\“ title=\”more “.$row->tag_name.” at {site name}\“ >”.$row->tag_name.“ tag_name.”\“ target=\”$target\“ rel=\”tag\“ title=\”find “. nuclear test site tour the above image is my followup to my nevada test site tour post from last month and comes courtesy of adam schneider’s very useful gps visualizer (you really need to see it full-sized{# &size=o}, though). i still don’t have a cable to connect the ancient magellan gps i used to a computer, so i manually entered the waypoints i marked into the form and selected a few options, and viola. …and then you realize you wasted your life i think i’ve been avoiding commenting on this issue for weeks because it hits so close to home. first i read it in biblioacid, then jenny levine picked it up, then richard ackerman picked it up at the science library pad: library catalogs are broken, and there’s no amount of adding pictures or fiddling with colors that will fix them. i nibbled at the edges of this in my iug conference presentation, but i didn’t say it as well or as clearly as roy tennant did in his widely quoted april library journal column: vonage ceo interview makes me feel old engadget’s interview with jeffrey citron, chairman and ceo of vonage gives an interesting peak into the world of the baby bells, through the eyes of an upstart. citron dishes about the competition, stomping at&t, working deals with the bells to make services work, and a possible palm version of their softphone. most interestingly is his notions about what their customers want and expect. …more and more people are deciding that they don’t even want a land line in the house…? blog software switched i’m almost ready to call the first stage of my wordpress migration done, except it looks like the comment submission forms aren’t working. while i’m working on that, please note the new feed urls: rss . x, rss . , and atom. update: found a reference to the comment bug on the wp support site and in their bug tracking system. i didn’t find the answer there, though, so this is still a problem. switching blog software… i think i’ve finally decided to go to wordpress after all. i tried doing it too quickly last time and it almost worked, but i switched back when i realized i might need more than minutes to figure out how to use wordpress in production. since then i’ve found a set of plugins that do most of what i want, but it looks like i’m going to have to put together a stats tracking plugin of my own. crime and privacy on google maps annalee newitz last week posted a column on people’s fear of privacy loss as a result of google maps. her point: so while all these people are wringing their hands over how simple it is for strangers to discover the color of their roof on google, we forget that we can already be tracked everywhere we go using cell phones and the rfid chips in wal-mart backpacks. i honestly didn’t know people were up in arms about the maps and satellite images (which have been available elsewhere for years), and, like annalee, i’m much more concerned about the proliferation of real-time tracking systems like cameras, rfid tags in our driver’s licenses and consumer products, and other sensor technologies. eating my way through san francisco san francisco is a great city for a conference. it’s also a pretty good place to get lunch. the following is poorly written and incomplete. well, at least it’s something. sunday i was a little surprised to find johnny rockets on jefferson st. serving breakfast, but they did a fine sausage, egg, and cheese sandwich all the same. after visiting alcatraz, i had a delectable rueben at the buena vista on the corner of beech and hyde, where they’re known for their irish coffee. wasted minutes i can now say with the authority of experience that star wars episode iii sucked. update: zach’s right, my opinion of the original trilogy has fallen over time. but i stand by the statement that episode iii is worse than it should be. the real reason for the update, however, is to note a couple pictures of things seen and done while waiting in line: matt, with an oversized jug of generic cola and this oversized scorpion bowl. un food survey the proceeding was forwarded to me by my dad, who included a note suggesting that jokes may embody the only real truths we can know. a worldwide survey was conducted by the un. the only question asked was: would you please give your honest opinion about solutions to the food shortage in the rest of the world? though translated into appropriate local languages and delivered using local personnel, the survey was a huge failure. cool stuff made easy (rss, opengl d graphics, screensaver app) i have an appropriate fondness for engadget‘s how-to features, like today’s “make a customized rss screensaver in tiger.” macos x . tiger comes with a pretty decent rss screensaver (don’t miss the movie), which can be set to display feeds from any source that safari can read and bookmark. and if that’s all you want out of life, well then you won’t have any reason to leave your couch/chair/bathtub or wherever you use your mac. geolocating everything i’ve been excited about geolocating photos, blog posts, etc for a while. so this past month or so has been quite exciting. most recently, gps photo linker has been updated with mac os x . specific features: with spotlight in mac os x . , you can instantly search for the city, state and country information automatically saved by gpsphotolinker. additionally, mac os x . does support the gps metadata tags in photos. about that bookless ut austin library there’s a lot of talk about the new york times story about ut austin’s undergrad library throwing out its books. problem is, i don’t think it’s as exciting as people are making it out to be. first, the undergraduate library is one of libraries on campus and the real issue was space, not books. when priorities change, but you don’t have enough money to break ground on new buildings, you’ve got to re-use the old ones. flickr api the flickr api rocks. it helps that the developers are really excited about web services (pdfs converted from their original ppts). anyway, there are code libraries available for php , javascript and others. michael madrid’s oberkampf is a dead simple php library that looks easy enough for non-coders to use. and i found myself quite satisfied with the rest request format and the xml to array parser by eric rosebrock. do i want a lifedrive? after months of no news or no good news, and just as i’m about to knock palm news site src off my feeder, palmone starts leaking details of their lifedrive “mediacentric handheld.” then somebody leaked the whole datasheet, and src was there with the deets. engadget was on the story the next day, and summarized as follows: it’s . x . x . inches in size, weighs . ounces, runs on palm os garnet . markoff, i wish i could trust thee trouble: john markoff has been doing tech stories for the new york times since the beginning of days, so it’s likely he’s written something you’ve read and enjoyed. but he’s also written a number of wrong or counterfactual stories that he makes little or no apology for. at the core of the claims against him is his coverage of kevin mitnick, the accused cyber-criminal who was held for over four years — including eight months in solitary — without a bail or sentencing hearing. google’s war on hierarchy, alert the librarians via ernie miller i saw a link to john hiller‘s story about google’s war on hierarchy, and the death of hierarchical folders. googlization is a concept libraries have been strugling with for a while. and while it’s hard to say wether the change is good or bad, i can say that failure to change makes libraries irrelevant among patrons who’ve grown accustomed to google and other exemplary services. so john’s story caught my eye and had my full attention for a while. sunrise on mount monadnock i’ve loaded some more of my old photography, inlcuding this shot of sunrise on mount monadnock (info) from the spring of or . josh stands on the outcrop in the foreground. i held the exposure open longer than appropriate for true brightness and color, but i like the effect. other photos: another sunrise on mt. monadnock, photos from around harrisville, panoramas of the nevada desert and london, a rose, and a set of stairs. library portal integration i’ve been back at work less than a week now, and i’m already behind. i’ve finally posted the handout and slides (as a quicktime movie, pdf here) from our iug presentation. i’ll submit them to iug for their archive and add them to the plymouth state university library portal integration page in an update soon. as usual, presentation slides don’t stand on their own, but they should be helpful reminders of what was said. kwajalein atoll kwajalein atoll is a part of the republic of the marshall islands, lost in the pacific ocean (maptech makes it easier to find) along with more recognizable locations like bikini and enewetak atolls. the military presence is far from gone, however, as kwajalein is home to reagan test site, where the us army tests the last remnants of reagan‘s infamous star wars program. now reincarnated as george w. bush‘s missile defense, it survives despite its flaws and an unbroken string of failed tests. hilary rosen: sock puppet we’re all talking about hilary rosen‘s apparent about face, apparently pro-customer, anti-drm essay now (props to david rothman for taking the high road on this). in an update to his monday post, however, ernie miller notes that the riaa and hilary rosen’s history is that of blanket opposition to mp players (and fair use) in general. if the riaa had its way, there wouldn’t be any portable mp players. the only portable players you would be able to buy would play only drm restricted tunes. delicious, refreshing, old liquor bottles so grenadine isn’t officially a liquor, but it gets kept behind the bar and this one has a great label. the collection comes from the estate of a friend’s mother, who appears to have had a taste for old martini culture (not pictured are several bottles of vermouth). there’s more in my flickr photoblog. pointless, crude, badly drawn, unintelligent, offensive it’s a book review. it goes like this: pointless, crude, badly drawn, unintelligent, offensive. life-threateningly funny. buy this. another amazon uk customer wrote: funnier than the real people with tourettes the book is modern toss, by jon link and mick bunnage. cartoons and more info are online. when we can’t all just get along (the failure of logical centrism) i love the following quote from copyfight: frank field, responding to james boyle’s much–discussed ft column, deconstructing stupidity: “flat-earthers are harmless — until they start forcing you to write the specifications for your gps system in accordance with their views. then, you’re screwed.” and boyle’s column is pretty good too. former riaa head hates drm? today is sort of an anti-drm day here, so it was some pleasure that i just saw ernie miller’s post at copyfight regarding hilary rosen, the former head of the riaa. she’s complaining about the drm apple uses with its music store and ipod. she says: i spent years in the music business the last several of which were all about pushing and prodding the painful development of legitimate on-line music. give orphaned works a home david rothman at teleread is alerting us to something we should have done a long time ago, but, hey look, a caterpillar…. really, the us copyright office and library of congress are accepting comments to comments on the issue of “orphan works.” but, the deadline is today at pm est today! james boyle, addressed some of these questions in a column in the financial times recently: thomas macaulay told us copyright law is a tax on readers for the benefit of writers, a tax that shouldn’t last a day longer than necessary. broadcast flag smackdown the only thing that could have made friday’s news sweeter would be to have received the dc circuit court of appeals’ deciscion against the broadcast flag from the us supreme court instead. still, it’s enough to get most of the ip-aware blogsphere excited. to wit: here, here, here, and everywhere else. copyfight‘s synopsis was the best: the american library association, public knowledge, eff, et al. just won our joint challenge to the fcc’s ability to regulate consumer electronic devices that receive digital television signals, - at the d. t-mobile does coverage maps, verizon wireless baffled i’d like to make more of this, but it’s old news. we’re all sick of the “can you hear me now” ads, but that doesn’t stop verizon from talking up their network testing efforts. but when it comes to network performance, the ceo starts complaining about customers who expect their phones to work at home. what? yes. engadget reports: in an interview with the san francisco chronicle he asks, “why in the world would you think your (cell) phone would work in your house? time to change… time to rearrange… time to restore from backup… i’ve given up on my poorly timed and completely unplanned try at switching to wordpress. i started out thinking i’d experiment with it, then things got out of hand. factors contributing to my interest in wordpress: ecto via allforces.com a little compare and contrast with pmachine livesearch better rss/atom output flickr gallery a mostly functional pmachine importer a damn easy install a bunch of plugins factors that made me give it up for the short-term: what are you doing to shape the future of libraries? jenny levine recently posted a note about opacs and xml and maps wherein she makes two points: first, mike copley at north shore libraries in new zealand has been doing some exciting stuff to help patrons find books (go ahead, go there and click a “view map” link), then expands her post to address the struggles that folks like mike face to do some of these things. see, mike’s library system is converting to innovative (iii) soon, so the work he’s done is mostly for naught, as it’s very difficult to identify item locations with the detail he’s getting now. xml server applications well, it’s done. the [handout][ ] and [slides][ ] as presented are posted here, and i’ll add them to our [portal integration][ ] page (yeah, they’re sort of connected) when i return to [plymouth][ ]. the slides don’t stand on their own, but for those that were there, they should be helpful reminders of what was said what links we looked at. one of the attendees took me to task for recommending marc xml as the replacement for iii’s proprietary schema, saying that it fails to leverage the full value of xml. iii introduces “web works” where did this come from? innovative calls it “web works,” and describes them as “html-based interfaces for light-weight system access.” here’s the program description: webworks are new products that offer focused functionality for staff through a lightweight browser-based client. one web works client handles selection list processing while a cataloging client provides the ability to add and edit records. the session was hugely crowded, and i had to run off before i got to ask my question: “how do these fit in with any web services strategy iii may be developing? citing library collections on the web the example below uses a javascript to display bibliographic details about an item in plymouth state university’s library catalog. now imagine this link included information on the availability of the item, and a button to request or reserve it…. this post is intended to demonstrate how library catalog data can be used in places far from the catalog, perhaps in blackboard/webct, blogs, or elsewhere. i’m at the innovative users group conference, where i’ll use this post in my presentation on xml server, session l . iug : ldap is not single sign-on at innovative users group conference now. the most exciting thing today was using ldap authentication by john culshaw of university of colorado at boulder, and richard paladino of innovative interfaces. despite the title, the raison d’etre of the presentation was single sign-on, and the unstated hurdle was identity management. academic it departments are struggling with these two huge issues, but libraries often have even more limited it resources and are getting little help from campus it departments. prisoners of age at alcatraz found ron levine’s prisoners of age exhibit at alcatraz today. sadly, the website doesn’t appear give the prisoner’s stories, and, though the photos are well done, it’s the stories that hold our attention. leaving las vegas morning’s cold light shines harshly even on the strip, but this saturday morning on fremont street looks especially forlorn. i’ll be on a plane to san francisco for my conference in a few hours. golden gate hotel and casino according to the history printed on their diner placemats, the golden gate has been standing at the corner of fremont and main streets for years. kris had some good fun eating unhealthy quantities of cent shrimp cocktail at the gate. [update:] the stay wasn’t bad, in fact, i enjoyed the best sleep i’ve had all week. some were out trying to save souls, but i found fried twinkies. fatburger and henderson, nv my trip to henderson was a bust. i’ll eventually make a story about what i’d planned to do, but the only thing that worked out was a visit to fatburger in the sunset station casino. along the way i snapped this bad panorama of the vegas strip. the point here was to show the sprawl on what some are calling the city’s centennial. the shot goes better with the story i wanted to tell, but it fails even there. nevada test site tour toured the nevada test site today. no cameras allowed, but i did take along a gps and marked points of interest along the way. i’ll have to upload the track and landmarks when i get home, but google sightseeing has some interesting nevada destinations, including one for the test site area. but satellite photos can do little to show the human scale of things like the , foot wide sedan crater. waiting in long beach long beach airport is a small affair, seemingly more fitting for dubuque iowa than the south los angeles sprawl. gates one through three are in a pre-manufactured temporary structure that’s obviously been in use for some time, but the food from the one vendor is better than in boston and the queen mary spa offers massages hidden behind a partition in the corner. a five minute of scalp rub runs $ . beatnikside’s vegas photo gallery i can’t help but like beatnickside‘s las vegas flickr photo set. it’s one of the most photographed of cities, but these photos are fresher than that. sometimes enteraining, sometimes informing, the shots of vegas’s glitz and glamour show special attention to detail. this week is vegas week at maisonbisson, since i’m out here before heading to san francisco to present at iug . i have an inexplicable fondness for vegas. smart high efficiency car coming to us i got excited a while ago when i learned that daimler chrysler was bringing their little smart car to canada, and i’m even more excited now that i learn that it’s coming to the us via zap, a company originally formed to make and sell electric cars (zap stands for zero air pollution). though powered by a normal internal combustion engine, its small size and low weight allow it up to miles a gallon — much better than the . the long tail of violence it’s been a few days of “long tail” talk here at maisonbisson. stories about popularity vs. the long tail and aesthetics of the short head are just below. here’s one on the violence of the long tail. john robb at global guerrillas wrote about the “dark side” of the long tail in a march post to his blog. it’s a touchy one, so i’d better explain robb’s point in his own words: national weather service adds xml and rss feeds the us national weather service just updated the soap/xml interface to their national digital forecast database (ndfd) and rss feeds from their storm prediction center. i feel a little happier about paying my taxes when i see government organizations like the weather service posting answers like this: the national weather service is striving to serve society’s needs for weather information by evolving its services from a text-based paradigm to one based on making nws information available quickly, efficiently, and in convenient and understandable forms. tetris shelves gizmodo posted a picture and a little text about bravespacedesign‘s tetris shelves. more from bravespacedesign can be seen in this post at land+living. they’re all the standard tetris shapes constructed of walnut and ash. my previous attempts at cabinet making were miserable failures, but considering these shelves cost seven large — yes, $ , — it’s more likely that i’ll be making my own than buying them. question, though, am i violating copyright/trademark/patent law if i built my own for personal use? liblime/koha ils a comment to a post on the shifted librarian pointed me to the liblime collection of open source library applications including the koha ils. they’ve got demos for the whole collection, including the opac. it’s the first i’d heard of liblime or koha ils, but it’s good stuff and i certainly hope to see more of it. the dark side of networked information according to the website, mitre is: a not-for-profit company that provides systems engineering, research and development, and information technology support to the government. it operates federally funded research and development centers for the department of defense, the federal aviation administration and the internal revenue service, with principal locations in bedford, massachusetts, and mclean, virginia. all of this is interesting because blogsofwar points out that they’ve been presenting information on a project titled blogint: weblogs as a source of intelligence (with slides in pdf format): “short head” vulgarity and prurience chris anderson at the long tail blog quotes a passage from david foster wallace’s a supposedly fun thing i’ll never do again: tv is not vulgar and prurient and dumb because the people who compose the audience are vulgar and dumb. television is the way it is simply because people tend to be extremely similar in their vulgar and prurient and dumb interests and wildly different in their refined and aesthetic and noble interests. what is networked information? there’s data, then there’s information. information is meaningful and self explanatory, data need to be aggregated and analyzed before they become information. networks — ethernet and the internet — transmit data, but our web browsers and the back-end applications they connect to turn it into useful information. “networked information” is what results from building connections between multiple information sources. displaying an author’s biography inline with the library catalog holdings of books by that author is one example of how the value of information sources grows when they’re networked. credit where credit is due jenny levine’s mention of my work with innovative’s xml server wednesday drew a lot of attention, but there’s little online public discussion of innovative to give some of my comments context. innovative started started development on their xml server product quite a while ago (five years, yes?), before later standards like marc xml had any traction. they did it to create another very useful product, their airpac, a online catalog for mobile phones and handheld devices, and without any clear demand for xml server from customers. stanford library’s tech history collection i just discovered standford library’s collection of documents relating to the technology and culture in silicon valley and the development of the mac thanks to a link from gizmodo. gizmodo was excited about the <a href="http://library.stanford.edu/mac/primary/images/dayton .html” title="mice “wine tastings” “>mice “wine tastings” that apple did in its efforts to develop the first consumer mouse. elsewhere, however, i found this interesting little tidbit: reading it twenty years later, the most surprising thing about it is the amount of attention it gives to networking, and the degree to which the first macintosh was intended to be a kind of network computer. xml isn’t enough a lot of this is in my xml server presentation at the innovative users group conference in a couple weeks… jenny levine is an outspoken advocate for the use of rss in libraries. one example she cites is posting lists of new acquisitions to library websites. she estimates that folks in the libraries of her library system spend hours per year on that one activity, time that could be used elsewhere if automated by rss. new catagory: libraries & networked information thank or blame jenny levine of theshiftedlibrarian for this: i’ve just created a “libraries and networked information” category here. more to come. the long tail at maisonbisson content here at maisonbisson isn’t well focused, but a few stories have come out winners in the google sweepstakes of passing popular fancy. my story about a giant bear in alaska was one such winner, but i’m happy to see a few others are also getting read. my stories about stainless steel, the heat output of dell servers, and itunes vs. firewalls are obviously filling a need for technical information not readily available elsewhere. safari . supports for contenteditable wysiwyg melvin rivera reports on <a href="http://allforces.com/ / / /wysiwyg-comes-to-safari- /” title="safari . ’s support for contenteditable“>safari . ’s support for _contenteditable_. when decorum is entirely innapropriate it’s hard to find the words to introduce eric berndt‘s open letter to his nyu law school classmates. the nation said the following: justice antonin scalia got more than he bargained for when he accepted the nyu annual survey of american law’s invitation to engage students in a q&a session. randomly selected to attend the limited-seating and closed-to-the-press event, nyu law school student eric berndt asked scalia to explain his dissent in lawrence v. copyright and the internet david rothman at teleread linked to franklin pierce law center professor thomas g. field’s guide to copyright on the internet. field gives a clear overview of of the limits to copyright, the ways copyright applies to web sites and email, and the limited law on linking and framing web content. in his section on risks, he notes: copyright law precludes most uses of others’ works without explicit or implied permission. satelite imagery there appear to be two non-government-owned companies providing satelite imagery: space imaging and upstart digitalglobe (yeah, like they’re not both upstarts). digitalglobe is working hard to make friends with the media and regularly offers timely images of events, disasters, and wars to them. for the public, they offer some more scenic shots, like this one of the the boneyard at davis-monthan afb in tucson, arizona from august , . the boneyard serves as a holding place for out-of-rotation airplanes until their fate is decided; the dry, clear climate of tucson provides an ideal environment for the storage of aircraft, as they can sit indefinitely without rusting. focal plane shutter distortion henri lartigue’s photo of a race car shows one of the wonderful ways in which the camera records its own reality. spectators lean left while the speeding car tilts right all because of some facts about how his camera works. lartigue’s camera had a focal plane shutter, a two-part light curtain that slides to one side to expose the film while the second part follows a moment behind to again block the light. jeffrey veen gives presentation advice in seven steps to better presentations, jeffrey veen acknowledges the complaints against powerpoint, but explains that the real problem is “bad content delivered poorly.” his seven points have a lot more detail that what i’m quoting here: tell stories. show pictures. don’t apologize. ever. start strong. end strong too. stand. away from the podium. pause. my own opinion is that veen and tufte would agree more than they disagree. tips to flag designers (vexillographers?) the folks at the north american vexillological association get excited about flags. yeah, i had to look up vexillology too. anyway, they’ve got a page how-to about designing a flag, for “your organization, city, tribe, company, family, neighborhood, or even country!” their advice centers around these five rules of flag design: keep it simple use meaningful symbolism use - basic colors no lettering or seals be distinctive or be related each point is supported by examples illustrating both the “right” and “wrong” way to do it. cat and girl makes me laugh i can’t get enough of cat and girl and this one just hit my funny bone. thinking of comics, comic life makes it easy to lay out your digital photos and add comic-style speech balloons. looks interesting, though i’m not sure it’s worth $ bucks. geolocating everything i just added jonathan crowe’s the map room to my daily read. it was there that i learned that geourl is back, and that’s got me thinking about geocoding things again. i spoke of geolocating photos in a previous post, but my interest has broadened. i now want to geolocate my blog posts, i want lat and long recorded with my atm transactions, i want my emails and phone calls to have location information. urls i need to bookmark on my clie and phone google local for mobile devices may be the most useful thing yet. but then, i’ve been slow to get even the regular google search for mobile devices bookmarked. see, when the president does it, it’s different, somehow it’s a reasonable story: guy gets ipod, buddy puts a few favorite tracks on it, everybody jams happily because they can share their little bits of culture. in a way it’s an extension of the mixed tape so romanticized in high fidelity, but in another way — the riaa’s way — it’s probably a copyright violation. this is about the time you’d expect me to announce a new round of charges from the riaa, more claims of theft and lost profits due to the scourge of technology and hordes of uncaring, music copying punks. modern day opium craze in a story in the sacramento news and review, peter thompson writes about his drug use. at he tried making mead, but when that failed he continued to look elsewhere: i began to see the supermarket and drugstore as potential drug dealers. i drank bottles of cough syrup before i knew what dextromethorphan (dxm) was. i ate catnip and didn’t feel anything. i ate nutmeg and felt everything. there was no internet to guide me and nothing in the library about morning-glory seeds. apple finally unleashes tiger apple announced the availability of mac os x v . tigertuesday and is now accepting pre-orders. the product is to be in stores on friday, april (beginning at pm?) and will sell for $ , or $ for the mac os x v . tiger family pack, a five seat household license. amazon is offering tiger for $ , after rebate, though the rebate doesn’t appear to apply to the family pack. apple’s been selling family packs for a while, but it’s added some new family features to the os that surprised me. our underequipped military forces a story over at defensetech is reporting that four years after the september th attacks and during a time when us personnel are involved in armed action on the ground in arabic speaking states, the military still doesn’t have a plan to train their soldiers in the language. it seems the pentagon can spend bazilions on failed missile defense systems, but hasn’t the money or interest for language instruction. i’d say get the folks in green some ipods and in flight arabic, or the more extensive pimsleur quick & simple arabic (hey, the amazon reviews for it are positively glowing), but i’m thinking both lack important vocab for people who have to deal with car bombs regularly. most cmss suck i’ve been slowly struggling with the question of how to replace pmachine, my cms engine here. i haven’t really liked any of the alternatives that others i know are using (link link link link), though i’ve been hard pressed to identify exactly what my complaints are. among the points in making a better open source cms, jeffrey veen names a few of the most frustrating for me: hard-coding of site layout in the cms, mixing of content with site administration in the interface, and, sometimes, lax security. who doesn’t want a caboose? perhaps it’s the lasting effects of watching the station agent too many times, but i went looking for a place to buy a caboose. they’re big; as much as ′ long, ′ tall, and ′ feet wide. and they’re heavy, perhaps tons. but they can be moved on roads via big trucks and cranes, but then, they also move brick houses. caboose disappeared from the railroads in the s, after about years of service. molecular visualization in mac os x a while ago i went looking for alternatives to mdl chime on mac os x, as mdl is still choosing not to support os x. sure, you can run it in netscape .x in classic mode, but that’s getting increasingly frustrating. what’s great about the mac, however, is how many great solutions there are from small developers who take on the “big guys” and do it better. evidence: piotr rotkiewicz’s imol. declaring bankruptcy on old stories i often use the maisonbisson blog as a sort of annotated bookmark list, keeping track of the things that catch my interest for one reason or another, things that i’d like to return to or share. but i often get ahead of myself in identifying the things i’d like to look at further and never get around to posting an annotated link here. for those, i’ve been keeping a text file with urls that i’ve sometimes revisited and sometimes posted stories on, but the list is growing, and it’s becoming clear that i won’t ever get to around to posting stories for most of the urls there. does size matter? a while ago i asked a friend why short sentences were so pleasing to read and write. he had no answers, but agreed that brevity is its own reward. some (though i can find no reference to it) suggest that technological developments have changed and simplified sentence structure by allowing writers to write and revise freely, while typewriters and pens required forethought and concentration to avoid scribbling out unwanted, half-formed sentences. verizon wireless’ wardriving rig (can you hear me now?) it turns out that verizon (and all the other carriers, presumably) really do go around asking “can you hear me now?” the actual test conversation sounds different (possible source?) and the testing is automated, but there really are people out in the world doing real coverage testing. i guess i naively assumed that it was all theoretical and computer modeled, or something. anyway, mobiletracker rode around tampa, fl, with a verizon wireless test guy levy rippy back in february: of bricks and progress… this post is about a couple of things. first, it seems cory doctorow has issued dmca takedown notice to the folks at boringboring.org for their parody of doctorow’s boingboing. what nobody knew at the time is that gakker has also been on the scene, doing doctorow parodies, and all. which is where thing ™ comes in: this post about bricks highlights an ongoing concern of mine. what is the real difference between a long-existing thing with a variety of uses, some of them illegal, and the thing not yet developed with a variety of uses, some of them illegal? the riaa’s logic and ‘declining’ music sales blogger mark cuban listened politely to riaa chief mitch bainwol stumble into the logically fallacious argument that: it was obvious that illegal downloads were hurting music sales. it was obvious because the advent of file sharing coincided with a decrease in music sales. therefore a lead to b. (i’m quoting cuban, who’s parapharsing from <a href="http://www.ce.org/events/event_info/downloads/industry_leaders_react-ip.pdf” title="bainwol’s cea blather speech”>bainwol’s cea blather speech). but instead of arguing with bainwol’s logic — it’s too easy, and too many others are doing it — cuban is using it to prove the contrary. archiving realaudio streams on mac os x standard players for rtsp streams like those for realaudio don’t cache the files they download, meaning they require a net connection to operate. i found an ezboard forum message that identified hidownload, net transport, oep-oee and streamdown — windows-only applications that can download rtsp streams and save them to a playable file. but those trick ponies do nothing to help mac users. audiohijack has been around for years now, but it only captures the audio stream as it leaves realplayer and heads off to your mac’s audio output. gas prices (finally) affecting car sales? a mainichi daily times story announced today sales of energy-efficient japanese cars soar in u.s. toyota and nissan both saw % sales growth, with toyota’s prius sales jumping to % their numbers from a year ago. honda, which usually wears the energy efficiency leader’s hat, saw a nearly % increase in sales. ever prideful, mdn notes: in sharp contrast, the sales of new cars sold by general motors and other american automakers in march posted decreases from a year earlier. tator-tot pizza so my challenge is to prove that i can be both trite and serious in the same day. here, tom chows on tator-tot pizza with ranch dressing and chipotle chile tabasco sauce. it’s part of the tator-tot pizza set at flickr. there’s no good reason to make tator-tot pizza, but we had both, plus all the sauce, so what else is there to do. that’s trite, this is serious. serious saturday i’ve lost my way a bit and been posting a bunch of trite stories here lately about my kitchen and in my photoblog. i’m sorry. i have made a few attempts at serious discourse. if you look carefully you’ll see stories on grokster, rfid passports, a library conference, a chilling look at the death penalty in texas. looking a little further back, you’ll find new stories in the very serious copyrights & intellectual property and politics & controversy categories. can you eat it? food bets seem harmless, but they look funny. everybody likes the old “can’t eat four saltines in seconds” bet, and it’s likely that many of these foods would never get eaten except on a bet. then there’s the story of two guys who took a bet they could eat ramen noodles — only ramen noodles — for a month. it’s probably apocryphal, but they story ends with them getting scurvy and giving up. it’s friday! over at caravie: peace, nonviolence and conflict resolution i found the lies, lies, lies, lies, lies, lies, lies, lies, lies, lies, lies, lies, lies, lies, lies, lies,lies, lies, lies, lies, lies, lies, lies, lies music video. also at caravie i found a link to this ‘zine, with a selection of videos, like this one. it’s a perfectly enjoyable way to waste a friday afternoon. [update:] this is confusing. new us passports will serve as terrorist beacons i cannot say it any better than it was said in today’s issue of effector: the us state department is pushing for what may be the most misguided and dangerous travel “security” plan ever proposed: putting insecure radio-frequency identification (rfid) chips in all new us passports. these chips would broadcast your name, date of birth, nationality, unique passport number, and any other personal information contained in the passport to anyone with a compatible rfid reader. reporting late on grokster these things take time and can often be hard to read, so while we all wanted the high court to look at the entertainment industry lawyers and tell them to take a hike tuesday, we’ll have to wait until summer to know what actually went down. but there is one interesting thing so far… it was in nina totenberg’s wrap-up for npr that alerted me to this turn in the arguments: life of a kitchen blueskygirl alerted me to the life of a kitchen group at flickr in a comment on a photo of my remodeled kitchen. so, of course i joined and had to upload a pile of related pictures from my back-file. there’s some great stuff from a bunch of contributors up there, despite the trash i tossed in. in the photo above, sandee makes homefries for a brunch with our neighbors back in july . cheap lcds for in-car-computers a powerpage story alerted me to a couple of inexpensive touch-screen lcds: innovatek and lilliput. take this as an update to my story on carputers. that story, of course, connects with mobile carrier networking (with followup), and gps. kitchen it was done in quite a rush and there’s some touchup to do yet, but our kitchen is now more complete than it’s been in six years. late notes from october library conference i just re-discovered my notes from dartmouth biomedical libraries’ october conference for and found a number of things i wish i’d remembered earlier. academic libraries are facing declining use/circulation of traditional materials (books, print periodicals, fiche, etc). it’s not that students and faculty don’t care about libraries or learning, the problem is that libraries aren’t serving their patrons with the forms of information they need at the time and place they need it. considering the death penalty texas executes a lot of people. during the years through , texas executed inmates, making then governor georg w. bush the killingest governor in history. a march amnesty international report titled the death penalty in texas: lethal injustice notes that “public support for the death penalty in texas remains strong,” and a later news release states “texas is so proud of killing people that it issues press releases for the executions it carries out. choppin’ ice corey chops ice from my walkway on sunday afternoon. dinner went well, despite worries that our new kitchen wouldn’t be completed in time. i guess i’m a huge fan of pictures with particle action. here’s another, where will cuts it up with a circular saw. crunch: three more days there are at least two ways to appreciate easter: to some it’s the most important religious event of the year, while, to others — your hosts here at maisonbisson, for instance — it’s yet another good reason to gather friends and family ’round a table and celebrate good food, good wine, and all that makes us human. but there’s a problem: we dismantled our kitchen last week in anticipation of our new kitchen…which is taking longer to install than i expected. the risks of googling one’s self well, actually it was a , but the results are just as scary. there’s a fellow named gerald dewight casey on deathrow in texas and an asian language site has a picture of the bisson battlesuit. wifi my world i’m in hooksett today waiting for the my kitchen cabinets to be delivered. why hooksett? because ikea won’t deliver to warren and i’ve got in-laws in hooksett where ikea will deliver. i’ve just setup my old router and wireless base station here, so at least i don’t have to slum it without network. and that’s sort of what this great onion infographic is all about. take note of the point: “facilitates blogging while/about doing laundry. of life & death… i’m not sure i could say it any better than david rothman did when he went off topic over at teleread to make note of some important issues related to the terri schiavo matter. rothman points at the bigger issue, but doesn’t come out and say it: all life concludes with death; indeed, the leading cause of death is birth. i’m not being flippant, i mean this. life is filled with serious and difficult choices, including some related to the end of life. dis-intermediating pop culture via copyfight via deep links: fiona apple, that grammy award winning gal you remember from the criminal video, apparently put together a third album a couple years back only to have sony music shelve the thing. now that it’s gotten out, her fans are “demanding that sony release the album so they can pay for it.” which fred von lohmann describes as “a substantial noninfringing use of p p networks if i’ve ever seen one. sunshine week i’ve failed to live up to my potential this week. i’ve wasted a lot of time on stories about useless video cameras, home theater, whining about my kitchen remodeling, and lamenting some lost stories when i should have been paying attention to sxsw, etech, copyright issues, and sunshine week. please accept my johnny-come-lately mea culpa on all of that. sunshine week is intended to bring public attention to concerns about goverment secrecy. shuffleboard fridays joe, tami, sandee, and john throwing weights on the shuffleboard table friday night. extra: shufflboard rules at mastersgames, suffleboard rules at shuffleboard.co.uk. shuffleboard tables and tabletop shuffleboard accessories can also be found online. wish i was there: etech just as i was about to cut the future tense blog (from the public radio show of the same name) from my list, jon gordon steps up with a few good stories. of course, he had good material to start with. he’d been at the o’reilly emerging technology conference, and it looks like it was quite a show. many many has a couple notable stories about etech events, including wikipedia and the future of social computing and folksonomy, or how i learned to stop worrying and love the mess. your eff needs you a couple stories in the electronic frontier foundation‘s email newsletter need our attention and support. well, they all do, but here’s the most important: grokster: eff this week kicked off a new campaign to celebrate the technological diversity protected by the supreme court’s “betamax ruling,” which found that vendors cannot be held liable for contributory copyright infringement if their products are capable of significant noninfringing (legal) uses. eff will post information about a copying technology with substantial legal uses every weekday leading up to the march th supreme court hearing in mgm v. maisonbisson: the lost tapes…. i discovered recently that my content database is [missing a bunch of stories][ ] from the first weeks of . i tempered my feelings of loss with the knowledge that i couldn’t remember the title of more than one of the missing stories. while looking into a question about my out-of-date rss feed today, i discovered that it had clues to the content of twelve of my missing stories. they clearly weren’t that important (what is? small video cameras this fiddling with video has me looking for small cheap video cameras. security products has some, but pine computer has them cheaper. better yet, they’ve got a ca sub-mini video camera with interchangable lenses for $ . the standard mm lens has only a degree view angle, but an available (+ $ ) . mm lens should result in a much more useful degree view. the cameras all have composite ntsc outputs, but a usb video converter make them “digital. home theater there are bigger problems in the world than my home theater, but that’s not what this entry is about. i’ll get back to political ranting in a while, but for now — now that i have <a href=”/post/ ” title="a cheap inexpensive projector”>a cheap inexpensive projector — i’m interested in figuring out how to play videos from my computer. some people don’t need to ask why, but for those who do, let me offer this: most the video i create is better seen on the small screen, but fair-use dvd rips and content downloaded from the internet film archive. liability & license it turns out that the quicken website is full of legal tips and advice. what caught my eye was a description of implied warranties. implied warranties don’t come from anything a seller says or does. they arise automatically when a product is sold. under the uniform commercial code, there are two kinds of implied warranties: that the product is fit for its ordinary use, and that the product is fit for any special use the seller knows about. loss i discovered today that my content database is missing about entries from the first weeks of . the feeling of loss is pretty thick, but i get these feelings pretty easily — hey, don’t pick on me. of the stories, i can only remember the content of one of them. i think the story was titled “web apps rocked ” or something like that and was basically all about the goodness of xmlhttprequest. too exhausted, busy to blog i’ve got to tear down the last cabinet, get all the junk to the dump, clean, spot-sand and clearcoat the floors, and…. i probably won’t get it all done today. watch yesterday’s video for an idea of what’s going on, otherwise, today is re-run day. the archives are yours to explore. kitchen destruction time-lapse movie it’s all part of the plan, but this is a bigger mix effort and uncertainty than expected. i’d hoped to have everything cleared from the kitchen by mid-day, but i’ve got another cabinet to remove sunday. the uncertainty? we don’t yet have the new cabinets in hand. if those are delayed, we could be without a kitchen for quite a while. worse: we’re hosting easter and i’ve only got next weekend to install the cabinets and put the kitchen back together. “shred it!” engadget‘s got a story about ssi shredding systems and their action videos of their equipment doing the job on refrigerators, medical waste, steel drums, couches, concrete, boats…. engadget reccomends the washing machine video “for its rather endearing inclusion of one of the bystanders’ enthusiastic cries of ‘shred it!'” best new music trilok gurtu and robert miles on miles_gurtu listen in at itunes or amazon. bonobo’s dial m for monkey listen in at itunes or amazon. bonobo’s animal magic listen in at itunes or amazon. the bad plus give listen in at itunes or amazon. virtual kvm solutions folks are increasingly aware of screen sharing apps like vnc, but what about solutions that allow you to control multiple computers with a single keyboard and mouse? back in the day, there was an interesting macos hack that would send mouse and keyboard input from one computer to another (after some very easy configuration), today, in the days of os x, i can find two solutions: the powerpage tipped me off to kmremotecontrol a while ago. …and copyright law is broken too (duh!) i was looking for a way to includes these in my story about the brokeness of patent law, but they just wouldn’t fit. so here they are separately. increasingly, content owners are taking advantage of the vagaries of the “public domain” to make us pay for rights we used to take for granted. for instance, when you buy a chair, you expect to be able to use it however you wish. cliff likes the ‘works a flash and long manual exposure caught cliff and me setting up the ‘works, then their launch and aerial explosion on a cold night in january. the camera sat on my mitten in the snow while luck worked in my favor to get a couple good shots (and not burn my camera). just to be clear: neither of us was anywhere near the launch tube when the ‘works went off. today is warren’s town meeting day meeting has come and gone. the issue in the selectmen’s letter was postponed indefinitely and the meeting adjourned around : pm. on rss, taxonomies and folksonomies copyfight went somewhat off topic to point out joshua porter’s paper on how content aggregators change navigation and control of content at user interface engineering. this quote says exactly what i needed: every time someone makes a list, be it on a blog […] or a list of groceries, content is aggregated. the act of aggregating content (usually content that is alike in some way) makes it more understandable. instead of looking at a whole field of information, you choose smaller, more logical subsets of it in the hopes of understanding those. “so computers were worthless ten years ago?” jenny, the shifted librarian, related a story that show’s her son’s innate understanding of metcalfe’s law. here’s a completely truncated quote: “…before you were born, there wasn’t really an internet or the web or email. there was a very basic form for people in the military and at universities, but there were no web sites to visit and no web games to play.” “so computers were worthless ten years ago? all conversations in warren revolve around heat on jan th i noted that i’d burned through half my wood pellets for the season. i’ve burned another bags since, making it three quarters of my pellets for the season. now i’m hoping it feels a lot springier by early april, when my last bags will likely run out. what’s your nerd score? there in my referrer tags was planetilug.draiocht.net (though i can’t figure out why), where i found a link to the nerd test. two posters who’d taken it scored and . just as gareth easton said “i thought i’d give it a go… i answered truthfully (i’m ashamed to admit) ;-)” my score? th percentile: supreme nerd. apply for a professorship at mit now!!! of course, i’m a sucker for even the most ridiculous of personality tests. cuttin’ it up will cuts stuff up like…well, like a guy who cuts stuff. true to form, cliff points. they were over last saturday helping with with some remodeling projects. the luan is going to cover the bits of old horsehair plaster that still cling to the lath in the closet of what is becoming our laundry room. more of will and cliff can be seen in the plastics museum and museum of bad art, all part of the weird museum tour . vacation in the luxury of my own home i’m taking a spot of vacation here. expect nothing more from me today, and not much more in the days to come. — – — as before, the flickr photos have nothing to do with the post. and, no, this is not at all like martha‘s house arrest thing. sweet deal on home theater projector the sharp pg-b s projector isn’t the best out there, but it rates pretty well according to projectorcentral.com. their stats show it to be a lumen, × projector with a : contrast ratio and a long lamp life of up to hours. the projectorcentral.com user reviews suggest it has a good picture with great color rendition. macuser uk concluded: the pg-b s showed excellent detail from our presentation slides, with accurate colours and well-defined text, and it coped particularly well with solid blocks of colour. stay free!: copyright activists the are few things as joyus as the excitement of discovery, so it was a great pleasure to learn that stay free! magazine has a new blog: stay free! daily. the blog has a number of stories about intellectual freedom and copyright oppression that resonated with me. take a look at silent disobedience, christo’s policy of photographing the gates, and wizard people screening in nyc. anybody following discussion of the fcc’s broadcast flag mandate will be amused by an old movie studio and broadcaster psa arguing against subscription tv services. beware the cheap pc; beware the company that advertises them i’ve been saying for years that there’s no such thing as a cheap pc, but now a class action lawsuit against dell is claiming the same. according to arstechnica: it accuses dell of bait and switch tactics along with breach of contract, fraud and deceit in sales and advertising, and false advertising. the computer manufacturer is accused of advertising low-priced computers to consumers, but when consumers try to to buy the advertised machines, they find they are not available at the specified price. food and kitchen gadgets gizmodo just popped two stories about kitchen or food related gadgets that i love: a knife block worth having and a banana wrapper you didn’t know you needed. i might as well link to the sites themselves, as i can’t really think of anything to add: banana bunker and viceversa knife block. picture phone threats: they’re not what you think in a story that couldn’t have been much better timed, arstechnica is reporting on a camera system from that reads license plates and automatically looks up vehicle registration details. with some glibness, the article claims: “you just drive around and point the camera — it’s that easy!” though, it does note: as previously unconnected networks and systems integrate, this will increasingly be the case, and as scott mcneally said way back in , when sun microsystems had a bright and shiny future, “you have zero privacy anyway, get over it. (re-)programming the sony rm-v multifunction remote control in case you find the batteries dead, and the programming lost, sony’s instructions for configuring the rm-v multifunction remote control are online. you’ll have a heck of a time finding them, however, what with all the lousy epinions and nextag listings getting in the way. ignore those. codes for all the rest of sony’s remotes are online too. here are some seeds for google and the others: sony remote control codes for programming sony multifunction remote controls, like the rm-v are online at sony remote control support site. macs vs. pcs: tables turned? yale daily news reports on how windows is increasingly being pushed aside by macos x and linux. according to the article, yale information technology services’ registration records show that nearly percent of university students and percent of faculty choose macs over windows pcs. this is quite a change from the late s, when university it departments made news by trying to eliminate macs from their campuses. so what’s going on? iug : library portal integration & xml server applications elaine allard and i will be presenting on library portal integration at the iug in san francisco, ca. the session is scheduled for the : to : time slot on wednesday. from the program description: portal integration: what works at plymouth state university lamson library began its portal integration in with the launch of plymouth state university’s first portal, myplymouth. within this single point of service students can register for classes and check their grades, faculty can review their rosters and post grades, staff can review benefits and vacation time, and, of course, everybody can use the library. extra quotes most of these are a rehash, but i like them…. — – — a zdnet news article from december remarks: “apple buyers tend to have higher incomes and greater technological sophistication than the pc audience as a whole.” — – — regarding the first time her phone was hacked, a spokesperson for paris hilton is said to have claimed: she was pretty upset about it. it’s one thing to have people looking at your sex tapes, but having people reading your personal e-mails is a real invasion of privacy. international symbols enterprise language solutions has an interesting brief by yves lang on how to use symbols and icons in localization. cultural differences challenge the design and implementation of icons and symbols for international use. what is meaningful and natural for one group may be ambiguous, unintelligible, or arbitrary for another. fundamentally, communication is subjective, as a person’s perceptions are influenced by their environment. since their start in the olympics, the number of icons has grown remarkably. feature: privacy in the st century this is the story that gives me an excuse to name paris hilton here at maisonbisson. here’s a fact of st century life: pieces of our life that, taken one by one, are seemingly insignificant are being gathered and indexed by a handful of companies that re-sell that data to phone marketers, the cia, and many others. information that we recognize as somewhat more significant and often more private, like our driving records and tax information, gets sold and traded right along with the rest of it. feeling very sleepy around noon saturday sandee asked “why don’t we go to ikea?” the closest one is in new haven, connecticut, and we got there around pm. they close at pm, but after loading our u-haul, it was almost pm when we got on the road. we got back to the house around am, and now, after too little sleep, sandee has me assembling the catch. in short: no meaningful updates today. today in sports: le parkour troy pointed wildly and excitedly at a video showing his new favorite sport: le parkour. the video appeared on a site normally devoted to the fun of macromedia’s flash communications server: i recently saw the film film ‘jump britain’ on channel and was impressed by what i consider is an art form. it’s like skateboarding without skateboards, brilliant. le parkour consists of finding new and often dangerous ways through the city landscape — scaling walls, roof-running and leaping from building to building. retro handsets for mobile phones pokia is setting the world on fire with their retro phone handsets. they’re taking apart phones from the s s and s and rewiring the handsets to plug into today’s mobile phones. they’re selling on ebay, but most of the offerings are knock-offs. now mobilemag reports that boost mobile, the carrier that sells overpriced wood veneered handests is taking the idea mainstream. their retro phone handset has the look of a s, bakelite molded phone, but, i presume, without that funky feel that old bakelite has. feature: patent law is broken us patent laws are broken. adam b. jaffe and josh lerner say so. their ieee article is filled with equal measures of anecdotes and facts about why patent law is doing more to limit advancement in the arts and science than to support it. and that isn’t just wrong, it’s unconstitutional. there are a lot of ways to interpret the us constitution, but article , section is quite clear: “to promote the progress of science and useful arts, by securing for limited times to authors and inventors the exclusive right to their respective writings and discoveries. shameless commerce my beef t-shirts aren’t exactly mass market, so it’s a pleasure to see sales to california ( ), florida ( ), illinois ( ), kansas ( ), new york ( ), ohio ( ), oklahoma ( ), pennsylvania ( ), and washington ( ). i’ve just added a beef trucker’s hat for real retro fashion. it’s also a pleasure to see that the other designs are selling a bit too. brocolli and stump are the most popular (behind beef), but swine, cream filled, and killer get some attention. unusual hotels i recently discovered unusual hotels of the world, “the online guide for travelers interestedinstaying somewhere truly different,” and was pleasantly surprised to find a few hotels in north america i’d like to check in to some day. jules undersea lodge. source. want to slay a night under water? jules undersea lodge in key largo, florida, is for you. i have secret interest in trains, so i’d like to know more about the station restaurant & sleeping cars in ithaca, new york, and the aurora express of fairbanks, alaska. google maps rock, hacking them rocks more people are going wild{# } over google maps, but i honestly didn’t get too excited about it until i saw glen murphy’s movin gmap project. it’s a python script that reads location data from a connected gps and pans the gmap to follow. upon seeing this hack of gmaps, i went looking for more. hack a day shows us how to get maps for a set of decimal coordinates from both terraserver and gmap (terrabrowser will do some of this for macos x). students take academic technology into their own hands jenny levine, the shifted librarian, points out a recent survey that finds % of us college students own a cell phone. nationally, . million americans have cell phones. and cell phones aren’t just for talking, as we americans are sending . billion text messages a month. jenny’s point: “you can tell yourself that these trends won’t affect libraries, but you’d just be burying your head in the sand.” coincidentally, ken “caesar” fisher posted at arstechnica about student technology trends as well: all about stainless steel i’ve been contemplating the idea of welding/fabricating a stainless steel counter top, but i’ve never attempted any welding before, and most people say stainless steel is difficult to work with. thanks to this pdf, i know everything there is to know about stainless steel finishes, but nothing about working with the material. azom, “the premier on-line materials information site, supplier and expert directory” has a guide to stainless steel fabrication with rules for machining, welding, soldering, and brazing the various types of stainless. inflate & collapse two perfectly paired books: blow-up by sean topham and collapsible by per mollerup. one explores inflatable forms in art, architecture, and science. the other explores the somewhat broader range of things whose size and shape are meant to change as their use changes. they both look absolutely delightfull. . moving about on one, two, or three-wheels we’ve come to expect certain things. cars have four wheels, for instance. and we expect two-wheeled vehicles look like bikes or motorcycles or scooters. then came the segway a few years ago and shifted the two-wheeled concept around. now, a number of stories regarding vehicles of one, two, and three wheels have come out. they’re all interesting, some are awkward, some are to die for. one wheel wheelsurf. snow day! as cliff likes to say, “cur-tailed, the sweetest two words in the english language.” the snow started falling wednesday night and didn’t stop. even now, big, puffy flakes like oversized cotton balls are falling. [update:] photos added. also, here’s a snowy panorama from early january. geolocation tagging photos there’s a new version of jeff early’s gps photo linker, which allows you to combine tracks from your gps (time and position data) with your photos (time and image data), so you end up with a bunch of photos with embeded gps coordinates. jeff notes: apple has confirmed that macos . will support the gps metadata tags in photos. this will open up a whole realm of opportunities for users and developers to take advantage of the position data on photos. conspicuous consumption: the plan after some scraping and saving, and our refinancing, we’re remodeling our kitchen. our first attempt at doing this failed when i realized — too late — that i’m not actually capable of making cabinets. by that time, we’d filled the kitchen with a bunch of poorly made and unfinished junk. sure, there’s a sink and a fridge and stove top and an oven, but there’s one counter that’s been bare plywood for five years now, and there’s a bunch of other stuff that can never be finished because it was never built according to a plan that would ever actually work. marmite today i give props to bunchofpants‘s flickr photoset on marmite. i don’t really know what marmite is, but the marmite faq claims: marmite is dark brown-colored savory spread made from the yeast that is a by-product of the brewing industry. it has a very strong, slightly salty flavor. it is definitely a love-it-or-hate-it type of food. and, yes, marmite competes with vegemite, and both appear to be made of the same stuff. fast sofa…imac g fast there are a lot of folks who will tell you how “wrong” it is that apple integrates the monitor and computer in so many models, so i guess there’s a bunch of them that will tell you the same thing about how bluebroc is integrating the a sweet-looking couch and an imac g . “you’ll have to replace your couch every time you upgrade your computer! gosh (said napoleon-style).” there are probably even people that recommend dis-integrating the ipod from its display. ipod giggles ipod giggles **»** paul bourke, of the astronomy department at swinburne university of technology, has developed an ipod stereoscope. his system uses a pair of ipods in an old-style stereoscope viewer to display stereo-matched photos. » somebody at iaxb has come up with some renderings of a giant ipod shuffle sitting around the house like he or she owns the place. » more enlighteningly, canadian broadcasting corp. has a story on the the evolution of portable audio. standing up for clam juice okay, so i’ve been doing at least a post a day since about september and a few people got concerned when i missed a couple days{# }, but i am alive. gosh (said napoleon style). i’d probably pass on posts again today, but i was looking recent comments on my flickr photoblog and got a smile when i found evil angela‘s defense of clam juice: you know, it’s kind of like fish sauce. folksonomy is my new love okay, i’m excited about folksonomies. my introduction to tags was at flickr, where i’ve been amused at how they help connect people, photos, and concepts. then jenny levine at the shifted librarian started talking about them, with david rothman at teleread echoing and expanding many of her points. that was about when i found many to many, where i read about technorati’s tag project (plus documentation). wanna see it in action? copyright terrorism the dunhuang grottoes are one of china’s richest archaeological treasures. built during the th through th centuries, they are a , -year-old ancient art gallery of cave architecture, sculptures and murals. rediscovered in , the region has been listed on the unesco world heritage list since . despite over years of exploration and study, the mysteries of the grottoes are as great as the lessons they teach us. now, it would seem that the dunhuang academy is claiming ownership of all images associated with these year old treasures. looking for the energy drink tv ad? based on the search terms people come to this site with, i know that there’s a bunch of folks looking for the “energy drink ad,” or “k-fee tv commercial,” or “scary german,” or some such. most people end up finding my story about zygo energy vodka, and completely miss my story about the (deceptively titled) serene, calming video where i first linked the energy drink tv. let me eliminate the confusion now. all conversations in warren revolve around heat i have burned . tons of wood pellets so far this winter. the significance of the number isn’t its size, though . tons is a lot. the significance is that it represents bags of pellets, each pounds. the significance is that it represents about half of the pellets i’d purchased for the heating season. by the almanac, it looks like i should have ordered more pellets, as we’re not yet at midwinter and i’ll probably run out. big bear photos circulating my dad forwarded me the following pictures and story: these pictures are of a guy who works for the us forest service in alaska and his trophy bear. he was out deer hunting last week when a large grizzly bear charged him from about yards away. the guy unloaded his mm mag semi-automatic rifle into the bear and it dropped a few feet from him. the big bear was still alive so he reloaded and shot it several times in the head. language is of the people i am always amazed at the lengths we’ll go through to communicate or express or simply transliterate an idea, and further amazed at how we represent the result. take this for instance: th string| --- --- --- -- -- ---- - - - - - - - --- - -- - once you figure it out, you’ll likely not be able to get it out of your head. and this: sort of related, and much more ridiculous. wikipedia vs. brittannica; folksonomy vs. taxonomy a post on techdirt notes: you may recall that we somehow got involved in a bizarre battle over wikipedia, when i got into a discussion with a reporter who told me that wikipedia was “outrageous,” “repugnant” and “dangerous,” mainly because it’s not reviewed by “professionals.” despite a valiant effort, i was unable to ever convince the reporter, al fasoldt, that regular encyclopedias, complete with their experts, make mistakes too — and, in fact, the problem is that those encyclopedias can’t then be updated and fixed. the tyranny of copyright last week i pointed to will shetterly’s “the people who owned the bible” as an example of what might happen if copyright/intellectual property law continues to favor short term commercial interests over long term public interests. it’s worth noting that the original copyright laws, developed in s britain, allowed for only a seven year monopoly (that’s what copyright is, after all). us law started by doubling that to . the current term is or years, but it doesn’t matter because the music and film industries will lobby congress in a few years to make it or so. cold weather operations force powerbook pmu reset batteries don’t work well in the cold, and with the - °f nights we’ve had, i think i can say it’s been cold here lately. i woke my powerbook from sleep in sub-freezing temperatures this morning and got a few minutes of work out of it before it put itself to sleep again. i popped it into my computer bag and ran off to work, where i was troubled to find it refusing to wake from sleep — even when plugged into the ac adapter in a warm room. using your mobile phone as modem i’ve been following cell-carrier wireless data options here at maisonbisson (here and here), but i have to admit that i don’t actually use any such solutions. i live and work (and usually travel) in range of ethernet and wifi, so i might get a pass on this but the real reason is laziness. engadget has a nice write-up on the process with cdma-based phones like the ones you get from sprint and verizon. edward tufte gives presentation advice edward tufte‘s passion is the graphical display of information. but his nemsis the visual lie. so naturally, he has a special dislike for powerpoint. his poster on the cognitive style of powerpoint gave me this line, which i will likely find myself repeating at a time when it is both most accurate and most politically suicidal to do so: why are we having this meeting? the rate of information transfer is asymptotically approaching zero. palm travel guides mypalmlife is running a story about some new travel guides that run on your palm-powered device. produced as a collaboration between rough guides and visual it, they also support pocketpc and symbion devices. london, paris, rome, new york, and san francisco are available now at an introductory price of $ each. “further cities will be released over the coming months.” according to the website, the rough guide city maps include: feds go beyond carnivore; artists embrace carnivore defensetech reports that the fbi has given up on carnivore, the electronic snooping application that it used to force on isps serving suspects. it seems that the folks in dark suits are now using commercial software instead. this probably has no effect on artists — yes, artists — who use an open source app inspired by the feds as the center of their networked interactive art. called carnivorepe, it’s the back-end of over two dozen art installations, most graphically: police state. microsoft: bad for browsers; bad for air travel i just discovered this is broken and couldn’t help but explore the archives. first i discovered brill.com‘s weird search results. the problem is that a search for bond funds returns a list of stories that have little to do with financial news. it looks like somebody has entered a bunch of bogus stories in their database. they might have been hacked, but i’d be more suspicious of a disgruntled employee. the saddest part is that the problem was reported on september , and they haven’t fixed it yet. browse happy browse happy, by the the web standards project is urging people to give up on microsoft’s internet explorer. their solution? firefox, mozilla, opera, and safari. mac os x performance questions i was a little bummed to find my cpu busy all morning yesterday. and though i still don’t understand exactly what was causing it, it seems no longer to be a problem. a lot of people don’t know how to see what their mac is doing, to see what it’s busy with. here are some hints: start with activity monitor in applications > utilities. from there you can see and sort applications and processes that are running on your computer. problems and pre-dated stories due to problems with the site all this week, a couple of time-sensitive stories that i wrote but coudn’t post have now been posted with pre-dated timestamps. i’ve been following every news item about the mac mini with likely more interest than it deserves. what can i say, i like the little computer. as it turns out, the mini is smaller than it looks in the pitures. and thinking of pictures, a few shots of bill gates vogueing with circa- computer equipment started circulating early this week. candy karen forwarded me a link to juicy panic‘s “you drive me oh oh oh” video by torisukoshiro + autophene. more animation and illustration by torisukoshiro is linked from the main site. then she sent me this link to how strange, a site full of odd, interesting, and weird images. . . . palm news & goodies gizmodo mentioned the new garmin ique a gps palm for pilots this morning. there’s a long write up about it at mypalmlife, but the gizmodo story linked to palm . once there, i found a link to instructions on putting the wikipedia on a palm. well, you’ll need a gb sd card, but that’s okay, right? it all depends on tomeraider, an interesting app and fileformat for searchable, hyper-linked e-content. palm is also running a contest to win a free copy of trip boss, an all-in-one travel manager. problems happen my hosting provider has a us-based datacenter and uk-based staff. it’s an odd mix that may or may not be helpful when things go all to heck, like they did on saturday and again on tuesday. the first acknowledgment of the problem saturday explained that “the server is reporting a kernel panic.” then four hours later, it was reported that “there is a major fault with the boot sector and kernel on the server prevent it from loading into the lilo prompt, or booting from a new kernel due to damage. mac mini vs. cheapo pcs charles jade at arstechnica has written both a mac mini preview and a macworld expo show walkthrough. the expo is about a lot more than the stevenote, and jade does a fine job walking us about the show floor. also entertaining is an osviews story on the mac mini that concludes the mini is far less expensive than home-built pcs. not that there aren’t a lot of people arguing with that conclusion in the comments. the mac mini is _small_ i said the mac mini was the reincarnation of the cube last week, but gizmodo has posted a picture of the two, um, together. we all knew the mini was small, but this shows how reall small it is. the unoffical apple weblog has a list of things people are planning to do with their mini as soon as they get their hands on one. now add to that list a mini-based synthesizer. where’s my video jukebox? yesterday i posted a story about using a mac mini in my home entertainment center. i noted that i’d already replaced my cd player with itunes on an old imac and i wondered if i could do the same for dvds. i ignored the facts that some provisions of the dmca may make this illegal. the music revolution was made possible because courts recognize our right to encode cds from our collection as mp s, and cds (mostly) lack copy protections that prevent us from doing that. bill g just wants to be cool gizmodo has two pictures of a young bill gates vogueing on a desk with . -inch floppies and a circa- pc monitor. oh, wait, is that a mac on his desk behind him? the pics were reportedly published in tiger beat, and gizmodo is offering a reward for the original issue. update david heisler wrote to gizmodo to offer this correction and detail: [those] are not from tiger beat. according to snopes. mac mini as media player more than a few people are looking at the mac mini as a new component in their home entertainment center. cds are unknown in our house, where itunes and an old imac entirely replaced our five disc changer some time ago. correction: cds are used as an input medium. new cds are ripped into itunes on their first play, then left to gather dust on the shelf. video seems ripe for a similar shift, and to many, the mini looks like the perfect platform for it. michale stephen’s twelve techie things michael stephens’ twelve techie things for librarians deserves a look. user-centered technology planning, rss, acnd convergence lead his list, but other items speak directly to the role of the library in the internet age. pmachine discontinued, where to next? i learned today that pmachine pro — the software behind this site — has been discontinued. i’d expected the announcement for some time, seeing it today reminded me that i should be looking for a new blog/cms solution. expression engine has largely replaced pmachine, and i know at least one person running it, so i’ll likely be giving it another look soon. i’ve got a list of things i’d like to solve here, so this news sort of fits. oil star this super-cool s-styled logo adorns the side of a trailer in the backwoods of new hampshire. more photos from maisonbisson jailed for a song trying to quote lyrics for his book, planet simpson to understand how current copyright law is already limiting legitimate work. lots more stories of copyright law gone amok in the maisonbisson copyrights & intellectual property index. the tyranny of copyright if you read nothing else all year, read this. will shetterly’s “the people who owned the bible” is a tale of copyright gone amok. it’s the clearest, plainest, and funniest of all such works i’ve seen. note: my title is based on a new york times story about copyright from a while back. am i in trouble? steve jobs introduces ipod shuffle in his macworld expo keynote today, steve jobs introduced the ipod shuffle. from macnn: apple introduces ipod shuffle…flash based player. smaller than most packs of gum. weighs the same as quarters (less than ounce). volume/up dow. simple led to provide feedback. no display. either shuffle or album-based playback. usb transfer connector under connector at the bottom. -hour rechargeable battery. steve jobs introduces mac mini steve jobs, in his keynote at macworld expo today reintroduced a redesigned mac cube as the mac mini. from macnn: apple introduces mac mini. new member of mac family slot-load combo optical drive. play dvds, burn cds. quiet. tiny. firewire, ethernet, usb . , both dvi/vga output. very tiny. height is half the size of an ipod mini. byodkm. bring your own display, keyboard, mouse. vonage wifi voip handset is real all the world is atwitter about vonage’s new wifi voip phone today. wifinetnews got the hint from engadget, who appears to have broken the story today, and links to a usa today story that says: with a wi-fi phone, they could make internet calls from home without the need to run wires to the broadband line. customers could use the phone number of their existing vonage service or a new one for no extra fee. video fix today might be [wierd|strange|funny|scary] video day. or something. these are probably not safe for work, though your mileage may vary. here’s the list of things found last night: rainbow the site explains/claims: “rainbow was a credible children’s tv show from the s and s. this clip was actually broadcast and watched by millions. …there’s no way these could have been done by accident. innuendo all the way.” supermodelmeat classic and independent movie theaters a story in the december /january issue of arthur frommer’s budget travel magazine alerted me to ross melnick and andreas fuchs’s cinema treasures. it was an annotated list of seven theaters still operating today: cape cinema: this dennis, mass., theater was built to look like a church. the senator theatre: a -year-old art moderne classic, it shows new releases in baltimore. oriental theatre: head to milwaukee for this $ . the future of libraries roderick (also, check out roderick’s new blog) forwarded me a story about the challenges facing academic libraries from the chronicle of higher education. the author, dennis dillon, whose full title is associate director for research services at the libraries of the university of texas at austin, begins by relating a conversation: “couldn’t you move your technology to mumbai and hire some english-speaking indian librarians to catalog the books and answer reference questions over the web? backfill i should admit to it now before it becomes a scandal. i backfilled some content this weekend. some of it is stuff that i wrote in the past for work (edited for publication here), but i feel may have some public value. specifically, two stories about wireless: one about its vulnerabilities and another about (then) current practices in the academic community. i also posted my wife’s first story for maisonbisson: a recipe for fish tacos. a decadent and debauched slave of foreign culture i first learned of wei hui and her first book shanghai baby on npr a few years ago. according to the story, wei hui is among a “group of young, attractive women known as the ‘beautiful writers’ churning out novels that graphically describe the hedonism of modern urban china.” wei hui’s book was so controversial that it chinese authorities banned it, causing a nearly immediate surge in popularity at home and abroad. tech roundup it’s getting a little late for these roundup things, but i’m too tired with post-new year’s party haze to come up with much of anything better right now. annalee newitz subtitles her website with “technology, pop culture, sex.” her index of stories isn’t actually a roundup per se, but it’s good material if you’re too lazy to leave the couch and find a book to re-read off the shelf (because you’ve read all you new books by now, right? wrapping up a year of controversy alternet had a good line of stories this weekend to round up the old year and ring in the new. i’m running a little late on such things here at maisonbisson, so let me just quote from theirs instead. — – — daniel kurtzman’s list of the dumbest quotes of includes this doozy at the number spot: “all of a sudden, we see riots, we see protests, we see people clashing. slacking is universal in yet another reminder from mainichi daily news that american’s and japanese aren’t so different, now they’re reporting: coeds say college guys ‘childish, irresponsible, stupid.’ a survey of female students selected from universities located in either osaka, kyoto or kobe reveals: a majority of the women polled said that their main impression of male students is that they are childish, the . percent given to the most frequent answer followed by the percent who thought guys are kind and . ipod hacks hack-a-day has just given me the best reason i’ve seen yet to take a closer look at ipod linux: audio input without the cheap dohicky accessories and at up to khz x bit. the five step instructions couldn’t be much simpler (well, it might be more complex once a person actually tries it, but the comments suggest good success). hack-a-day is covering lots of ipod hacks (much to the consternation of some readers, but they’re just jealous ’cause they don’t have one). terminal holiday for k+ i got to spend the holidays near home this year, and with everything else going on i didn’t really pay much attention to the comair/delta problem that stranded over , passengers last weekend. now that i’m starting to pay attention to the news again, though, i was interested in arstechnica‘s discussion of the software glitch that made everything go wrong: at the core of the problem was an application created by sbs, a subsidiary of boeing. let fly the macworld rumors everybody is gaga (links: one — two — three — four) over the thinksecret story: apple to drop sub-$ mac bomb at expo. many people in the mac community have been agitating for a low-end ‘headless’ mac to compete on price against cheap pcs. the rumored specs include: . ghz g cpu mb ram combo drive – gb hard drive usb . national geographic society not so environmentally conscious i know i’m complaining here, but national geographic seems to have done this wrong. i purchased the complete national geographic — years of national geographic on cd-rom a few years ago. the collection of cds is an archive of every page of every issue published from through . it was a joy to explore that archive, but let’s face it, i wasn’t spending every night doing it. today i got the notion to reinstall it to search for something, but discovered that the application is far out of date and no bug fixes are available. google the economist has a very concise explanation of how google works, and how it became today’s dominant search engine. mr brin’s and mr page’s accomplishment was to devise a way to sort the results by determining which pages were likely to be most relevant. they did so using a mathematical recipe, or algorithm, called pagerank. this algorithm is at the heart of google’s success, distinguishing it from all previous search engines and accounting for its apparently magical ability to find the most useful web pages. high speed wireless michael sciannamea at wirelessweblog noted that: bmw, audi, daimler chrysler, volkswagen, renault, and fiat have all received grants from the german government to develop a car-to-car wireless data network using . a and ipv technologies to link vehicles to each other to pass on information about traffic, bad weather, and accidents. they’re calling it “now: network on wheels,” and there’s more at wi-fiplanet.com. my comment: static mesh networks are so . chernobyl followup i posted a story about a tour through chernobyl a few weeks ago. the story still gets a lot of hits, and somebody pointed out a few related wikipedia links about the accident, the ghost town, and the controversy about elena filatova, the author of everybody’s favorite online chernobyl tour story. separately, peace.ca reminds us about the dangers of war, nuclear contamination, and more. free palm apps, now easier to find jon aquino‘s holiday gift to us is to make freewarepalm useful: why this work was necessary: freewarepalm contains a goldmine of ratings of palm freeware. but it does not provide a way to sort the programs by rating. that is why i extracted the ratings and sorted them. with over listings, there’s a lot to choose from, but, as jon says, no way to sort those listings. jon has crawled freewarepalm with “cygwin lynx, xemacs, and a -line ruby script” and done what freewarepalm couldn’t: made a list of apps sorted by rating. heart warming holiday tale for hackers i recently stumbled across ron avitzur’s story of the the development of graphing calculator, the little application that makes complex math easy to visualize. if there was a collection of essays titled “chicken soup for the silicon valley soul,” this would be included. pacific tech’s graphing calculator has a long history. i began the work in while in school. that became milo, and later became part of framemaker. over the last twenty years, many people have contributed to it. requisite holiday email forward mark turski‘s holiday message: avoid carrot sticks. anyone who puts carrots on a holiday buffet table knows nothing of the christmas spirit. in fact, if you see carrots, leave immediately. go next door, where they’re serving rum balls. drink as much eggnog as you can. and quickly. like fine single-malt scotch, it’s rare. in fact, it’s even rarer than single-malt scotch. you can’t find it any other time of year but now. happy holidays the warren rocket stands in the the snow on december , . happy holidays photo taken december , , just north of warren on nh route c. the snow is real (and much deeper now), but i added the lights for the holidays. regular updates to maisonbisson will return after a short holiday break. coincidence is too general a term engadget had a laugh over a story in the keene sentinel: so the other day a ups driver in new hampshire was on his way to the cheshire medical center in keene to deliver some much-needed parts for a piece of medical equipment when he got into acrash. he suffered a head injury and was taken by ambulance to the very same hospital he was headed to, but they weren’t able to do any of the tests they needed because the brain scan machine was broken — and the parts needed to fix it were sitting in his wrecked truck on the highway. apple fans mod macs joseph deruvo jr.’s i-tablet is this year’s mac mod. wired’s leander kahneyusually covers the story, but deruvo published this one himself at macmod. kahney covered jeff paradiso’s converted ibook tablet as part of his story on mac modders. he followed that up in with a story about a pyramid-shaped powermac that glowed blue. the mac mod thing is international, as kahney points out in this story about japan’s mac mod culture. cross-country journeys in time-lapse i feel a tinge of jealousy every time i see something like this: lacquer sound’s road trip. similar: i covered matt frondorf’s mile markers project a while back. (picture from mile markers). gary webb: a journalist who dared alternet ran an interesting story about gary webb‘s recent suicide and the events that may have led to it. webb was the -year-old former pulitzer-winning reporter who in , while working for the san jose mercury news, touched off a national debate with a three-part series that linked the cia-sponsored nicaraguan contras to a crack-dealing epidemic in los angeles and other american cities. the resulting firestorm swept the country. fcc’s complaint system gamed i’ve got a backlog o stories to post here, including this old one about broadcast programming complaints to the fcc. the fcc reports that it received a mere complaints in , but , in . so what can account for the nearly -x increase? the fcc did some homework on the matter: according to a new fcc estimate obtained by mediaweek, nearly all indecency complaints in — . gps happy my brother and his wife surprised me with a rayming tn- gps this holiday season. what’s so great about it? it’s a tiny usb powered brick that interfaces easily with a laptop. the plan? wardriving (yes, it’s sooo three years ago), better geolocation while traveling, matching gps coordinates to photos, and as much mayhem as can be had with a computer-connected gps. software options rayming is mac friendly enough to offer a page of links to mac gps resources and include the necessary driver on the cd. seacoast industry sometimes a story will popup as a clear reminder that the world is not always as it seems. i will admit both surprise and amusement when i found that foster’s daily democrat reported saturday on the content of a federal indictment of a kittery, maine, health club. geography lesson: foster’s covers new hampshire’s seacoast — all miles of it — and kittery is a shopping destination squished into the southernmost corner of maine. the indictment accuses gary h. reiner of running “an interstate prostitution ring.” foster’s reports that the club has operated under various names, most recently the “danish health club,” owned by “kittery health club inc.” reiner was apparently both the owner of the club and the former town council chairman and had a role in shaping the local regulations of spas and health clubs. the story clearly had some history, and i’m fortunate the web, and foster’s archives, can educate me. displaying word docs and pdfs in safari royce asked: how can i disable or tweak download manager so that files can be read in line with the download and manually launch through the download manager? i want to be able to click on a pdf or word doc and have it open inline without having the download manager handle it to the desktop first. context: some people say the inline display of pdf and word documents enables bad habits that are making the web less accessible and harder to use. fun with license plates jameson wrote me today to point out that he can get a new hampshire moose license plate with the text “-brk m” he found my story about new hampshire license plates, including the bit about nh’s online plate lookup. then he pointed out that he could get a purple heart plate with the text “fugw” political messages on license plates seem to usually go one way: from government to people. this rare one reverses it. isight accessories and beauty tips macdevcenter published a guide on how to look great on ichat av back in march. the point? video is changing telecommunications: no longer can we sit in grubby geek glee, protected by our avatar shields, wearing only uniforms of underwear. endangered are the days where we can pass digital transmissions and gas simultaneously, picking our noses with one hand, and stuffing pizza down our throats with the other. slowly but surely video is changing that, and sooner or later you’re going to find yourself beamed up into someone’s ichat av window. weird palm apps canalpda, a spanish-language pda info site has released an english version of their story about the weirdest palm os programs. you’ll have to follow the link to read about why they thought the apps were so weird, but the titles give some clue: voodoo palm mirror bistromatic fakecall palmasutra fdic divination scare the doggy bubble wrap emulator darn comment spam <a href="http://flickr.com/photos/maisonbisson/sets/ /” title="canned meats at flickr"“>now that most email clients have reasonable spam filtering capabilities, spammers are targeting comments systems on blogs, guestbooks (i thought those had disappeared, but i saw one yesterday) and other open submission forms that post to the web. ip banning probably never worked, as spammers have been using open proxys for years. word blacklists (like ignore comments with “online-casino.com” in them) require regular maintenance and could result in false positives. beware the cheap pc the public radio show future tense did a story monday that asks “will you regret buying a cheapie pc?” computers are cheaper than ever. but if you’re looking at a new machine this holiday season, dwight silverman of the houston chronicle says beware of the low, low prices. why will you regret it? the machines are ram starved, have lousy video hardware, bad monitors, processors that are slower than their mhz ratings make them look, small hard drives, and often lack even a cd burner. more about google print prediction: we’ll talk about google print until they debut the beta, then we’ll talk about it more. copyfight posted some followup on google’s announcement earlier this week. of note was a quote from michael madison: a first thought: it’s one more example, and a pretty important one, of the fading of the lines separating copyright law from communications law. is google print an information conduit? a massive, rogue p p technology? ipod supplies tight; holiday sales to exceed four million summary: four million to be sold this holiday season; adoption rate higher than for sony’s walkman. from macnn: an article in the the wall street journal today says that ipods are becoming scarce at retailers around the country. the report says that amazon.com, buy.com, and other online retailers are now out of stock and “apple is contending with what appears to be an immense demand for the gadget,” and it suggests that apple is dealing with manufacturing and distribution constraints due to the ipod’s ‘near-cult status. wireless security: wep dead wifi net news is saying r.i.p. w.e.p. after news of a new version of aircrack was released that can break wep in seconds after passively sniffing only a small number of packets. the result is that it takes only two to five minutes to crack a key. even keys changed every minutes are thus susceptible to an attack that might allow several minutes of discrete information. unique keys distributed by . usb headset microphone i went looking for a usb headset microphone, and the telex h- usb digital computer headset seems to be the cheapest one that doesn’t suck. amazon’s users comments for the other headsets in that price range (under bucks) spoke of bad sound, uncomfortable fit, and fragile parts. the customer reviews of the telex h- , on the other hand, all rate it out of and commend its quality. serious question about funny picture sometime ago i saw this picture among a bunch that were circulating in those emails that get forwarded all over the place. the site i first saw it on dissappeared shortly after, and i haven’t seen this shot again until now. it looks like this page is a copy of the one i saw in early , and it includes this picture. my question is, where did it come from. i haven’t seen anybody name the source or context for this photo. i’m now an expert on kabbalah okay, that’s a lie, and it’s probably a little insensitive. sorry. what i really mean is that the monday edition of fresh air — that npr talk show with terry gross — was all about kabbalah. terry’s guest was arthur green: historian and theologian arthur green has long studied jewish religion and culture. among the many books he has written is his latest, a guide to the zohar. […] in addition to being dean of the rabbinical school of hebrew college, arthur green is also on leave from brandeis university. google stuns libraries, again arstechnica seemed to sum it up best: today, it is expected that google will announce an agreement to scan and create databases of works from five major libraries. according to news reports, google will digitize all volumes in the university of michigan and stanford university library systems along with parts of research libraries at harvard, the new york public library, and oxford university in england. more information on the scope of projects at the individual institutions can be found at news. exploring coudal last week i noted the shhh project to hush noisy cell phone users by draplin and coudal. today, i spent some time surfing the coudal site and found a few things. jewelboxing is coudal’s answer to lousy cd jewel boxes and dvd cases that aren’t much better. the super jewel box king was developed in conjunction with phillips at the same time as the dvd. the standard was designed and introduced shortly after. new hampshire’s teen drug use high, teen crime rate low katherine merrow, senior research associate at the new hampshire center for public policy studies recently released a study on teen drug use and juvenile crime in nh. the following is quoted from the study’s executive summary: two recent surveys indicate that new hampshire teens use drugs at rates significantly higher than their national counterparts. one survey placed new hampshire among the top states in the nation in terms of the proportion of its teen population abusing either alcohol or drugs. laughing at your idol while following the story about bad teachers, found the mathcaddy blog. the only relation mathcaddy has to the other story is that steve, the unfortunate student runs his blog on a subdomain there. the post that got me interested at mathcaddy was i walked on water… i think i can walk to the door: in one of his forty-eight dozen interviews about the passion of the christ, mel gibson said there have been more than a hundred films made about the life of jesus. holiday deals on macs macnn gave me the heads up that apple had reshuffled its refurb and discount shelves late last week. shoppers got as much as % off selected items, with previous generation models being unloaded at the best discounts. thing is, the deals were picked up quick, and the store seems to be empty of the best of them. the ghz ibook that was current until this fall was going for $ , and the . teacher proves — once again — that schools are averse to free thought copyfight‘s donna wentworth passed along this “sad and perverse story of a teenager who was given an “f” for writing a paper attempting to distinguish between piracy and stealing.” copyfight quote’s boingboing‘s story: geluso, an “a” student, recently completed an in-class exit exam for his language arts class. the goal of the exit exam was to write a comparative essay on a topic of the student’s choice. being a student who enjoys a challenge, he wrote an essay contrasting piracy with stealing. cult of mac, cult of newton, cult of ipod no starch press recently released leander kahney’s the cult of mac. bookblog notes: are there trade shows for toasters? of course not. so why is there a twice-yearly show devoted to a type of [computer] consumer? well, a computer isn’t just a computer when it’s a mac, and macintosh fans will go to great lengths to celebrate their devotion. the book is a followup to the regular cult of mac reporting in wired news. gear and gadget reviews gizmodo popped a link over dan washburn’s gadget round up. dan had been on a four month road trip through china, and has now posted the results of how his gear stood up to the trek. on the trip he took an ipod with a media reader, extended battery, and voice recorder mic; two cameras — cannon s and s ; an ipaq with keyboard and gprs modem; and a garmin etrex. writer goes solar for electric, hot water, and heat o’reilly author brian mcconnell hasn’t gone off the grid, but he’s reduced his dependance on it and in so doing, lessened his footprint on the environment. electric generates % of his home electric consumption. solar hot water heats his hot tub, eliminating much of the remaining electric consumption. forced hot air solar heats his house, eliminating half of his natural gas consumption. total cost of system was $ , , rolled into his mortgage. saab is latest car maker to get excited about ipods macnn reports that saab has released an ipod integration kit: saab has quietly introduced its own ipod/mp player audio integration system. the new system, listed in the most recent saab accessories catalog from october , offers direct input for and control of the ipod on its saab - , according to one macnn reader: “i spoke with the parts department at my dealership and they confirmed that it’s available. evidently it’s wired through to the center console armrest and will be out of site. smack the shhh down on noisy cell users gizmodo was excited enough about the draplin and coudal shhh cards: two designers have made these warning cards for obnoxious cell phone users, available in convenient pdf download-and-cut-out form. it’s a good way to make it clear to people they’re talking too loudly, and a good way to eventually get into a good, american fist-fight. then someone can hand you a card that explains why they found your teeth in their soda to be “more than a little annoying. missile week at maisonbisson it’s missile and space weapons week at maisonbisson. one item, the increasing pace of missile development in hostile and semi-hostile countries as a reaction to the us missile shield, is real news. the others are softer. i wish i’d planned it. don’t miss russia’s space battle station or warren’s home-town missile. copyright lessons from waffle house to round out my week of quoting stories from lquilter.net, today i’m putting forward this one about intellectual property (originally from critical montages): ever notice the waffle house menu’s insistence that double waffle is for <a href="www-wafflehouse-com-whmenu.pdf” title="“dine-in only, no sharing"“>“dine-in only, no sharing”? a common prohibition at low-end restaurants, it’s also a small-print reminder of what capitalism is all about. from enclosure to enforcement of intellectual property rights, capital’s message is always no sharing. mobile carrier wireless networking, take i took a long look at mobile wireless data service back in september. now, engadget says: they’re currently test-marketing a new wireless data plan called mobile media that costs fifteen bucks a month (the same as sprint pcs vision) and gives you unlimited data usage and access to their new streaming video service […] assuming everything goes as planned, they’ll be introducing the new service in january. i guess i have to look at sprint pcs again, because last time i looked, prices were $ to $ . reader report: pie ipod input adapter a reader, mike, wrote in to reccomend the precision interface electronics aux input adapter to connect the audio from my ipod to my scion’s factory head unit. i don’t know if you ever found a solution to connecting your ipod to your scion head unit, but if not, you can use this adapter to add an aux input to the scion factory head unit. i asked mike for followup and details, and he offered this: pictures of the warren rocket warren is blessed with a rocket. it was once an intermediate range ballistic missile, but it’s basically the same rocket that launched america’s first astronauts allen b. shepherd and gus grissom into sub-orbital space. it’s enough to be proud of, anyway. roadsideamerica.com has a story on our rocket, but it’s based on reader reports and it seems people just don’t know what town they’re in when they see the thing. the christian right and the sanctity of marriage lquilter.net pointed me to an interesting entry at newdonkey: the christian right and the sanctity of marriage as we all know, the christian right has now made defense of the institution of marriage, as defined as a union of a man and woman, not only its top political priority, but the very touchstone of christian moral responsibility. i’ve always found this rather ironic, since the protestant reformation, to which most christian right leaders continue to swear fealty, made one of its own touchstones the derogation of marriage as a purely religious, as opposed to civic, obligation. missiles are the new fashion defensetech reported today that “russia is leaning more and more on its nuclear weapons, as its conventional military falls into the toilet.” elsewhere at defensetech today was a link to armscontrolwonk, which leads to news that the us isn’t working with the iaea. this isn’t good. the ap, via defensetech is reporting speaking at a meeting of the armed forces’ leadership, putin reportedly said that russia is researching and successfully testing new nuclear missile systems. russian battle station polyus defensetech reported, some time ago, on the old ussr’s space battle station (or, communist russia’s answer to reagan’s star wars program). more pictures are in a forum at militaryphotos.net. called polyus, it was ridiculously huge — as with all things russian. sadly, (from a purely scientific perspective) defensetech reports “it couldn’t get itself into a working orbit, probably because of ‘a faulty inertial guidance sensor,’ according to the encyclopedia astronautica.” us senate on porn i’ve been reading the archives at lquilter.net, where i stumbled across this amusing yet scary entry: …on the first amendment side of things, wired has a great new story explaining how recent senate commerce committee, science, technology & space subcommittee hearings have shown that internet porn is the worst scourge this nation has seen since cia-sponsored heroin. [wired / ] “pornography really does, unlike other addictions, biologically cause direct release of the most perfect addictive substance,” satinover said. shock tanks gizmodo alerted me to these shocking remote control tanks. for bucks you get two remote control tanks with which you and a pal will do battle. it’s a game of “maneauver and fire, evade,” or something like that, with the additional carrot that if you hit your opponent’s tank, he or she will get an electric shock. the stick is that if your opponent hits your tank, you get the shock. dog sled racing justin at the start of his four-dog sled race in meredith, new hampshire. the video of justin’s finish is also online. snow started falling early friday and continued through saturday morning. it’s the heavy, wet snow you get when the air is still warm. the frost isn’t deep and there are still-soft patches of ground here and there, so the snow is melting in parts, but it’s snow nonetheless. it’s snow enough that justin might be able to run the dogs on the sled, rather than on his bike as he does through the fall. cool tvs and rc aerial photos gizmodo went gaga for plus minus zero, a little electronics shop in japan where “they hand-design a selection of products, then contract the production of the units out for a limited run.” the post includes a picture of one of their products, an lcd television that looks like one of those classic tube tvs from the s. then gizmodo linked to this radio control aerial photography discussion board with some great pix. bush on tape cliff over at spiralbound.net posted the video of bush flipping the bird. it’s not as exciting as i’d hoped, but it’s on video. then there’s the dubya movie. it’s a fantastic mashup of old don kotts movies, but that’s already giving too much away. go watch it, you’ll laugh. a night at the hip hopera i’m not really sure how to describe the kleptones and their album a night at the hip hopera, but i can tell you how i found it. disney sent takedown notices to those who were mirroring the work, raising the ire of the copyfight community. you see, the kleptones are really quite good, but their album is a mashup of queen songs, and disney (who owns the rights to queen’s music), got itchy. states rights lq wrote at lquilter.net about looming challenges to federalism i’ll be interested to see how the conservative, pro-federalism, pro-states’ rights, gop-run government (and the conservative intelligentsia which carries their theoretical water) handles some of the upcoming challenges to federalism: medical marijuana laws state & regional initiatives on global warming: for isntance, california’s mandatory cap on greenhouse-gas emissions will have to be signed off on by the epa before it goes into effect i tried to comment, but wordpress kept ignoring me. instead, i’ll post here and trackback. james loewen writes, in his book lies across america, that “states rights” is the call of whatever party doesn’t control the presidency. the republicans made a lot of noise about it during the clinton years, but will likely have to adjust their position now. some readers will likely point out, however, that the unspoken republican tenet (at least since the early s) is “might makes right.” sadly, the bush administration has already supported challenges to local environmental regulations. i can’t remember the specifics, but a federal court struck down a california law that required clean-burning busses and trucks in the state. maybe republicans are more tolerant of cognitive dissonance than liberals. maybe they don’t care. flickr random selection email is for dinosaurs in south korea a south korean newspaper is predicting the death of email. a poll conducted […] on over , middle, high school and college students in gyeonggi and chungcheong provinces in october revealed that more than two-thirds of the respondents said, “i rarely use or don’t use e-mail at all.” it seems email just isn’t fast enough for these wippersnappers. …it’s impossible to tell whether an addressee has received a message right away and replies are not immediately forthcoming. lycos-europe’s spam plan smartmobs reports that lycos is planning to raise the cost of spam with a gentle ddos attack. yes, gentle. lycos-europe is distributing a free downloadable screensaver called make love not spam that directs a low-intensity distributed denial of service attack (ddos) at urls contained in spam messages. the bbc article quoted at smartmobs reports: mr pollmann said there was no intention to stop the spam websites working by subjecting them with too much data to cope with. wifi seeker, finder, detector roundup handtops.com has published a wifi seeker, finder, detector roundup. the five models they reviewed include: smart id wifi detector – wfs- pctel wifi seeker kensington wifi finder plus hawking technologies wifi locator – hwl canary wireless digital hotspotter – hs my favorite, and it’s not based on any experience with any of these products, is the canary wireless digital hotspotter. it’s the smartest of the bunch and shows the war on fair use somebody somewhere, probably a lawyer in the entertainment industry, has a list titled “rabid fair use advocates” and david rothman is near or at the top. not that i mean that as a criticism, or that mr. rothman would take it as such. it’s just a likely fact. today, however, i’m playing a game by quoting his post about the war on fair use in full: doubt there’s a war against fair use? encompass for digital collections and resource access we’re looking at encompass for digital collections and resource access here. it’s an expensive product, but has a lot of interesting and useful features. some sites we looked at in the demo today included new zealand national library, ut dallas, and alabama mosaic. bloody saturday in the soviet union: novocherkassk, i had a long conversation with my brother about communist russia last night. it’s not really an area i can talk about, execpt that i’d recently read enough to make me look semi-smart. my reading was of samuel h. baron’s bloody saturday in the soviet union: novocherkassk, . review from library journal: baron (history emeritus, univ. of north carolina; plekhanov in russian history and soviet historiography) brings to light events of nearly years ago that foreshadowed the demise of the soviet union. robert berger’s wifi will beat up your wimax from wifi networking news: wimax hype, . reality wi-fi will out evolve and deliver connectivity at costs dramatically lower than wimax. wimax / . is just starting on its path to evolution, has a much smaller base of innovators and chipset growth volume. wi-fi is already far along on its core learning curve, has an easy order of magnitude larger base of innovators / investors and chipset growth volume. wimax hype will sputter out to reality of a niche backhaul and rural marketplace, wi-fi/ . will evolve and grow into many more realms and dominate the local area network (lan) / neighborhood area network (nan) / metro area network (man). berger’s conclusion is based on the history and development of earlier, wired networking technologies, where ethernet is the clear winner. he reminds us that “token ring, then . anylan vg, then atm” were all once considered leading technologies that would replace lowly ethernet, but didn’t. today, . products are shunned by wireless carriers, but their spread and market dominance will be hard to beat by wimax and . . ipod integration kits proliferate for home and car macnn reports the sonance iport will ship later this month, which must mean next week. anyway, the iport is a wall mounted dock that hides all the cables — audio, firewire, dock, others — in the wall. the macnn story includes nice pictures of the unit, including the beauty shot and a view of the ports and connectors. sonance makes no end of “architectural-audio” equipment, including those speakers you sometimes find hidden in the wall. falljuahinpictures fallujapictures (soon to be at falljuahinpictures.com) posts pictures too sad or scary to appear in most newspapers or even on this site. geolocation stumbling block: geourl host down a an old john udell piece at infoworld hints at geourls, but the goeurl site is down, and has been for a while. the concept sounds interesting: you mark pages with coordinates, then use gis to map those pages to geographic locations, finding pages and people of interest along the way. to join geourl, you add this kind of metadata to your homepage: i got interested in this sort of thing (geolocation) a while back, and i haven’t quite given up. copyright czar cometh? david rothman at teleread echoed the following: “buried inside the massive $ billion spending bill congress approved last weekend is a program that creates a federal copyright enforcement czar.” – lawmakers ok antipiracy czar, via cnet. sealing history democratic underground published a may story about bush administration efforts to replace the national archivist. the national archivist is the keeper of the nation’s records – the archives. the national archives control what information gets released to the public – and what does not. with so much power over how what history we see, the independence of the archivist’s position is paramount, lest one political party usurp that power. people who know these things were afraid when the previous archivist announced his intention to resign early, despite previous signals he intended to complete his full term. these people were doubly surprised when they learned the bush white house has […] nominated allen weinstein for the position, one who is held in dubious esteem at best, who has been criticized for having a penchant for privacy not becoming a national archivist and, to the surprise of many, was nominated without any consultation with outside experts – the first such time ever since , and in direct contravention with the wishes of congress as expressed in the house report accompanying the law that made the archives independent. had the previous archivist fulfilled his term, he would have presided over the release of george h. w. bush’s records. the new archivist will be able to lock up those records and along with the “w” files for the next ten years. with a straw man in place, the bushs can rest comfortably, but can we? liberty vampire jokir flickr’d this, writing: “great work — alex ross is one of my favorite artists…plus – it pretty much nails what’s up in the world, right?” ross’s website has mostly shows his comic book art and superhero imagery, and it took some time to find a reference to this piece. apparently it was for an article in the village voice and appeared on the cover. ross writes: wb says you’ll pay here’s the irony: an academic writes a paper that references and quotes relevant prior work, and is commended for the work. but, a journalist working on a book that quotes elements of pop culture risks a copyright infringement lawsuit if he doesn’t pay for his quotes. the fact is, “fair use” is not protected, and it can only be determined in court. fact is, the risk of lawsuit is enough to make most authors and other content creators license work for uses that most agree should be covered by fair use. u cozies to apple i’ve been warm and lukewarm on u for a while. i can’t deny that they’ve done some great stuff, but i’ve failed to appreciate some of it. take the band’s previous work, all that you can’t leave behind, for example. it seemed like a sad attempt to capture a younger audience, and was out of line from the band’s other work. aging is tough on everybody, but neither the band-members, nor their fans are getting any younger. the kinkos conspiracy engadget raised my fears a bit when they announced your laser printer will give you away: it was big news last month when a couple of researchers at purdue announced a way to trace documents back to their original printer or photocopier, but it turns out that xerox and most other laser printer and copier makers have been selling devices that encode serial numbers and manufacturing codes on everything they print out for years. click fraud arstechnica has a story about new google lawsuits. the company is getting sued by a porn purveyor for copyright infringement and is suing another company for “click fraud” — fraudulent clicks to google’s adsense advertising links. having recently taken on adsense links here at maisonbisson, i couldn’t help but pay attention. the ars story leads to one at c|net that explains: click fraud is perpetrated in both automated and human ways. predicting the computer of in (fake) steffan o’sullivan writes: “this is from a edition of modern mechanics magazine, predicting what the home computer will look like in . i think i worked on that printer once… how can i get a steering wheel like that on my office computer here?” the caption reads: “scientists from the rand corporation have created this model to illustrate how a ‘home computer’ could look like in the year . chernobyl tour update: there’s more pictures, even some video (look for links marked with the quicktime logo), and a bundle more nuclear and chernobyl-related stories. i almost fell into a trap that has snared quite a few before me. bookofjoe recently pointed to the story of elena, a motorcycle riding woman who claimed to brave the radiation to tour the area around chernobyl, the nucluear reactor that exploded disasterously in . a commentor quickly pointed out that her story has some history and is surrounded by controversy. google scholar arstechnica and bookofjoe both heralded the beta release of google scholar. my questions: “is it accessible via the google api,” and, “what does this mean for academic libraries?” i’ll be exploring both in time. in the meantime: library portal integration. how blue is my country? my father sent along a link with the following annotation: we all know the expression that “one picture is worth a thousand words.” well, here are several pictures of the same phenomena that tell the same story but give very different impressions. they illustrate clearly how pictures can be misleading (or should that be ‘leading’ ?). i found them very interesting. please look at all of them. the link lead to a web page by michael gastner, cosma shalizi, and mark newman of the university of michigan offering maps and cartograms of the us presidential election results. science of coercion roderick sent me a link to a story at common dreams: killing the political animal: cia psychological operations and us, by heather wokusch. a cia instruction manual entitled “psychological operations in guerrilla warfare” provides some clues. written in the early s (coincidentally, soon after bush sr. headed the agency) the document was part of the us government’s crusade to bring down nicaragua’s leftist government, by providing training and weapons to the contra rebels. coldplay i didn’t think i’d become a coldplay fan, but then i heard don’t panic in the garden state soundtrack and i couldn’t help myself. now i’m liking clocks. my only problem with all this is that everybody else likes it too. reviewing fcc rules on wifi use i wasn’t really paying attention in june when wifi net news reported on a fcc decision regarding control of wifi: the fcc says landlords, associations can’t regulate part use: the fcc’s office of engineering and technology says that the function of regulating and coordinating frequency use is reserved to the fcc itself. it’s a clear refutation of mall owners, airports, and condominium associations to limit use of wi-fi and other wireless technologies. why we fear the fcc the engadget headline on monday appeared at first exaggerated: the fcc says it has power over anything that can receive and play a digital file. but, the short news entry reveals the truth of the headline: in a brief filed in a suit brought against the broadcast flag by the electronic frontier foundation and publicknowlegde, the fcc argues that not only do they have the right to regulate that all digital tvs, settop boxes, digital video recorders, satellite receivers, dvd recorders, etc. ken nordine’s word jazz ken nordine may have the best voice ever. in the pantheon of deep soothing voices, ken nordine’s stands above the magnetic fields and mc honky, and about on par with barry white. content management below are loosely organized speaking notes for zach’s essentials of web development class that i guest-lectured/substituted on monday, november th. either we do the content management, or we get the computer to do it for us what is redundant and repetitive about web management? placement of branding elements. placement and updating of navigation elements placement and tracking of ads updating of lists, indexes, and other info as a site’s content changes these tasks consume time, but do not require great skill. what’s up with lowell and donuts? see the full what’s up with lowell and donuts flickr photoset with slideshow. follow that with the post-donut tour photo set. story/explanation/narrative to follow. sometime. donut shack eat-a-donut still hungry defensetech compares book to practice in fallujah the news from fallujah is grim. casualties are heavy on all sides, the city is being bombed to ruin, and those few civilians that remain are without water or power while bodies rot in the streets. defensetech reported on the fallujah push last week and included some quotes from the army’s new counterinsurgency operations field manual: concentrate on elimination of the insurgents, not on terrain objectives… get counterinsurgency forces out of garrisons, cities, and towns; off the roads and trails into the environment of the insurgents… avoid establishment of semipermanent patrol bases laden with artillery and supplies that tend to tie down the force. dangit: freefonts a part of me hates freefonts.com. it’s the part that has too often found just the right font, only to discover that the free or cheap knock-off version that i had didn’t have all the characters, like quote-marks and other punctuation. then i see a font like “accidental president” and realize what a sucker i am for font shopping. thanks to bookofjoe for the link. also, high tech-styles (get the pun? shatner’s return: has been william shatner has a new album out. most people receive this news with a smirk, or a chuckle, or a dumbfounded look. let me assure you, he can’t sing any better than you think, and probably not any better than in his previous albums. but here’s the thing: the first single common people, really is good. well, good in one way or another. i laughed the first time i heard it, and the second time, and again and again. ludicorp will be flooded with under-qualified applicants job ads reveal a lot about a company, what technology they use, what they’re developing, and what sort of culture they have. this one from ludicorp/flickr caught my eye: starting immediately, we’re looking for a great technical operations person. the ideal candidate can grow into a leadership role in technical operations and has broad practical experience on both the systems and networks sides. requirements: years system administration experience with linux and apache (some network administration experience strongly preferred) experience with both and bit systems experience with both hardware and software approaches for load balancing web serving and database traffic experience in firewall administration and best practices for security basic network design and administration current knowledge of hardware systems (servers and networking gear) prior experience running mid-sized systems ( servers) bonus characteristics: fish tacos oh decadence! veterans day provided not only a chance for reflection but also a rare thursday free from the classroom. so what to do with this open period of time? the answer was easy, dinner party. i have wanted to have my colleagues roxanna and john over, but time is always an issue. i phoned them up and they accepted. now the fun began — menu planning. while vacationing with my parents in vegas last summer we went out to marvelous food chain, the cheesecake factory. high tech-styles foof started out by making some interesting ipod sleves. now they’re offering foofbags for your ibook and powerbook. if you are looking for a funky alternative to neoprene, rubber or plastic to protect your apple technology from scratches, then we think that this site is for you. our foofproducts are handmade, simple and beautiful. foofproducts were originally created in a martello tower (dublin, ireland). they are now currently handmade using a pinnock sewing machine (sydney, australia). delicious library & earthcomber & what? i’ll be saving my pennies, because delicious library may be the coolest new app in a while. ars technica revied a beta and gave it an . out of — for a beta of a . product. people are right when they suspect that something very different is going on over in the mac corner of the software development universe. is it something crazy, or something sublime? you be the judge. money grubbing you’ll notice there are more ads on the site recently. it’s not because i need to recoup my investment in the site and need the pennies i get for these ads; it’s just because i’m a money grubbing bastard. anyway, this is the response i got to my application to the target affiliate program: we regret to inform you that target.com has chosen not to accept you into their affiliate program at this time. the campaign for klem the killer klown jones soda, the folks who make the extra-flavored pop with the intersting photos on the label have an online gallery where you can submit works to appear on future labels and vote on works already submitted. roderick’s girlfriend toni submitted an piece and he’s campaigning for it: hey there. toni is trying to get her klem the killer klown banner on a jones soda bottom. help her out by voting for her image! wpa cracked yesterday’s story about wired and wireless network security, and policy-based networking (sort of) was really just preparation for wifi net news’ wpa cracking story. glenn fleishman’s lead is quite direct, “we warned you: short wpa passphrases could be cracked — and now the software exists.” he explains further: a weakness in shorter and dictionary-word-based passphrases used with wi-fi protected access render those passphrases capable of being cracked. the wpa cracker tool is somewhat primitive, requiring that you enter the appropriate data retrieved via a packet sniffer like ethereal. better networks through policy back in the fall of , psu was still considering its wireless plans. things were moving slowly, and the decision makers seemed to be looking for answers in the wrong places. i’d been agitating for better answers, a simpler solution, lower costs, and more progress. my criticism landed me on the hot seat, and i was soon asked to be more constructive. my answers are in this presentation, the accompanying handout, and a handout for a followup meeting. at the time, the networking staff was leaning towards a proprietary . x-based authentication scheme that required specific client software and had limited hardware support. the package was rather pricey, would have required additional client software and hardware purchases, and was restrictive in its support of student computers. at an institution that supports over users, most of whom purchase and maintain their own equipment, the plan seemed to have a lot of shortcomings. i wanted the school to look at the wireless isp model, and consider the options used there. i also wanted the networking folks to explore network security over-all, rather than just wireless security, as most network threats affect wired and wireless networks in similar ways. i no longer work in the it shop, where i was a sys admin at the time, but this presentation and my arguments may have been successful. the school selected a commercial captive portal authentication system, just like the wisps. a lot has changed in the wireless market over the intervening year, but i’m offering the presentation here anyway. getting schooled on trademark law krispy kream, the donut folks, are itching to get krispy kream drive in on route in belsano to change their name. i’ve no idea where belsano is, but ower christina hoover says “we’re an ice cream fast food stand. it’s a drive in.” it’s been the hoover’s bread and butter since . what krispy kreme is really arguing is dilution of their “famous” brand. since going ipo a few years ago, krispy kremes have popped up everywhere across the county, from sbc park in sf to the excaliber in las vegas. ipod news galore ipodlounge has posted a lengthy buyers guide for the ipod and accessories. it’s a whopping pdf — they call it retro because it’s in magazine format. whatever, it’s packed with details and includes comparison reviews. mac is offering up a chatty review of the ipod photo. tera poked around and found an odd “photo import” command lurking in the menus. could this be the feature that allows camera users to import memory card contents directly? recovery lawrence lessig picked out a comment by adamsj that resonated with him: “i’m going to spend time these next few days looking for the america in my heart. it may be a while before i see it anywhere else.” the response was strong and swift. the first few comments were highly critical, even personally critical. john‘s comment seemed to sum up the republican view: you may also find it in the scores of millions of voters and nonvoters in between manhattan and san francisco whom the democratic party has repeatedly mocked, ridiculed, called stupid/ignorant/intolerant, and excluded for the past or so years. stealing from the bookofjoe once again, i’m echoing a lot of content from bookofjoe. i just can’t help myself. without the blog, how would i know about products like the flatulence deodorizer? the flatulence deodorizer — u.s. patent no. , , — is “guaranteed to eliminate embarrassment from odors associated with flatulence – forever – or your money back.” says the site: “try it, you’ll like it – and so will the others around you. bookofjoe says cia, nsa, defense, and others will make kerry president “the old guard of the cia, threatened and beleaguered as they haven’t been since the disclosure of ‘the family jewels’ by the rockefeller commission in , is striking back.” when bush turned to the intelligence agencies to produce “evidence” to support his neocon plan to invade iraq, they ponied up. to them, that’s what you do when you work in the executive branch and the executive gives an order. of course, much of the intelligence community’s behaviour was formed in the days when the buck stopped at the desk in the oval office. fear the takedown, part ii: homeland security copyfight and teleread both picked up on an ap story about homeland security agents enforcing trademark law. pufferbelly toys owner stephanie cox “was taken aback by a mysterious phone call from the u.s. department of homeland security to her small store in this quiet columbia river town just north of portland.” calls from law enforcement agents get noticed. calls from organizations charged with securing america from terrorist threats get fretted over. halloween : the movie food, booze, fire: halloween . links: picoserver and ivideo picoserver: japanese firm package technology is coming out with a x . x mm box called the picoserver that’s essentially a web/mail server with an ethernet port and three sockets for sensors (one out, two in). this could be a packaged implementation of the ibutton tini ics from dallas semiconductor. then again, it might not be. either way, it’s interesting and convenient. i just wish they were cheaper than the $ or so engadget claims they’ll cost. the october surprise npr’s senior news analyst, daniel schorr, reported wednesday that the bush administration has been busy keeping the bad news it has known about for months out of the press and away from the public scrutiny. iraqi explosives the bush administration knew about the tons of missing explosives a year ago, but still claims no knowledge of how they went missing or who might have taken them. their knee-jerk reaction, of course, is to say the explosives went missing before us troops invaded, but tv news video that has recently come to light shows us troops inspecting the explosives then being ordered away. what have you done for me lately, dubbya? unionvoice.org asks are you better off now than you were four years go? in his four years, george w. bush has taken away overtime pay, presided over the first net loss of jobs since herbert hoover and the great depression, proposed a percent cut in funds for children’s hospitals, sought tax breaks for companies that export jobs overseas and signed a medicare prescription drug bill that helps hmos and drug companies more than seniors. grandma had more sex fleshbot pointed to a story in the guardian that reports on a study by prima magazine that suggests married women of today have less sex than married women of the s. women in the s had sex an average of twice a week. but a survey found two-thirds of today’s women said they were too tired to manage that much. when i mentioned this to sandee, she echoed what prima says about it: warmonger ≠ support our troops on the heels of “<a href=”/post/ ” title="there _were no international terrorists in iraq until we went in“>there were no international terrorists in iraq _until we went in_” comes a story from alternet: “bush has failed the military on almost every level — marking the difference between being militaristic and pro-military.” discounting that he sent american troops into iraq on false pretenses, a real commander would fight for the welfare of his troops. fictional story asks: is there a right to life after death? the story focuses on the brain as an organ, in this case, an organ donated for medical research after the death of the host. what has prompted the lawsuits, protests and threats just over one year after the procedure is not the facts of the initial donation, but the university’s decision to terminate the experiments, and therefore the care, of the brain. what the [right to life groups] and their supporters claim is that brian schultz, the nine-year-old organ donor who legally passed away one year ago, is actually alive and well in the research lab. c&d = takedown = chill = limited creativity = limited speech ericka jacobs at copyfutures found my fear the takedown story about bits of freedom’s takedown study. she over-stated my effort; all i really did was quote text from copyfight, which they quoted from doom , but that’s how blogs and the web work. more importantly, erika explained a lot more than i did, including detailing takedown proceedures and safe harbor provisions under us and european copyright law. finally, she ends by quoting a report by chilling effects, a copyright resource center maintained by the “electronic frontier foundation and six law school clinical programs. prepare to get screwed by drm copyfight is picking up on something i started talking about a while ago: content owners want to re-sell you the things you already own. digital isn’t about copying, it’s about not having to re-purchase music just because the record company releases it in a new format (album, cassette, cd, beyond cd). the real threat: me me is about just that. hbo, for one, is very straightforward in its faq that the goal is to take away your time/space shifting rights in order to sell them back to you. the sweet taste of lead bookofjoe reports on a october washington post story titled: lead levels in water misrepresented across us. what the headline really means, however, is that lead levels are under-reported accross the us. “the problems we know about are just the tip of the iceberg,” said erik d. olson of the nonprofit natural resources defense council, “because utilities are gaming the system, states have often been willing to ignore long-standing violations and the epa sits on the sidelines and refuses to crack down. serene, calming video turn up your speakers to enjoy the serene music and pastoral scenes in this relaxing video of a car ad. [update:] the original link is broken; look for current links to the video in the text and comments of this newer story. malware, osx on old macs, brass knuckles arstechnica reports linux and mac os x get some love (?) from malware writers: some of you may have seen e-mails purporting to be from the red hat security team. the e-mail contains a link to fedora-redhat.com and prompts users to download and install a patch for fileutils- . . , stating that a vulnerability could “allow a remote attacker to execute arbitrary code with root privileges.” the “patch” actually contains malicious code that will compromise the system it is run on. “there were no international terrorists in iraq until we went in” it made some news when former british foreign secretary robin cook, who resigned from the cabinet over the iraq war, said: “there were no international terrorists in iraq until we went in. it was we who gave the perfect conditions in which al qaeda could thrive.” now, news organizations around the world are quoting the iaea in saying: nearly tons of conventional explosives that can be used in the kind of car bomb attacks that have targeted us-led coalition forces in iraq for months have vanished from a former iraqi military installation, the un nuclear agency said monday. ribbons a story on npr’s morning edition this morning declares: yellow-ribbon magnets carry complex meaning. the library of congress’s american folklife center tells the history of the yellow ribbon. though its conceptual beginnings are mixed, penne laingen was the first known american to tie a ribbon ’round an ole oak tree in hopes of the safe return of a loved one from conflict or captivity. it was , and her husband was among the hostages taken that november in teheran, iran. duties and responsibilities “i really don’t know what he did for us.” — said recently about me by my old manager to a former co-worker. cliff points at stuff so, cliff points at stuff a lot. it turns out that he’s pointing in every picture in my photoblog that he appears in. sure, it’s only five out of five photos, but it’s still %! more photos from maisonbisson in car ipod, take engadget echoed a story from autoblog (duh, i just noticed that they’re both from weblogs inc.) about an ipod integration kit that works with most all -or-newer cars: ipod car. first, it gives a clean line-in to the stero from the ipod, then it gives next and previous track as well as rewind and fastforward control on the stereo. sure, you can buy a bmw and get the same deal as an option, but this is cheaper. digital camera reccomendations a friend asked me what digital camera she should buy. her criteria were that it be small and inexpensive. my answer: the pentax optio s with a mb or gb sd card. why? it’s less than an inch thick, is hovering at just over $ , and works well. my slightly upscale alternative is the olympus stylus , but xd memory cards are much more expensive than their sd cousins. still, olympus’ new stylus verve looks like a winner. red sox the red sox did an amazing thing last night: they won. there’s a lot of talk about how historic the four wins in a row come from behind victory is, but for most people, it’s enough simply that they won, and they beat the yankees. close to home, psu students, and students all over new hampshire and massachusettes, expressed their joy over the sox’s victory in a way that has mature adults™ shaking their heads everywhere. i’m no economist, but… it’s an old story, the growing gap between rich and poor, and it’s probably booring as hell to most. thing is, i fear it’s shaping america in more ways than can be counted. i’ve been at a loss to make a clean argument about this, so all i can do now is give you this: across the great divide: in , ceos made times as much as production and non-supervisory workers. fear the takedown copyfight points me to doom which reports on bits of freedom‘s recent project: dutch civil rights organization bits of freedom has run an interesting experiment: they put up a text by a famous dutch author, written in to accounts with different isps. then they made up an imaginary society that is supposed to be the copyright holder of the author in question, and sent copyright infringement takedown notices to those isp via email (using a hotmail account). “try a florsheim maneuver” quotes from the bookofjoe: “the bleeding always stops.” …my favorite of the zillions of wonderful, pithy, often-harsh apothegms i’ve heard in my years in medicine. there’s more: “try a florsheim maneuver” [kick him to see if he’s dead or faking] “we won’t know until the autopsy.” [actually spoken on internal medicine rounds by a resident when i was in med school, in response to the question, “what’s he have? tv-b-gone wired news ran a two page profile of the inventor and his creation. just two weeks before the us presidential election, npr found time run an interview with the inventor. gizmodo rants angrilly about it. clearly, a device that shuts of televisions gets attention. tv-b-gone is a one button remote control who’s only purpose is to turn off televisions, whereever they may be. from wired news: the idea for tv-b-gone was born at a restaurant in the early s, when altman and his friends kept paying attention to a tv in the corner, not to one another. monday politics sex and politics, voter registration at strip clubs “ashcroft used to care more about pornography than terrorism,” says scot powe, professor of law at the university of texas. “the guy is a throwback to the early s; maybe that’s being too generous.” <p> […] </p> <p> david wasserman, a first amendment attorney, [says:] “my fear is that a second bush administration will unleash a slew of prosecutions against adult entertainment web sites, video stores and producers of adult films. monday copyfight disney thieves peter pan from copyright-holding childrens’ hospital charity peter and the starcatchers by dave barry and ridley pearson and published by disney’s hyperion books is billed as a prequel to the children’s classic, peter pan. […] but the hospital charity says [it] is getting nothing from peter and the starcatchers — which has been on the new york times best seller lists, has had an extensive author tour and has its own web site. monday tech now that wifi access is common, wifi-dependant applications are starting to appear. providers are finding out that the key to encouraging usage of hotspots and the key to leveraging hotspots to boost business is by offering applications that customers can use. <p> </li> <li> <a href="http://wifinetnews.com/archives/ .html" title="rest stop wifi roundup">rest stop wifi roundup</a><br /> <blockquote> <p> texas has signed a contract to install wi-fi at locations by oct. sunday links links: starting with politics, going to copyfight, ending nowhere. on the mediathis week is reporting on the controversies about sinclair tv and bush’s wiring, looks at why there’s a dearth of local real local news, and, most interestingly, compares bush’s lies to kerry’s exaggerations.the whole show is available as mp . realclearpolitics lists polls in swing states and elsewhere. earthbrowser (for mac) gives us a glimpse of the world, showing swirling clouds and other weather, but hiding the politics and tension. football injuries joe was telling his son, justin, about his college football days. it was mostly a tale of his injuries, including one that required he have fluid drained from his knees daily before practice. he says it hurt. it hurt a lot. it hurt to drain the fluid. it hurt to practice on it. it hurt throughout the day and night. justin asked why he would do such things to himself. because he could not imagine doing anything else. local cinemas while yahoo movies is okay, it doesn’t track all the local theaters. fortunately, many of them are online: the nugget, hanover lebanon lincoln cinemas smitty’s/chunky’s tilton then there are the drive-ins: meadows drive-in route , woodsville, n.h ( ) - fairlee drive-in theater fairlee, vt. ( ) - st. louis i’m ashamed to say that st. louis, missouri, wasn’t on my list of must-see-cities™. it’s not that i thought i wouldn’t like st. louis, it just never crossed my mind to go there. i’d also forgotten about the arch. i ended up in st. louis because it was hosting the library information technology association annual conference. i did the arch friday morning, before the conference. the day was rainy and gray, but the arch still stood out as an amazing structure. veicon thin client solutions the theory is that thin clients save money over the long-haul because they require less maintenance and management, have longer useful lives, and can be purchased for about the same or less money than the pc you might have otherwise used. the problem is that it’s very different from the normal practice and not many people can explain exactly how it works. so, in the absence of good information, most people go on like they always have and ignore the possibilities of thin clients. qr codes qr codes are starting to appear everywhere. i’m intrigued and i want to know more about them. here are some links i dug up and hope to return to: wikipedia on qr codes schubart’s wikipedia on qr codes jphonegames on qr codes qr code generator qr codes and php a better qr code generator winging into cleveland the wing dips toward the ground while turning for the cleveland airport. lake erie is visible underneath the clouds at the top of the frame. two more photos from this series are posted in my new aerial & scenic set at flickr. what liberal media? now on cnn.com: sinclair broadcast group, owner of the largest group of television stations in the nation, plans to air a documentary that accuses sen. john kerry of betraying american prisoners during the vietnam war, a newspaper reported monday. this story is bigger than it looks, and i almost let it slip by without mention because i couldn’t fully address it. but ignoring it won’t make it go away, so…. libraries under fire komo tv is reporting big brother™ is watching, even in small communities off the beaten path. deming, washington, a town of with a library that “isn’t much larger than a family home” is facing a showdown with the fbi. the fbi wants to know who checked out a book from a small library about osama bin laden. but the library isn’t giving out names, saying the government has no business knowing what their patrons read. redlightgreen teleread reports: redlightgreen.com, a creation of rlg, searches through million books based on such criteria as author’s name, title, and subject matter. not full text search–but still useful. over at redlightgreen, they say it “helps you locate the most important books and other research materials in your area of interest, and find out whether what you need is available at your favorite library.” foggy st. louis from the top of the arch this is my second try at stitching these photos together. i decided to give up the illusion of the single shot, and added the white borders to make clear that this image is a composite. the resolution is way up on this one, and it shows. the baseball stadium is clearly visable on the left, the football dome is on the extreme right. click the picture for larger (or smaller) views. the rumble in st. louis this text has been moved from the scenes from st. louis story so that it can be filed, more correctly, in politics & controversy. unable to get into the “town hall” to take part in the debate personally, i went looking for a place to watch it. sadly, the sox game pre-empted the debate at most bars, but the drunken fish was showing it, with subtitles only. regarding the debate, oliver willis has a clip titled “watch your president flip out of his gourd” and everybody is asking is this bush’s dean scream™? bowling museum and hall of fame things learned at the international bowling museum and hall of fame (and easily repeated as quotes from their online history page): sir flinders petrie, discovered in the ’s a collection of objects in a child’s grave in egypt that appeared to him to be used for a crude form of bowling. if he was correct, then bowling traces its ancestry to bc. […] there is substantial evidence that a form of bowling was in vogue in england in , when king edward iii allegedly outlawed it to keep his troops focused on archery practice. copyfight friday microsoft ceo steve ballmer did another one of his monkey acts when he went ape about music and drm. most people still steal music…we can build the technology but there are still ways for people to steal music. the most common format of music on an ipod is ‘stolen’. it could just be a picture of what happens when microsoft wakes up and realizes it doesn’t own and can’t control everything, but it also reveals a lot about where the company is going. ballmer could have said that the shifting of purchased music from one device or format to another is a legally protected form of fair use (at least for now). instead, he argued something like “microsoft’s drm is the only solution to piracy.” anyway, it’s a crock of shite. teleread (always an anti-drm advocate) has picked up on it. — and — riding mower gizmodo has this picture of what they describe simply as a “homebrew riding mower.” i can’t help but like it, and i have a feeling my friend joe will be trying to make one of his own soon. stealing from the bookofjoe as long as i’m quoting content from bookofjoe, i might as well post these two other links i got from there this week: douwe osinga’s visited states dynamic map dohicky and awfulplasticsurgery.com. fox news just makes stuff up most people know i’m not a huge fan of fox news, at least in part because fox news is no great fan of mine. al franken and eric alterman are rather detailed their explanation of just how conservative fox is (it’s like the tower of pizza leaning toward texas; actually, it’s like the tower layed down in texas). but you’d have to figure that even conservatives would have trouble keeping a straight face while making up lines like this: “‘didn’t my nails and cuticles look great? st. louis wifi panera offers free wifi in about locations. the odd thing is that even though their listings didn’t name a location near my hotel, a proximity search found one in my hotel: westport plaza westport plaza maryland heights, mo then there’s also apple store west country: west county center des peres, mo …just a quarter mile east of on manchester. eccentric or autistic, you decide bookofjoe ran a story about eccentrics by david weeks. his story is really just a listing of the characteristics of eccentrics as quote from the book, but it makes a good game to calculate how eccentric a person is. try the list on for size: nonconforming creative strongly motivated by curiosity idealistic: wants to make the world a better place and the people in it happier happily obsessed with one or more hobbyhorses (usually five or six) aware from early childhood that he is different intelligent opinionated and outspoken, convinced that he is right and that the rest of the world is out of step noncompetitive, not in need of reassurance or reinforcement from society unusual in his eating habits and living arrangements not particularly interested in the opinions or company of other people, except in order to persuade them to his – the correct – point of view possessed of a mischievous sense of humor single usually the eldest or an only child bad speller what isn’t so funny or joyful is his later story about autism, accompanied by the iconic diagnoses sheet pictured at right. feel safer now? i guess somebody will sleep better at night knowing our department of homeland security is shaking down music and video pirates. their new plan: strategy targeting organized piracy (stop), a crackdown on the theft of u.s. intellectual property such as pirated compact discs and knockoff auto parts. the effort is consuming the attentions of attorney general john ashcroft, commerce secretary don evans and u.s. trade representative robert zoellick and senior officials from the department of homeland security. weird museum tour, september travelling buddies, willberry & cliff i should thank roadsideamerica.com for making a rainy day a _fun day_™. will and i were supposed to go on a hike, but the rain killed that plan and most anything else we could come up with. roadsideamerica.com gave us alternatives. randmcnally gave me directions. cliffy met me in warren, we picked up willberry in manchester, and headed off to our first stop in leominster. tales of woe i just got im’d by my friend karen. her sister got married this past weekend and they were all in new hampshire for the event. here’s the transcript: hi – sooooo sorry we did not call the wedding was insane everything kept going wrong all weekend i didn’t really expect you to call. not that i didn’t want to see you guys, but weddings are crazy stuff. the rehersal restaraunt closed, the chef for the reception quit, the organist over booked, the salon canceled our reservations, my wedding dress never got finished, it rained during the party at my mom’s house…. cocktail manifesto we’re huge fans of the new joy of cooking by marion rombauer becker, irma s. rombauer, and ethan becker. hardly a meal goes through our kitchen that isn’t shaped in some part by the recipes and general information in its pages. a recent discovery was joy’s description and defense of cocktail parties. so, when a book as serious and valuable as the new joy of cooking raises alarms about the declining future of cocktail parties, we listen. canned meats monday some time ago, a box with the above pictured contents went to chuck robidoux. he wrote back: nothing starts a monday off like kippered seafood snacks and deviled ham with a side of spam and potted meat food product followed by vienna sausage, all washed down with some icey cold clam juice. now i am ready to face the day. yours meatily, dr. meaty mcmeat meatofski meatovich hamkowsky-beafeau porkson politics, terror, & sexual identity i hadn’t given it the slightest thought, but then i read tinynibbles.com’s travel advisory (this site has been referenced previously at maisonbisson). what do politics, / , & sexual identity have to do with each-other? read: traveling when you do not appear as the gender on your identification is much more tricky…. if your driver’s license says “f” and you look like an “m,” you’ll have some explaining to do. with the patriot act, when they run your license through at the airport, it automatically links to all other federal databases, and if there are any discrepancies, again you’ll have some explaining to do — and a possible delay. nixichron & techno-retro lust decades ago, nixie tubes were used as indicating devices in many different types of instrumentation, and ultimately replaced by the cheaper – and unattractive -led display. having been obsolete for almost a quarter century, these glowing bottles of ionized gas have attracted another generation who appreciate their beauty and mysterious function. the display tubes may be decades old, but the clock is gps accurate. those who’d rather just fiddle with nixi tubes than spend a pile on on a clock (though we all agree it would be well spent), can buy bare tubes here. feeling the web: pulse, buzz, zeitgeist flickr zeitgeist blogpulse yahoo! buzz google zeitgeist round one: kerry , bush thank npr for putting audio of thursday’s presidential debate on their site. spin-masters will be working this one over for a while, but the original is the most important. there were people who expected bush to come off in his casual, frat-boy manner, but he didn’t. he stumbled, he got red-faced, and he never answered any questions. republicans like to stay on message, but their message, already short on details or plans, has grown stale. the mac vs. pc debate i generally don’t get into this, but a series of columns by paul murphy at linuxinsider (linuxinsider!) caught my attention. in macs are more expensive, right?, he compares apple’s offerings to dell’s and finds the pcs cost about the same or more than similarly equipped macs. at the low end…the pc desktops are marginally less expensive than the macs — if you can do without their connectivity and multimedia capabilities — and considerably more expensive if you can’t. film performance licensing in case the notion strikes me again, i’m putting these links here so i can find them in case the notion strikes me again. the aforementioned notion is one of wanting to do public performances of movies, who know why. this would be easy, except for copyright, so these links are for information about getting performance licenses for films. wisconsin department of public instruction’s information on performance, with links to disributors. cultural revolution-era clip art book oldtasty has posted a collection of pictures scanned from the pages of a clip art book of the cultural revolution. i’ve always enjoyed look of communist art, and i’m particularly pleased with this showing. things you can do with isbns jon udell has been working on librarylookup and other mechanisms for finding library content on the web. in the meantime, librarytechtonics, library stuff, and the shifted librarian have picked up on it. part of it is about oclc making their records available to search engines. now both yahoo! and google in the game. so what you do is put your isbn in the properly formatted url and you’ll be given links to libraries that hold it: via google and via yahoo! a day in the life of joe i’m not sure of the origins of the following text. there’s nothing patently false in it, so i’m posting it here for all to ponder. joe gets up at a.m. and fills his coffeepot with water to prepare his morning coffee. the water is clean and good because some tree-hugging liberal fought for minimum water-quality standards. with his first swallow of coffee, he takes his daily medication. his medications are safe to take because some stupid commie liberal fought to insure their safety and that they work as advertised. all but $ of his medications are paid for by his employer’s medical plan because some liberal union workers fought their employers for paid medical insurance – now joe gets it too. korean thanksgiving jong-yoon kim emailed to tell me today is chusok, the traditional korean thanksgiving day, when families gather and give thanks to their forebears. according to the lunar calendar, today, sep th, is aug th, the korean thanksgiving day. tonight, we will have the biggest and the brightest moon of the year. traditionally, we pray to the moon for our hope and believe that the moon will listen to us. enjoy the moon and have a great day. google news gamed? what happens when machines edit our news? what happens when news sources game google news to raise their ranking? online journalism review is asking that question, and has some interesting answers to report. it seems conservatives and conservative-biased news or quasi-news organizations use people’s full names, while mainstream sources and those with a liberal bent often use only the last name. the result: google newsing for “john kerry” results in some incredibly negative stories, but “george bush” is largely positive. ultra portable i’ve been interested in ultra-portable computers for some time. my first such computer was a newton message pad , which remains useful despite its age. the newton was replaced by a palm m that cost less and did less. no more email, web browsing, no writing or word processing. in short, nothing more than addresses, calendar, to-do lists, and a note or two jotted down using the infuriating graffiti text recognition. home-made arcade i found retro gamer magazine on the rack last week and couldn’t hep but pick it up. it’s issue six with a feature story on building both stand-up and cocktail arcade cabinets with pcs running mame (which isn’t to say you couldn’t use a mac instead). for now, i want to keep track of these related websites: check ultimarc for arcade buttons, sticks, and fancy interfaces to make them work. throwing google a bone for cliff cliff worries that his website, spiralbound.net, doesn’t get indexed by google often enough. he’s a good guy, so i figure i’ll prime the pump for him. here, google google. solaris docs: migrating veritas volume manager disk groups between servers{# } solaris docs: solaris disk partition layout{# } solaris docs: copying a boot drive between disks with different partion layouts if you’re looking for those, you should also take note of these here at maisonbisson: configuring sun t storage arrays and things to remember while doing upgrades on mission critical sun equipment. techlinks dartmouth college in the wifi limelight, again as they replace their . b aps with a+b+g aps. wifi net news wonders how wimax will change dartmouth’s plans next time around. foof makes some snazzy looking ipod and laptop cases. michelle has set up an example of the worst designed web page ever. it’s a counter-example thing. brad templeton brought a voip phone to burning man. it’s automotive week in the blogs first gizmodo published a feature on in-car computers. arstechnica got into the automotive theme by reporting the international cxt story. not to be outdone by gizmodo, engadget reported on the ultimate car computer install: a tatra with a mac in it. for some reason, i went looking at the tatra car-mod and found tatra trucks which seemed to connect back to arstechnica and caesar’s gushing about the hemtt. after all, the largest of the tatras is called the kolos (colossal). roderick’s sites roderick has been sending me links and i’ve been lax about posting them. some of these links are nsfw, and one of them is a present back to roderick. i’m not going to comment, because i’m lazy because i don’t want to prejudice you. corporate mofo a fundraiser billionaires for bush hello laziness: management tips from the executive slow lane kite aerial photography i got sort of excited about kite aerial photography a couple of weeks ago in a post about photoblogging. i was amazed with scott haefner‘s work and especially impressed with his vr picture of slain’s castle in scotland. scott is pretty serious about kap, and it shows in his description of his rig, but what’s an amateur or naive fool to do? engadget is doing features on things to do with an old digital camera, and this week they tackled kite aerial photography. scenes from the museum of bad art the museum of bad art (moba) in the dedham community theater. it’s in the basement outside the men’s bathroom, illuminated by a single fluorescent light hanging from the ceiling the moba slideshow. more photos from maisonbisson. sandee’s clothing donations it’s photos, but i think there’s actually only items. no, i’m not sure why i photo’d each one. more photos from maisonbisson the plastics museum the plastics museum is in leominster, ma, and online at plasticsmuseum.org. the national plastics center and museum is a non-profit institution dedicated to preserving the past, addressing the present and promoting the future of plastics through public education and awareness. the educational staff has supported this mission throughout the years by conducting hands-on science programming for schools, organizations and the plastics community. and, if you’re a lucky kid, your school might get a vist by the plastivan: the bellingham accident i pulled up to the stop sign at the end of north st., looking to turn left onto route in bellingham, ma, at about : pm on saturday september when i saw a red dodge neon coming down the hill towards me with its brakes locked up. it was a busy intersection and with roads still soaked from the heavy rains that had had been falling all that day and the day before but had recently cleared. funky time gizmodo pointed out this fancy clock by kikkerland. being the clock-fiend i am, i had trouble not looking for more. ship the web seems to have kikkerland’s entire catalog of clocks, which is more than enough to make me drool. of course i want this one and this one and this one and this one. “i wanted a tatra, so i got a tatra” engadget picked up on the story about the tatra with a mac in it. i couldn’t help checking for changes since i first saw the story. there’s a new version of dashmac, the control software, and it seems he can now control his car via sms messages, but most things seem in-line with where he was going. the thing is, i can’t help but get interested in the car itself. i sort of went gaga for tatras after seeing the original story and doing some research. megapixels, cheap engadget was quite excited about the gateway dc-t megapixel camera, now selling for $ at various retailers. i know more than one person who wants a cheap digital camera that doesn’t suck, so i went looking for reviews. steve’s digicams has some really detailed reviews, so i was excited to see they covered the dc-t . they say it’s a rebranded toshiba pdr- . their review is based on a price of $ , so weigh that when considering their so-so conclusions. mobile carrier wireless networking i put together a list of wide area wireless networking options in semi-rural areas for a friend recently. it’s far from complete and may not be accurate, but it’s a start. the coverage area i was looking for was north of portland, me, but we all know coverage maps lie and local conditions vary. i focused on pc-cards, but most carriers sell phones that can be attached via usb port. these aren’t campaign commercials ebaum’s world added a couple of funny bush videos recently. what is soveriegnty? bumble mumble. two things: if he was a lot smarter, he would have known the meaning of “sovereignty,” but if we was just a little bit smarter, he would have known that the question was about how his government would treat native americans and answered that. the claim is that this is a video of george w. techlinks the save betamax campaign has nothing to do with videotape and everything to do with the fair-use rights that allow us to legally convert cds to mp s or legally use tivo to keep up with our favorite shows. these rights are under siege by content producers who want to charge consumers for every use. copyfighters look here. rumors are that oqo will release their ultra personal computer soon. be better dork: command line stuff be geeky and look at the apache modules: ``` /usr/sbin/httpd -l compiled in modules: core.c prefork.c http_core.c mod_so.c ``` set your path: ``` path=$path:/usr/sbin export path ``` project censored’s annual roundup project censored has released their list of the most censored stories of - : # : wealth inequality in st century threatens economy and democracy # : ashcroft vs. the human rights law that holds corporations accountable # : bush administration censors science # : high levels of uranium found in troops and civilians # : the wholesale giveaway of our natural resources # : the sale of electoral politics # : conservative organization drives judicial appointments # : cheney’s energy task force and the energy policy # : widow brings rico case against u. high and mighty i can’t help but steal the title to keith bradsher’s excellent book about the titanic rise of suvs on our highways. bradsher, in his book, makes note of efforts at freightliner and mercedes to release uber-suvs based on the companies’ commercial truck bodies but weighing in at just under the limit at which commercial drivers’ licenses would be required to operate them. both companies eventually decided against it, but now international is going forward with similar plans. the international cxt is the latest entry in the super suv market. at nine feet tall, over feet long, and cruising at six to ten miles per gallon (diesel), it’s the kind of vehicle any texan could love. ars technica went off-topic to give me the heads up. along the way, caesar got all excited about the hemtt. sewer in the woods, unknown flower found the left image in the woods near warren nh this weekend. photo is composite of four smaller pictures taken with my clie th , but the scene is entirely real. seperately, i found the flower on the right a week before, while hiking around the other side of the lake where the sewer scene was found. i’ve no idea what it is, but i’m not against finding out. more photos from maisonbisson pepper pad i can’t help but want one of pepper computer’s pepper pad hand-held computer thingies. it’s available for pre-order now at only $ . but what is it, you ask? according to pepper, it’s “either as a user’s only wireless computing device or […] a convenient, easy-to-use accessory to a pc.” it’s a linux-based palmtop computer with gb hard drive, x . ″ display, . b+g, and a bunch of other stuff. in-car computers the age of the in-car computer has come. one vendor calls them “carputers,” and gizmodo lays it out for those who want an intel-based cpu in their trunk/under the seat/in the dash. what to do with a computer in the car? now that computers have moved out of the den to become part of the home entertainment center, users are anxious to use that library of downloaded music in their cars too. claim: beverage choice = politics i’ve been a little slow to blog these things lately, but this comes from beverageworld magazine. they published the results of a poll connects beverage choices to political affiliation. they break the politics down into six choices: democrat, republican, independent, independent liberal, independent conservative, and none of these, then they compared booze and soda-pop choices for each. of booze, democrats and “none of these” drink the least. the three varieties of independents seem to drink the most. conservative independents are % more likely than the national average to tipple some variety of whisky, while liberal independents are % more likely to drink imported beer. overall, the liberals are more likely to drink than the conservatives, but republicans are more likely to drink than democrats. the implication, of course, is that candidates can woo swing drinkers by offering the right drink to the right person. which, as my wife would say, is just good manners. claim: sleep position = personality about a year ago, reuters reported on the results of some sleep research from professor chris idzikowski, director of the sleep assessment and advisory service and a visiting professor at the university of surrey in southern england. the story is still online now at wellspan.org and netscape news. in summary, your sleep position is a reliable indicator of your personality. here’s how it goes from netscape’s version of the story: nh license plates for a variety of reasons, i was happy to discover that nh allows drivers to check the availability of vanity plates online (though, somewhat nervous find that the state uses microsoft servers). the search enlightened me to a variety of plates i didn’t know about. we’ve all seen the “veteran” and “purple heart” plates, and a few “antique” plates, but i’ve never seen a “street rod” plate. but there are even more plates available. in car ipod without wanting to get into the rest of the story, i’m now trying to figure out how to plug an ipod into a scion xb. the xb comes with a stereo by pioneer, but i haven’t been able to get details about what inputs it supports. installer.com and logjam both offer connection kits that appear to give me rca aux inputs to the radio head unit, but pioneer offers a simple ip bus adapter that might also do the trick. photoblogging, etc. i think i’m a fan of flickr. it makes photoblogging easy and fun. easier, anyway, than setting up an email to blog solution on my own, and the community features are more fun than i’d expected them to be at the outset. flickr more or less automatically puts up a blog entry for each photo i upload (though i still have to configure the layout features to my satisfaction). anyway, in related web surfing, i came across the following: mini golf minigolf is very serious business. very serious. more photos from maisonbisson texas’ crony politics and the presidency i finished cronies by robert bryce recently and i can’t help but tell people about it. i hadn’t really wondered why so many presidents and vice-presidents have been from texas, but bryce did. “two of the last three american presidents — and three of the last eight — have been texans. each of them got to the white house by exploiting a network of money and power that no other state can match. co-worker it turns out that one of my co-workers is blogging over at live journal. rnc eve nyc’s sex workers expect to be extra busy while the republicans are in town. there’s been talk of terror alerts. get some backstory here, then read ridge issues alert for u-boatattacks on northeast coast (and laugh). google seems to think maisonbisson and alandwilliams are similar. there, i found pleasure boat captains for truth and cabbies against bush. it seems the cabbies are offering free rides to kennedy and newark airports for gop delegates who are willing to go to iraq to fight. muppin tongue muppin wags his tongue, leaves slobbery mess on lens. more photos from maisonbisson republican national convention to be windfall for nyc’s sex workers the new york metro reports that the sex industry is expecting a to percent uptick in business while the republicans are in town for the republican national convention this week. mary, a stripper at ten’s cabaret speaks from experience. she worked the rnc in philadelphia and expects the strip clubs in nyc to be “really crowded” during the convention, adding, “the girls have been talking about it literally since june. heat: dell server thermal load (btu/hour) it’s a shame that dell doesn’t list the thermal loads of their products in the datasheets at the online store. it’s a shame that it took several google searches to get close to a link with the info, then mine the google cache of a dell support forum and find/follow a chain of links before i could get that detail. as it turns out, there’s some dell and the environment page where they list all their products and their environmental properties/certifications/regulatory compliance. camera goes all to hell, bits recovered from memory card sandisk is playing this as the coolest thing that ever happened. some photographer planted a couple cameras to photo the demolition of a bridge over the mississippi, the explosion was bigger than he expected, he lost one of the cameras, but the cf card survived in working order. mobilemag has the story. sandisk has a press release. and every blog in the western world is echoing it. the photographer is don frazier, a staff photographer for the southeast missourian newspaper. o’reilly mac os x conference i trust o’reilly’s books, so when i see they’re running a conference about something i’m interested, i get excited. the third annual o’reilly mac os x conference is like that. with speakers like andy ihnatko, david pogue, and rael dornfest and tracks covering digital audio, “insanely great mac”, programming & scripting, and system administration, this could be the summer macworld that no longer is. the effect would be complete if it were one the east coast. clie annoyances, part the clie th stylus is one of the most annoying parts of the palm os-based handheld. it’s small, too small. it telescopes to an almost usable length, but it’s still too narrow to hold comfortably. so i’m a little reticent to buy a replacement for the one i lost. also, you’d think the clie could have come with a decent sync cradle, or any sync cradle. and, while i’m whining, why can’t the keyboard also work as a sync cradle? making a dat/dds tape drive work on red hat enterprise linux we could see messages about the tape drive in dmesg, but it wasn’t giving the device name. we tried working with /dev/st , but we kept getting errors. everything seemed right, but it didn’t work. it turns out our scsi card was the problem. it wasn’t being properly recognized. after a tip, we tried the following: /sbin/modprobe aic xxx where “aic xxx” is appropriate for our adaptec card. we checked lsmod and found the aic xxx stuff properly initialized there (shortened output): itunes vs. firewalls itunes on the pc on my desk (notice i feel more possessive of the desk than the pc) hasn’t been able to share music to or from itunes on my powerbook. blame the firewall. a moment of googline led me to travis saling’s guide to enabling itunes sharing through a firewall. here’s the ports that need to be open: port tcp port udp however, he notes: the conservatives vs. the academy alternet has a story by joshua holland about the right’s crusade against lefties on campus. as i saw with my experience with the conservative sniper that was trolling here not long ago, the conservative mission is to criticize everything that’s off their message. holland describes this as “backlash” politics: the backlash came about when traditional big-business conservatives, tired of facing the resentment of ordinary working-class americans, stumbled onto ‘wedge’ social issues in the s. configuring sun t storage arrays sun’s t documentation is available online: the sun storedge t and t + array configuration guide explains physical configuration. the sun storedge t and t + array administrator’s guide explains the software side. the short course: creating volume ‘v ’ using half the disks: vol add v data u d - raid standby u d vol init v data vol mount v creating volume ‘v ’ using the other half of the disks: vol add v data u d - raid standby u d vol init v data vol mount v listing volumes: faith-based missile defense defense tech is reporting on the progress and prospects of missile defense (and their title is too good to pass up). early in his administration, president bush put a whole lot of stock in “faith-based” initiatives to solve domestic problems. now, the president seems to be taking the same approach to military matters. defense tech quotes slate’s fred kaplan: in the past six years of flight tests, here is what the pentagon’s missile-defense agency has demonstrated: a missile can hit another missile in mid-air as long as a) the operators know exactly where the target missile has come from and where it’s going; b) the target missile is flying at a slower-than-normal speed; c) it’s transmitting a special beam that exaggerates its radar signature, thus making it easier to track; d) only one target missile has been launched; and e) the “attack” happens in daylight. fbi investigates a friend sent this along yesterday: i was visited, a couple of weeks ago by an fbi agent investigating whether or not i was involved in terrorist activities. seems one of my neighbors (i don’t know who) placed an anonymous call saying that “[name deleted], who works for [airline name deleted] and lives [address deleted], resembled a terrorist on a watch list.” so, the guy had to come over here and make sure i was not evil. galleries of oddness i ran across darren barefoot‘s hall of technical documentation weirdness, where he catalogues “wacky, bizarre, surreal and otherwise strange examples of technical documentation.” considering the number of poorly done or just weird technical illustrations we’ve all seen, you’d think the gallery would be larger. when done with that, go to the snope‘s urban legends reference pages photo gallery. you’ll laugh at some of the images (and you’ve seen at least a few of them already), but the real entertainment here is in the stories that supposedly explain what’s true and what’s false. mac consulting i get a number of requests for help with people’s macs. they’re are often willing to pay, but the truth is that computer support (on any platform) is one of the things i least like to do. a typical question looks like this: we’d like to upgrade or replace our aging mac and have questions about how to upgrade or what to buy. we’d also like to network our computers on opposite ends of our house and are wondering about wireless. extra links swim-up, floating blackjack tables for your pool. yes, the hard rock las vegas has similar stuff, but their minimum bet is too high for my game. there’s a sock subscription service, and it’s been around for five years. a chinese dvd player manufacturer has developed a unit that excels at playing china’s famous black market dvds. i’m not that excited about case-mods, but this predicta case-mod gets my nod. flying car options in commenting on the space race story, zach pointed out that the moller skycar is still under development (which is better than going bankrupt or just disappearing — like so many other good ideas have). if you poke around the site you can find video of flight tests and sales info. yes, they’re taking deposits for deliveries they hope will start in . meanwhile, the sky hasn’t fallen on the trek aerospace millennium jet either. o’reilly covers rss ben hammersley’s content syndication with rss has got me back on the rss wagon. hammersley covers the history and context of rss’s development in more detail than many other tech books have given their subject. i’m ashamed that i didn’t know rss got its start as “hot sauce” in apple’s research labs. you won’t find it on the web now, but hot sauce was an interesting technology demonstration in / . i’m also ashamed i didn’t know of the connections between efforts at creating the “semantic web” and rss ( . random/color-light/balloon lamp im jealous i didn’t think of these things before kyouei ltd. released them as a product. a dvd that fills your tv with solid colors to illuminate the room. a cd with tracks for tones: “when using the ‘random’ function, the cd will automatically select random tones, and make a new melody.” a combination of battery, led, and balloon that results in a glowing glob of latex. the only thing cooler than these is a little book titled count sheep that was filled with pages of identical sheep arranged in rows and columns, ready for counting. rnc anarchy writer paul schmelzer has a list of (civil disobedience?) actions against the rnc in nyc. among the actions planned: bikes against bush, radio jacking, backback broadcasts, wifi on wheels, and accurate crowd counts. crowd counts? it seems government bodies like to undercount the number of people protesting against them, so a few hactivists will be using technology to gather crowd images from above and use image analysis software to do the counting. we the media dan gillmor’s we the media caught my attention. from the publisher’s description: for the first time, bloggers have been awarded press credentials to cover the national political conventions. …grassroots journalists, including bloggers, […] are dismantling big media’s monopoly on the news. through internet-fueled, interactive vehicles like weblogs, these readers-turned-reporters are transforming the news from a lecture to a conversation. they’re publishing in real time to a worldwide audience that’s eager to read their independent, unfiltered reports. look ma, no fire protection alternet is featuring a story about the bush administration’s attempts to reduce nuclear power plant safety requirements. this news might have slipped by unnoticed, except mainichi daily news is reporting on a steam explosion at a japanese nuclear plant that killed four and injured seven workers today. bush’s plan, against this background, seems haphazard. at least this accident didn’t result in a radiation leak, the the tokaimura nuclear accident did. space race heats up it’s been almost years since sputnik began the space race and years since a few men hobbled about on the moon, but i don’t yet have a flying car and i can’t take an orbiting vacation. folks, the space race wasn’t won, it was abandoned. and that’s why we have the ansari x prize. burt rutan’s team seemed to be in the lead earlier this year with the successful launch of spaceshipone, the competition has been in the news lately. strange days this story is too complex for me to do it justice, but too interesting to ignore: the mainichi daily news is reporting chess champion bobby fischer has been jailed in japan. fischer, a one-time world grand master who represented the us in cold war grudge matches against the ussr, but has since mostly fallen out of public view and, perhaps, gone a little crazy, was arrested in japan for passport violations. juliusblog on coincidence: bush ratings vs. terror alerts juliusblog has a chart comparing approval ratings on a timeline with terror alerts. guess what? juliusblog makes the following observations: whenever his ratings dip, there’s a new terror alert. every terror alert is followed by a slight uptick of bush approval ratings. whenever there are many unfavorable headlines, there’s another alert or announcement (distraction effect). as we approach the elections, the number and frequency of terror alerts keeps growing, to the point that they collapse in the graphic. now listed in blogshares? i moment or two of ego-googling lead me to blogshares, where maisonbisson is trading me as a penny stock. oh well. cronies a co-worker just handed me robert bryce’s cronies. from the publisher’s description: texans are running the country — maybe the world. now the author of pipe dreams examines who they are, how they got into power, and how they reward themselves and each other, often at the expense of american taxpayers. no other province holds more political and economic power than the lone star state. two of the last three american presidents — and three of the last eight — have been texans. fear aint the word for it mix a born again christian who confuses christ and god (yup, check molly ivins for the quote), clinical and medicated depression, several million believers and call it the church of bush! fear is just the beginning. village voice: church of bush i started to make noise about this a few weeks ago in my story about fahrenheit / : i’m growing increasingly uneasy about the cult-of-bush-worship that brittany spears exemplified in her appearance in fahrenheit. the greeks expected questions and debate, so did the romans before the fall of the republic. egyptian pharaohs, mayan emperors, and soviet premiers may have killed or non-personed those who questioned them, but democracy demands otherwise. mysilo knowing that everybody wants a missile silo, bari has posted his for sale on ebay (thanks to defensetech for the pointer). silo world has the skinny on titan silo design, npr did a story on missile silo homes a few years ago, though most of the silos are empty, abandoned, and dangerous. still, there are one or two realtors that specialize in missile silos. news: bush bushed i hadn’t heard of capitol hill blue until a friend forwarded this story about bush’s paranoid isolation. first, i should say that paranoid isolation isn’t all bad. it worked well enough for ol’ <a href="http://retroplanet.net/ hughes.html” title="howard – “i’m not a paranoid deranged millionaire; goddamit, i’m a billionaire” – hughes">howard — “i’m not a paranoid deranged millionaire; goddamit, i’m a billionaire” — hughes, but then hughes wasn’t president and didn’t think he was on a mission from god. ‘pod happy the new ipod came monday. stepping up to it from the second generation ipod i had is amazing. most noticeable differences so far: i can now charge from the computer and play music (in the g ipod, it locks the interface and flashes “do not disconnect” any time it’s plugged in to a computer), the ui is faster or more responsive and is now customizable (a bit), it pauses playback when the external power supply turns off (especially useful in the car). things you have to believe to be a republican today my father forwarded this to me this morning: saddam was a good guy when reagan armed him, a bad guy when bush’s daddy made war on him, a good guy when cheney did business with him and a bad guy when bush needed a “we can’t find bin laden” diversion. trade with cuba is wrong because the country is communist, but trade with china and vietnam is vital to a spirit of international harmony. woody guthrie on copright copyfight is reporting on the infringement lawsuit threatening the creators of the presidential election parody animation that’s getting all the laughs. they’re quoting techdirt which apparently has a quote from guthrie himself: this song is copyrighted in u.s., under seal of copyright # , for a period of years, and anybody caught singin’ it without our permission, will be mighty good friends of ourn, cause we don’t give a dern. apple fusses over fuse fuse, a music tv network trying to compete with mtv by actually playing music videos done some bilboards in nyc that look a lot like apple’s silhouette ads, but with people pole dancing and masturbating and stuff. gizmodo came through and posted images of the ads so low brow people outside ny (like me) could be further corrupted by them (i’m not complaining here). let’s hear it for gizmodo. yeah! these aren’t cubes also at gizmodo: the volume macropod. they’re like cubicles, but cooler. they’re mobile, but useful. ad agency chiat-day made big news about giving up structured offices and such back around [cnn story & supervert.com story]. the point, of course, is to have people working out of cube farms because they’re cheaper, cheaper, cheaper. problem is, they feel cheap and they make employees feel unvalued. according to the cnn story: “employees who were […] looking forward to having a regular office the way they always thought it was going to be, and then they don’t have that. this land greg & evan spiridellis oever at jib jab have put together a damn funny flash movie about the presidential race. from the lyrics: … kerry: “you can’t say ‘nuclear,’ that really scares me. sometimes a brain can come in quite handy” … bush: “you’re a liberal sissy” kerry: “you’re a right wing nut job” bush: “you’re a pinko commie” kerry: “you’re dumb as a doorknob” life goes on… sandee called me from home friday to say she was having trouble playing music from our primary music server. every time she selected a song itunes complained that it couldn’t find the file. i had a plausible explanation at the time and didn’t think much of it, but sandee was really reporting something much more serious: the complete loss of all our music. over the past five years or so, we’d built a collection of about gigabytes of music, just under , files that could play / for over two months straight without repeating. mapparium, boston religious landmarks usually don’t interest me, but the mapparium really is a sight to see. …the mapparium, located within the christian science publishing society. a thirty-foot stained-glass globe room in lobby of the christian science publishing society gives one an ‘inside view’ of the world. standing on the thirty-foot glass bridge, which traverses the diameter of this large sphere, visitors can virtually be encompassed by the world. from pole to pole, you can journey through and explore the correct proportion and relationship of the earth”s land and water areas. you can take it with you: dvds on palm/clie junglemike has an interesting post on compressing video for palm playback at the src forums (n the cliesource forums): this guide explains in detail how you can prepare video to watch on you palm handheld. it [is usefull] for converting full-length . - hour movies to be stored on even a small mb sd-card with uperior quality. let me not fail, however, to mention that this seemingly harmless and legal use of technology puts users smack in the middle of the biggest land (property) war since napoleon invaded russia. fox and conservative pals out spreading more slander and libel welcome the flacks. i don’t get many comments on stories here at maisonbisson, so i was interested when i found a comment to my story about the outfoxed documentary just an hour after i’d posted it. here’s my theory, and it’s supported by stories in eric alterman’s what liberal media and al franken’s lies: conservative groups spend a huge amount of time identifying and attacking every liberal criticism. this mysterious matt (perhaps from ohio? outfoxed outfoxed: rupert murdoch’s war on journalism is out on dvd and vhs now. outfoxed examines how media empires, led by rupert murdoch’s fox news, have been running a “race to the bottom” in television news. this film provides an in-depth look at fox news and the dangers of ever-enlarging corporations taking control of the public’s right to know. i was hooked before i saw the outfoxed preview, but i’m definitely buying the dvd now. another military family against bush another military family against bush bumper stickers and other products available. another military family against bush value t-shirt another military family against bush long sleve t-shirt another military family against bush frisbee another military family against bush mug another military family against bush big mug another military family against bush messenger bag another military family against bush bumper sticker another military family against bush: all products why? my mother called in tears the other night after watching fahrenheit / . cheap food, cheap labor i’ve found myself in a number of conversations about food safety lately. eric schlosser’s fast food nation: the dark side of the all-american meal comes up regularly, but i keep wanting to mention bushwhacked: life in george w. bush’s america. why? because molly ivins and lou dubose did such great job explaining the political context in which the attrocities schlosser describes take place. “with republican control of the presidency and both houses of congress, you might want to consider becoming a vegetarian. old news, big story google just lead me to wage slave journal where i found an august story about american casualties in iraqi. it turns out fox news was comparing iraq to california and claiming the former was safer than the latter. fox can’t do math, but others can. should anybody ask, you should know that if californians were dying at the rate us soldiers in iraq are, the governator would be facing deaths per day. drm snuffs the constitution teleread brought me this story about a copy protected version of the us constitution that’s now selling on amazon. among the restrictions: it can only be printed twice a year. for those who don’t understand the irony already, the us constitution is in the public domain in so many ways it’s funny, yet a commercial publisher has created a version so locked up that it can’t be used and appreciated by all. fahrenheit / we expect fox news and the washington times to hate it, but the reaction from the left seems to prove the old adage that a liberal wouldn’t join his or her own side in an argument. my own arguments against it relate to how little new information it revealed. the audience at the show i saw laughed hysterically at the images of our government primping themselves for the camera and generally looking dim, but the facts of the film have been well reported in previous works. more japanese ice cream i got all excited about some unappealing japanese ice cream flavors when i found the story in mainichi daily news a while ago. i thought the lineup of fish, octopus, squid, ox tongue, sweet potato, fried eggplant, crab, corn, rice, wasabi, shrimp, eel, noodle, chicken wing, miso, and cactus flavored ice cream had everything pretty well covered, but now mdn has done it again. they’ve put up a new gallery of flavors of ice cream you’re unlikely to find in the us: more about clie th palmzone has a nice story about the th with a number of links to software, updates and more information. what everybody should appreciate is the link to the clie movie recorder. i thought i was so smart in an earlier story when i linked to the google query i used to find this file. that worked for about a month until my site landed at the top of the google index for that search. beef t-shirts rock beef t-shirts coming back: it was quite a while ago now that my cafe press shop was the top google result for beef t-shirt. worse, i haven’t linked to the shop from maisonbisson for a while either. so it was something of a surprise to discover that the products are still selling. yes, real people are buying these laughable t-shirts and other crap. they’ve been shipped to california, illinois, ohio, and oklahoma (as well as a few to me here in new hampshire). this is copyrighted? defense tech is reporting that the warner/electric/atlantic conglomerate of music labels gave up its defense in a copyright case against their artist wilco. it seems wilco sampled from irdial-disc’s compilation of recordings from mysterious radio stations that everybody expects to be related to espionage (and clearly emanate from government buildings and embassies). nobody argues that wilco sampled from a previously recorded work, the argument was weather irdial’s work was itself copyrightable. nauset beach panoramas more photos from maisonbisson taken monday morning, around : , before getting on the road to return to new hampshire. troy and karen were kind enough to invite me to the cape for the weekend, where i generally lazed about and did nothing. we did take in a double feature at the wellfleet drive-in (don’t miss the picture) and ate lots of ice cream, but the main point was being lazy. the letter not sent (re: lpfm, npr, nhpr, complaint) i was going through my files and found this unfinished letter to nhpr, my local national public radio affiliate, regarding the fcc’s proposed licensing of community-based low-power fm radio stations (lpfm). my point was (or it was going to be) that npr was afraid to compete against other non-profit stations. npr paints itself as an alternative to commercial radio (and it does a pretty good job most of the time), but it’s also a business. so npr joined with commercial broadcasters to kill lpfm before it could get off the ground. the fight included big broadcasting’s techs playing faked interference to scare lawmakers, but then they had to backtrack and call it “simulated” when somebody blew the whistle. sadly, it really didn’t matter what the played; they brought the money and the pols gave a bullet to lpfm. april , mr. sean t. gillery director of development new hampshire public radio north main street concord, nh - mr. gillery i recently received a letter from you regarding renewals to our nhpr membership and i wanted to take a moment to express to you my concerns over national public radio’s opposition to community-based low power fm radio. as you know, npr joined with the national association of broadcasters to lobby for legislation that has blocked the fcc from licensing lpfm stations. i believe that npr’s position on lpfm betrays the beliefs and philosophy that had once drawn me to public radio. can npr or nhpr be trusted to put its listeners’ needs first and its commercial interests last? not anymore. i am growing increasingly concerned that the recent and ongoing consolidation of the radio marketplace will further limit and degrade coverage of news, culture, and local events. npr has covered the consolidation and aired concerns about its negative effects: morning edition, “radio merger explosion” december , weekend all things considered, “black radio” august , all things considered, “radio consolidation” january , . all things considered, “radio merger” october , . unfortunately, coverage of the mergers ended when the fcc began considering lpfm in . since then, npr has run a handful of lpfm stories. each one focused on the potential for technical problems the lpfm law might create and the battle in washington to prevent the licensing of lpfm stations. but none of the coverage discussed the reasons why the fcc was proposing lpfm. none of this coverage put lpfm in the context of the earlier commercial radio consolidations. npr, of course, had to issue a very carefully crafted press release to explain their position. i can’t imagine what the response, if any, from nhpr would have been had i sent the letter. in the time that’s past, the republican controlled fcc has proposed measures that would lead to further market consolidation. ironically, an nhpr sponsored station is one of the few lpfm licenses granted by the fcc before the law ended further licensing. the station, which plays classical music in the concord area, went on the air just this year. comment spam first i was amused to see comments, then somewhat angered to discover they were spam, then amused again to find that comment spam etiquette requires that it be gratuitously patronizing. then i struggled to decide if i could delete the comments without feeling like i was censoring free speech. my solution (and it’s sort of evil) is to delete the comments (and the links they contained, i don’t want my (puny) google rank associated with them), but reprint them here: foiled troy has this image of a tin-foiled cubical on his blog. it comes from servers under the sun and is interesting enough. now that i’m checking his blog regularly, i’m sort of wishing he’d update more often (not that he doesn’t have a lot of interesting stuff in archive). . six months of books: the art of deception asmara bloody saturday in the soviet union the cockpit dangerous waters face to face with the bomb flight the iron triangle lies and the lying liars who tell them the new roadside america parting the desert reefer madness small things considered states of emergency an underground education wireless hacks audio books: bushwacked in a sunburned country re-reads: divided highways the race the real las vegas allconsuming.net allconsuming.net aggregates book mentions on the web, mostly in blogs. assuming bloggers can be trusted, the allconsuming stats can show a lot about what people are reading and talking about. david sedaris’ new book dress your family in corduroy and denim is ranking with mentions today and the day before (or, that’s what it was when i checked it last night). dan brown’s the davinci code consistently ranks near the top of each day, and both these books will get bosted a notch when allconsuming trolls me again today. all consuming is a website that visits recently updated weblogs every hour, checking them for links to books on amazon, barnes & noble, book sense, and other book sites. every book on this site has a list of all the weblogs that have mentioned it, and every weblog that has mentioned books in the past also has a page here listing which books it has mentioned. it’s more than a website, it’s also a set of web services by a guy who seems to know his way around xml, soap, rss and other incredibly useful acronyms. he even authored some chapters in amazon hacks from o’reilly press. anyway, call me a fan. faces richard coniff writes in the january smithsonian magazine about the work of uc san fran prof paul ekman and his study of faces. it carries pictures of a work by artists bill viola and his wife kira perov. yeah, sure, the face is capable of movements expressing , different expressions. yeah, bill’s work is interesting, but… i have two complaints. first, there’s all this talk that facial expressions are confusing. sun’s little marketing problem sun had to make changes. they’re (or were) getting their butts handed to them in the mid-range and entry level server markets, so those changes had to come fast. there was a time when the top of their low-end server lineup was the v with four ultrasparc iii cpus in a u rack enclosure. trouble is, it lists way over $ , . they can’t cut the price on it without bleeding money, and worse, they can’t scrap their old models because their inventory of pieces and parts is too much to swallow if they did. so what they did do is release a new line of low-end servers at half the price, but with some slightly different specs (and, i’d imagine, cheaper manufacturing processes) while preserving their older, more expensive servers in the line as the “better” machines. example: the v is similar to the v but has fewer dimm slots and sports ultrasparc iiii cpus. the usiiii doesn’t have the brains to do more than four-way multi-processing, but the designers used the chip real estate that freed up to put one mb of on-chip l cache. the usiii usually comes in machines with mb of external l cache, but it runs far slower than the cpu’s clock rate. eight mbs of cache is a lot, but arguments seem to favor a much faster one mb internal cache when performance is on the line. beyond the cache issue, the iiii sports a faster interconnect bus called jbus which further decreases the value of an off-chip l cache. access to main ram at almost the same speed as the l cache in previous cpus, and greater over-all throughput combined with the integrated l cache, how can sun argue that the iiii is slower than the old iii? but that’s exactly what sun is doing. their old manufacturing processes left them sitting on huge inventories for all manner of machines, and until they can clear those out, they’ll be sending some difficult marketing messages. the basics of it are like this: if you’re a regular sun customer and can afford it, then continue to buy the really expensive boxes. if you can’t afford it and might otherwise buy servers from our competitors, then take a look at these newer, cheaper models. and if you’ve never bought sun before, take a look at the speedy performance and low-cost of this v . how copyright law changed hip hop kembrew mcleod’s story about how copyright law changed hip hop in stay free! magazine is an interesting tale of how copyright kills culture. in the mid- to late s, hip-hop artists had a very small window of opportunity to run wild with the newly emerging sampling technologies before the record labels and lawyers started paying attention. no one took advantage of these technologies more effectively than public enemy, who put hundreds of sampled aural fragments into it takes a nation and stirred them up to create a new, radical sound that changed the way we hear music. jfk and mr. rogers look the same well, they sorta’ look the same. sorta. the real florida gators from an email from my dad: florida allows those who win permits to take three alligators. they sell the meat and hides , except the tails, which have the best cuts of alligator meat, and which they normally keep to feed their families. mal asked how the alligator meat is cooked; the lady said by cutting it into cubes and deep frying it. she said it tastes just like chicken. leadership who can complain about being compared favorably to ol’ jfk? (yes, in a really vain way, i was happy about it.) a co-worker was surprised to be matched with saddam hussein, but my boss was happy to be gandhi. numbskull, meanwhile, looks like abe. in another test, i was matched with indiana jones and raiders of the lost ark. what famous leader are you? what classic movie are you? extra stories a friend of a friend says his life is made up of places he can no longer go (or is no longer invited). sad, but somewhat true. he’s also a funny bastard. – – – sandee’s aunt had her th birthday not long ago. the aunt makes cakes on the side so it was no big thing when her daughters (who were planning the surprise birthday party for her) asked if she’d make a cake for some unknown group one of them was in. top google lamson library’s portal integration project tops google’s search hits for “library portal integration.” i’ve been crowing about it all over campus for a week now, and while you can argue about what real value it has, it’s still exciting. worldcat now available to world (via google) i’d heard that that oclc was opening up worldcat, their huge bibliographic database, to google. it seems to be online now. if you happen to google some very complete search terms for dan brown’s the da vinci code (look for the worldcatlibraries url), you’ll find a link to the public worldcat record. interesting, but i wonder where this will go. in fairness, this news is about six months old. jenny reported it in december. cliff’s piranha he’s named it officer angry, and it eats like a monster. it looks like a monster too, so that’s not so bad. videos of the fishy fellow eating are at cliff’s website: officer angry chases chow{# } and officer angry eats off a stick{# }. the second one is much better than the first. yes, i shot both, and just as an aside, they were taken with my clie th- (but edited with imovie). re: gasoline blackout day (wednesday, may , ) from jon link, who can also be seen at thenumbskull.com: i hate expensive gas as much as anyone but, this is a problem of our own design. we don’t need to stop buying oil for one day, we need to buy less oil in general. we love capitalism– supply and demand is it’s cornerstone… it can help or hurt us. it is just silly to think that one day without gas will do anything to supply and demand. jon link goes online with thenumbskull.com okay, his self portrait on my white board has nothing to do with his recent website launch, but…well…. thenumbskull.com more photos from maisonbisson japanese ice cream…novelties? fish, octopus, squid, ox tongue, sweet potato, fried eggplant, crab, corn, rice, wasabi, shrimp, eel, noodle, chicken wing, miso, and cactus. those may not sound like appetizing ice cream flavors, but it’s what they’ve got. the secret poetry of donald rumsfeld pieces of intelligence : the existential poetry of donald h. rumsfeld from amazon’s description: “until now, the poetry of secretary of defense donald rumsfeld has been hidden, ’embedded’ within comments made at press briefings and in interviews. his preferred medium is the spoken word, and his audience has been limited to hard-bitten reporters and hard-core watchers of c-span.” the unknown as we know, there are known knowns, dmcra vs. dmca get the word out. the fight is on to create sensible limits to the dmca. read arstechnica’s dmcra argument. copyfight, of course, is covering dmcra, and arguing for it. teleread is swinging for dmcra too. heck, they’ve even endorsed a congressional candidate based on his stand on fair-use. read those and act. tell your congressperson you support fair-use and the dmcra. now say it again with the eff: “i believe in fair-use. the twig it’s actually called the garlic clove, but for a variety of reasons, we just call it the twig. more photos from maisonbisson how do you sell a castle? when you call around for realtors to sell your ‘house,’ how do you tell them it’s a castle? i somehow found out about the martin castle in kentucky, but that lead to information about the dupont castle and that sites guide to over castles in the us. dupont reports there was a fire at martin castle just yesterday. the lexington herald-leader covered the fire. so, i guess the real question is “who do you call to insure a castle? in the window sarah left these as a gift for wendy in the window of her new toy. tesla’s history in colorado springs, colorado nikola tesla arrived in colorado springs on may , . he was met at the train by patent lawyer leonard curtis, and was taken by horse and carriage to the alta vista hotel, where he would reside while in colorado. tesla was greeted at the hotel by a group of reporters, one of whom asked him why he chose colorado for his operation. tesla replied, “i might as well tell you the truth, i have come here to carry on a series of exhaustive experiments in regard to wireless telegraphy — i come here for work. joe’s chickens and turkeys joe’s prized chickens and turkeys. in the brooders now. they’ll be in their coops by may. more photos from maisonbisson restaurant insider a link from wifi networking news points to qsr magazine, the trade mag for the quick service restaurant industry (think mcdonalds and taco bell). the connection here is that mcdonalds plans to offer wireless access in , locations. with mcdonald’s off the market, wifi hotspot operators are looking to hook the next big fish, and that’s why wifi networking news is linking to qsr’s top chains list. some technologists would speak about how we’re moving ever closer to the time when we have ubiquitous hi-speed wireless. music biz sales up uk markets first reported it, then australia’s record industry tried to suppress it, now us sales figures suggest the trend has spread here: record sales are up. yes, despite the riaa’s whining and lawsuits (and p p’s continued growth despite those lawsuits), record sales are up in the us. bbc news reports us record sales up % after a claimed four year slump. this story deserves more attention, but for now i’ll just have to link to my earlier stories about music industry wackiness: bringing digital video back to the living room you can burn dvds of your home movies (and you probably ought to, just for backups), but what if you want to make a movie library to match your computer-based music library? watching video on a computer is no more fun than listening to mp s on the computer’s tinny internal speaker. the solution may be one of a new generation of products that link the tv in the living room to the computer in the office. exploring the news newsmap displays current news in an explorable two dimensional space. headline sizes appear to be weighted based on the number of related stories. like plumbdesign’s visual thesaurus, it’s a truly new use of computer in the display of information. jacque’s cabaret bostonnoise.org says “jacques’ cabaret is boston’s oldest gay bar. the upstairs features live female impersonator shows five nights per week, including weekends. the downstairs basement is open only on friday and saturday, and hosts local bands.” jacque’s official website shows norell gardner & his cast of miss-leading ladies playing every friday and saturday upstairs. the raw bar, “a return to the old style of cabaret where artists entertain each other, for the pure art and enjoyment of it, creating a space for talented people who don’t have the opportunity to perform because their music or performance is more artistic than commercial,” was featured in the globe and plays downstairs at jacque’s underground on the second friday of every month. voip links vonage is starting to look like the ma-bell of voip. it’s not that there isn’t competition — there is, but they just don’t have the profile that vonage has. it looks like vonage has picked up the early adopters, now they have to start converting others. the market seems to have three fields: computer-to-computer only, software client with pots bridging, and hardware client with pots bridging. i don’t much care about the computer-to-computer systems, aim and ichat take care of that well enough. richard clarke’s insider tell-all tom maertens speaks on richard clarke’s insider story in a star tribune article dated sunday march . the troops who could have been used in afghanistan to capture osama bin laden and al-qaida were instead held back for the planned invasion of iraq. in contrast to the , men sent to iraq, only about , troops were sent to afghanistan, a force smaller than the new york city police. the result is that bin laden and his followers escaped across the border into pakistan. … clarke’s gutsy insider recounting of events related to / is an important public service. from my perspective, the bush administration has practiced the most cynical, opportunistic form of politics i witnessed in my years in government: hijacking legitimate american outrage and patriotism over / to conduct a pre-ordained war against saddam hussein. copyright war something doesn’t add up. aria, australia’s version of our riaa recently announced that sales continued to slide there this past year, while critics pointed out that they really had a record-breaking year with million album sales. thank arstechnica for the link. this matches news from the uk this past summer. so why is the industry lying? ignore for a moment the ironic story about the music industry using p p stats to improve their marketing and sell more records. why music biz loves p p jason shultz over at copyfight just posted this story about the mercury news’ story about how record labels use p p stats to boost sales. <a href="http://www.corante.com/copyfight/archives/ .html” title="record labels using “pirate” data to sell more cds">record labels using “pirate” data to sell more cds (posted by jason schultz) the merc has a great article on how the riaa bashes p p out of one side of their mouth while secretly using data from the networks to boost sales of their cds. political diagramming a graph from orgnet plots book purchasing patterns by politics.there’s not much middle ground there. “these political books are preaching to the converted. the extreme book titles on both sides reveal a focus on hate, instead of debate. in a year of presidential election, is this the new arms race?” could it be that our book readers are key opinion leaders in their communities? an opinion leader is someone whose influence spreads much further than their immediate circle of friends & family. what is the simputer? i just saw a pointer to the amida simputer, an indian designed and manufactured pda. the review at engadget sounds sort of down, but it comes from a company on a mission. it seems others are fed up with importing (and paying for) us technology, so they’re developing their own. take a look-see at the amida and mix that with a quick browse of the argosy eb , a chinese designed ebook reader. hmmm… boats it looks like a tug boat, but the great harbour could be a lot of fun. a magazine article talks about bareboat charters in the british virgin islands and the pleasures quietly exploring the coves and uninhabited areas on your own. nasa’s x flies nasa’s x scramjet test plane flew at speeds exceeding mach and altitudes of , feet today. i believe that’s a new air-breathing speed record. globalsecurity.org has a nice wite-up on it. american proprietary eponyms there i was googling “proprietary” for a story about misuse of the word when i came across this gem from r.krause: an eponym is a general term used to describe from what or whom something derived its name. therefore, a proprietary eponym could be considered a brand name (product or service mark) which has fallen into general use. yes, r. has a bunch of them listed, xerox, jell-o, velcro, and more. too bad it was last updated in . i wonder when “google” turned from brand name to verb. what does proprietary mean, anyway? googling “proprietary” results in lots of hits, but very few of them use the word in a positive sense. the webopedia computer dictionary offers: proprietary privately owned and controlled. in the computer industry, proprietary is the opposite of open. a proprietary design or technique is one that is owned by a company. it also implies that the company has not divulged specifications that would allow other companies to duplicate the product. thank chank the font designing folks at chank have a nice list of free fonts to pick from. sure, they’re not the fonts you use to design flyers for the church social or nursing home holiday dinner, but that’s sort of the point. isn’t it? anyway, they also link to nerfect where you’ll find other cool designey things. integrating library systems in campus portals information about lamson library’s portal integration at plymouth state university. i’ll expand this story later, but i want to put the link here now to get it in google’s index. update on pen twirling i did a story on the practice of pen twirling in japan a couple years ago. since then i have received an email from pierre etienne bastouil who is trying to organize a pen twirling competition in paris. despite the popularity of the sport in japan, he’s having some difficulty finding skilled pen twirlers in europe. so the call is out, interested pen twirlers should contact me and i will forward you on to pierre. schlossberg quote “the skill of writing is to create a context in which other people can think.” –edwin schlossberg squirm squirm little man far too often the mainstream press lets politicians get away with revising or misrepresenting their previous positions. far too often the press is complicit in their lies. not this time. hopefully quoticus will develop into a very useful historical truth machine to prevent revisionism. hopefully. ny times on netflix the new york times did a netflix story. the author, william grimes, seemed to like it, but… [my wife and i] each judge the other’s selections harshly. i scored a major victory with “mon oncle” by jacques tati, a director i once dismissed as tedious, annoying and far too french. he is now a god in our house. but i have had my back against the wall after “l’atalante,” a film i had never seen but knew to be, by expert consensus, a towering masterpiece. less than minutes after the opening credits rolled, the atmosphere in the living room grew frosty. i lost control of the mouse for a week. at least i had the foresight to sneak off and watch “russian ark” on my own. that’s the fun of netflix. along with savage recriminations, my home now resonates with high-toned animated discussion of directors, cinematographers and camera angles. once again i’m the moviegoer i was in college, when bergman, fellini and truffaut were in full stride, and adventure was in the air, and bright-eyed cinéastes could sit through a film like “el topo” and not demand their money back. it’s not available on netflix, alas, but the web site does propose an alternative, a compilation of “ed sullivan” shows featuring topo gigio. close enough. interesting enough, but netflix — and services yet to appear — are a sign of things to come: a world of entertainment shaped by the consumer, not by marketers. netflix executives say their edge over the competition is not their library but the way the library is presented to users, who are asked to rate the films they have seen. by sifting through the ratings, about million of them at present, and analyzing buying patterns, a company program called cinematch generates rental suggestions specific to each user. “lost in translation will outperform most $ million films for us, and that’s because of our ratings and recommendations,” said ted sarandos, the chief content officer for netflix. “monster will be huge for us, and that’s not because our subscribers are more sophisticated than the general moviegoing public, but because our merchandising system is much more specific.” it will be a world of what you want, and only what you want, as clearly marked by your previous purchases and selections. you’ll never be upset by products that you don’t want, even if you didn’t know you didn’t want them, nor will you have to tolerate contrary opinions or debate. dr. seuss was so political who would have figured old dr. seuss was so political? rick minear at ucsd has collected a number of the good doctor’s works as chief editorial cartoonist for the new york newspaper pm. “because of the fame of his children’s books (and because we often misunderstand these books) and because his political cartoons have remained largely unknown, we do not think of dr. seuss as a political cartoonist,” writes minear. turkeys on the lot! turkeys aren’t small birds. along the commute from home to work, they’re as common as pigeons in a city park, but it’s still odd to see a turkey in the parking lot (video link). the source video was taken with a sony clie peg-th and edited — just a bit — in imovie. wireless voip gphone is a bust for me, at least for now, but other solutions are available. ars technica pointed out an . b wireless voip phone from zyxel. then there’s the vocera voip communicator badge that everybody at dartmouth college uses. they were happy to show it off during the unleashed wireless conference they hosted last fall. [updated]: the voip market is heating up. vonage is set to offer a wireless phone soon to help compete against at&t’s new entry into the voip market. then there’s voicepulse and packet also making a play in the full-service residential/small business voip market. gamer’s delight: palm emulates gameboy, atari st and apple //e i saw a link for a palm-based gameboy emulator, then was stunned to read about an atari st emulator for palm. a quick google search later, and i found an apple //e emulator too!. it’s the old-timer in me, but i really enjoyed the games on those old systems. more info on the apple //e emulator for palm are at palm info center and freewarepalm. palmemu links up a number of emulators for palm. gphone doesn’t work on clie th i’ve given up on vli’s tech support for gphone, the voip software for palm. the download page said it was compatible with palm os .x devices, but was only tested on the palm tungsten c. i contacted support after trying it on my clie th , but fell into a loop where they kept recommending i try the same simple things and telling me that clies use non-standard audio hardware. i’d, in turn, tell them the results of those simple tests and explain that the th uses standard palm audio apis. hopefully they’ll find a solution, but i think the hangup with the gphone software is a network problem. recording video on clie peg th the cliesource forums are an excellent source of info. it turns out that installing the movierecorder.prc (version . ) from a ux onto the th allows it to record movies. the problem is getting that file…. isn’t google great? if that doesn’t work out for you, try searching at the palm user message board, where you might just find it. here’s the trick: you can’t just install the app via palm sync. scrabble aside from all the other online dictionaries, scrabble players may be interested in the following sites: hasbro’s word lists for tough times (including q without u, two letter words, x words, and more). wordplays.com’s tools for word games is a collection of web apps that would be handy to use (if it were legal to do so) during a game. mark has developed a number of word lists and other scrabble tools. wireless links the publicip zonecd is a bootable cd implementation of nocat’s nocatauth. nocatauth configuration help is available from amsternet and blyx. the leaf project intends to create linux-based firewall-in-a-box solution that has uses for wireless. linspot is a commercial hotspot-in-a-box software solution. nocat, less networks, portland community wireless, and newbury open.net are active community wireless operations. o’reilly wireless devcenter has loads of news. murphy’s junk on the list of places to visit next time i go out west: murphys surplus warehouse: located at n. johnson ave: el cajon, ca. (near san diego) fax , sq. ft. of military and industrial electronics, communications, and mis electronic equipment. sandee’s favorite bad songs s revivals may be played out and we’re not yet ready for s nostalgia. nonetheless, there are a number of songs of the time period that we’re a little ashamed to admit we love. without knowing why, and in no particular order, here they are: the humpty dance funky cold medina can’t touch this ice ice baby do me and poison hotstepper mama said knock you out goin back to cali mildly psychotic? eysenck’s test results extraversion ( %) moderately high which suggests you are talkative, optimistic, and sociable but possibly not very reflective. neuroticism ( %) moderately low which suggests you are relaxed, calm, secure, unemotional but possibly too unobservant of your feelings. psychoticism ( %) medium which suggests you are moderately offensive, uncooperative, and rebellious. take eysenck’s epq-r based personality test. clie memory stick, playing videos, and more… the lexar mb memory stick arrived. it sucks. it’s not really a mb stick, it’s x mb, and you have to flick a little switch to choose which mb you want to use at any moment. let me be more clear: you can only use mb at a time, and you have to eject the card and flip a switch to select the other mb. i don’t know if it’s returnable, but i think i’ll try. interesting site design just ran across - media.de. it’s a cool site. their flash design is top notch and i really like the metaphor. does it work? yes, in the limited context they’re using, it works well. best of all — or most disturbing, who knows — is the soundtrack. composed by yuko ohigashi, it’s haunting and mysterious. mac & palm/clie gps, maybe just learned of the rayming tripnav tn- gps receiver. it’s the type that has no display or ui and must connect to a computer (via usb) to be useful. it’s mac compatible and it appears there’s a slight variation (the tn- ) that works with sony clie palm compatible handhelds. the problem is, the company website is down now and i can’t get detailed information from the other sites. yes, google cache has info, but that’s more frustrating than helpful. of course, amazon doesn’t carry it, so i can’t view the reader reviews there. what i really want is a receiver that will work with both. but perhaps i’m just dreaming. then there’s also the question of what happened to sony’s clie gps cradle? finally, none of this would be an issue if i hadn’t also just read about tomtom gps navigation software for palm. return of dirigibles: delayed or dead? the s saw a resurgence of interest in dirigible airships. people believed their time had come again, but few are flying today. the cargolifter, a cargo airship designed for loads of metric tons (that’s over us tons), is in receivership, and little has been heard of the zeppelin nt. links and more info: story about the cargolifter (via google translations), and a cargolifter image gallery as well (also via google translations). going to see the goats went with will to see the mountain goats, will’s favorite band ever. plans included reliving the beef tatar at the korea garden. read my earlier story about it, but remember that it’s not actually called beef tatar. it’s “ok doi bi bim bab” on their menu. of course i wanted to take pictures of the beef tatar experience, but i also wanted to taste it again. it wasn’t the same as last time. what for wireless? planning for wireless deployments differs from wired network planning in many ways. unlike wired networks, the primary question isn’t bandwidth or reliability, but availability. wireless networking enables mobility — and mobile connectivity — in ways never before seen in the world of computers. just as movie theaters and television coexist despite their similarities, wired and wireless networks will coexist. each has it’s unique benefits and drawbacks. each is desirable for different purposes. bush’s fiscal felony matt miller’s npr commentary about the bush budget includes the following details: a deficit of billion means borrowing almost out of dollars in the budget. it includes billion in tax cuts that go mostly to the rich, but ignores the trillion dollar shortfall in social security and medicare that will start to come due in five years. bush plans to send an addendum to the budget to cover the growing costs of the us military presence in iraq and afghanistan after the november elections. flight planning software for mac i hope someday to have a need for flight planning software, so i’ll keep these urls around for a while: mac flight planner and flight math. vegas links now that the nevada test site historical foundation’s atomic testing museum. is open, you don’t have to wait for the doe’s occaisional tours of the test site to get your radiation fix. lawrence livermore national lab has a review of the new museum. we caught a show at the amargosa opera house (official site) in death valley, just a short drive west of vegas. the opera house deserves a story of its own and the views and scenery of death valley are just beautiful. shopping in new york, ny we watch queer eye for the straight guy a lot over here. it seems we can make time for about one hour of tv per week, and sandee’s decided we’ll spend it with the fab five. i’m sure the new york merchants featured in the show are expecting this, but we’ve started to keep a list of places we have to visit when we next go to the city. i’m posting it here for my use as much as anybody else’s. vegas! i might get around to telling the story later, but for now all i have is a couple movies and a few pictures. there’s a short video of the koi and gardens at the flamingo, an album of snapshots and nightlife, an album of pictures from the very unique amargosa hotel and opera house, and a short video of our short visit to crystal, nv. we saw zoomanity and a show at the amargosa opera house. getting to vegas i blame missouri. kansas city in particular. i’m sure there’s probably another airport like this somewhere, but i don’t know about it. kci, mo, is setup so that you have to exit and re-enter security areas just to change planes. then, if you need to use the bathroom or get something to eat, well then you have to go through security again then too. of all the airports to suffer a three hour delay in, kci might be the worst. dreaming of a sony clie peg th i’ve pre-ordered the just-released sony clie peg th and am anxiously awaiting its arrival. brighthand has a nice review that speaks (mostly) highly of the new palm os compatible handheld. high points were the integrated wifi, excellent battery life (compared to other wifi handhelds), large screen, integrated camera, and relatively good software bundle. low points were the email client, the low resolution of the integrated camera ( × ), and lack of bluetooth (which is included in the european and japanese versions). land of the loops was listening to land of the loops’ bundle of joy on the way home from work tonight. it somehow fit the mood and i found myself really enjoying it. yes, it’s loop/sample-based, but the results are anything bet techno or hip-hop. originally released in (i think?), it holds its tune seven years later. . . things to remember while doing upgrades on mission critical sun equipment…. a: sending stop-a with non-sun keyboards or over a telnet connection with a terminal server, the terminal is hardcoded to a “cli” interface which, in turn, telnets to the console port on the destination host. the point is to get the *telnet* to generate a break, which can be done by: press ctrl-] (or whatever is the telnet escape sequence) at the telnet prompt, enter “send break” newbury open net just saw a link to newbury open net, a community wireless project in boston. newbury open net describes itself: newburyopen.net is a network which provides high-speed internet services, in the form of free wireless and for-pay workstations, to boston’s residents, workers, and travelers. … we believe that high-speed internet must become like a public utility: cheap, simple to access, easy to find, and available to everyone, no matter their location or social status. macdevcenter on home automation first, i found this story at macdevcenter rather interesting: home automation with mac os x, part by alan graham — having more control over how your home operates isn’t just a geek fantasy. you can lower energy costs, improve security, and enhance the overall ambiance of your humble abode. alan graham shows you how to leverage your mac os x computer and get started. home automation is, of course, something i’ve wanted to play with ever since i heard about it. sure, itunes visuals are great, but what about programming all the lights in your house to work like a huge color organ to pulse with the music? but i was also amused by the o’reilly/macdevcenter website. along with the usual print and [email][ ] buttons they had a [blog this][ ] button. while they clearly wanted visitors to see the website as something more substantial than a weblog, they also wanted to cash in on the blogging public’s ability to create buzz and swing google rankings. we like the moon, biscuits, and more flash animation the folks at rathergood.com have no end of flash animations to entertain and delight. may i suggest starting off with moon song, and biscuits? along those lines, i also found (the far too obviously named) flash archive with even more great goodies. yes, you’ve seen some of these before, but there are some new ones there too. and, of course, regular laughs can be had at homestarrunner.com, where strong bad’s email (updated each monday, usually) will likely make you a repeat visitor. zygo: the last energy drink cola wars are one thing, but “altbev” sure has come a long way since soft drink makers identified the market segment in the s. coke’s fruitopia was among the entries from the majors, but, as usual, it’s the independents that have lead the way. water remains the leading altbev, but energy and “health” drinks are squeezing the market. just as coke and pepsi were developing their bottled water brands to catch up with poland spring (owned by nestle, by the way), red bull appeared and turned things upside down. useful dohickeys why can’t i find the sumajin smartwrap, a small cable management device that looks perfect for headphones and other small cables, locally? smartwrap, winner of id magazine’s design distinction award, is a cord manager for headphone cables designed and developed by sumajin, an industrial design firm in singapore. you snap the cord into place at one of two places then wrap and snap into place again. smartwrap comes in seven colors and are produced in limited quantities. /etc/hosts in macos x . i’ve run into a situation things would work better with a static host mapping, but my first thought/fear was that macos x’s netinfo would get in my way. google turned up some old info on reconfiguring netinfo, as well as a slightly more current netinfo tip. but as it turns out, panther is all setup to read your /etc/hosts entries and use those before going to dns or netinfo. so there you go. what is ibiblio? if -year-olds were old enough to remember bush sr., they’d think this bush monologue was the funniest thing all day. so, in the interest of educating and entertaining those -year-olds, let me explain that the current president bush is the oldest son of a previous president bush. bush sr. was elected in , his term of presidency included huge job losses and recession, and he got us entangled in a war in iraq and many other places. deep thoughts; timewasters here’s a graph to get you thinking about politics: job growth per president. who knows if the numbers are real, but it jives with my memory of the past years. this dark and slightly objectionable cartoon of life features a good soundtrack and really cool styling. finally, everybody likes latin translations of old rap songs. right? “magnae clunes mihi placent, nec possum de hac re mentiri.” peer-to-peer, dmca, riaa, lawsuits after six months of riaa lawsuits, you’d think this would be old news, but…. it’s been a while since i’ve reported on the music industry’s attempts to control online music distribution, but ars technica has been following that and the larger issues all along. the story took a turn in december when a three judge panel ruled that the riaa’s subpoenas were illegal. that was a win for the isps that had brought the appeal against the riaa and have now ceased cooperation with the music industry. tivo getting close to home. too close. the folks at ars technica are asking question that i first started wondering about during the patriot’s superbowl win. after the game, the tivo folks released an announcement that britney spears’ pepsi commercial was the most-rewatched ad of the game. their claim was apparently based on stats from the tivos in people living rooms. we’re all familiar with nielson tv ratings, but those viewers know their habits are being recorded. mit tech review’s ten technologies that refuse to die the folks at ars also pointed out an interesting story by the mit tech review. it’s all about things that were expected to have been passed by, but weren’t. it sort of puts us in our place. microsoft, in its biggest act of irony ever, issues security education posters microsoft corp, the software company responsible for producing some of the most notoriously (and dangerously) insecure software ever has issued a collection of posters aimed at, start your irony engine, computer security education. “educate your students, faculty, and staff on the simple steps they can take to protect their pcs,” says the microsoft website offering the posters. site updated um, not many people noticed, but this site was offline for a few months because the hosting company i was using shut down operations. well, i’m back, mostly. i’ve redesigned things (having stolen the design from another site of mine), but there are still a number of things missing. theoretically i still have a backup of the comments and members and stuff, but i may not bother looking. the redstone brewery info is in here, but the categories list is gone. how to get off an rbl it sucks to get on a email blackhole list. click “more” to find out how we got psu off att.net’s proprietary rbl. entertainment value first, take a look at < bushin seconds.org > . it will do more to make you mad than entertain you, but take a look and channel that anger into something meaningful. now that that’s over, take a look at < ebaumsworld.com > and waste the rest of the day laughing. there’s no shortage of video, cartoons, and other junk. enjoy it all. here are a couple links to get you started: super truck, paranoid, something else funny, and yet another thing. the unwired world is growing first, look at some numbers: “‘last year under percent of the laptops have wi-fi built in, this year it’s percent.’ says brain grimm, communications director for the wi-fi alliance” now consider that the quote appeared in a story in aaaworld (yes, the american auto association). their demographic is generally older and non-technical, so either their demographic is changing or “non-technical” is being redefined. i’m going to bet that the water is rising and, just as the world now accepts email, it now seems to expect some understanding of networking. hmmm. [update] and now the minneapolis federal reserve bank is reporting on growing wifi use in the mid-west! oh my. why superbowl ads matter last saturday was the th anniversary of the macintosh. apple announced the macintosh to about million households in a second ad during the superbowl. the ad, which has been lauded as one of the best ads ever and created “event marketing,” rocks. it was this theory of event marketing that lead advertisers to create ever larger, ever more expensive ad spots. and that’s when the ads during the superbowl became the the main event for some viewers. okay, now i want one there are two things you need to know about the the international streamlined tatra site: it’s cool, and they’re cool. i happen to love art deco advertising, and it seems tatra has some of the best. of course, i wouldn’t know anything about tatra (it’s a car company, or it was, they now only make trucks) except i stumbled across this story elsewhere. warren republicans vote democrat former vermont governor howard dean carried the polls in warren this primary night. the numbers for the rest of the state are still being counted, but what’s more impressive to me is the number of voters who went to the polls and the number of registered republicans who wrote in democrats on their ballots. twenty three out of republican ballots cast in this very conservative northern new hampshire town had democrats written in for president. czech it out! bad headline, yes, but what this guy has done with his car is pretty cool. antarctica in my name it’s good to know that there’s an antarctic outpost in my namesake. good ol’ casey station even has a webcam. [update:] here’s an interesting sattelite image of the area, found at this remote sensing project website. ethel’s holiday fashion nothing says holidays like leopard print. more photos from maisonbisson how to have fun like i just did start with approx cup of bacon grease collected over time just like jon’s mom said to do. pour grease into small disposable aluminum loaf pan. insert pan with grease into burning wood stove. wait. watch. wait. watch as oil ignites with a whooosh that’s vaguely reminiscent of a chimney fire. no, that woooosh is exactly how you remember that chimney fire. close stove air intakes and continue to watch fire. more complaining and whining the lousy red cross can’t get its act together well enough to schedule blood drives in plymouth (where i work each day) well enough so regular donors can go to all of them. the red cross knows that something like % of their blood comes from regular donors who make it a point to donate at every opportunity (and how many of us can there be in plymouth?). yet, they schedule a blood drive today, fewer than days since their last blood drive. o’reilly’s wireless hacks the question here is between . x authentication and web-based, captive portal authentication. the former has high client requirements, the later seems too simple. rob flickenger’s wireless hacks has fired me up for captive portals. an excerpt, dispelling the myth of wireless security, makes clear the need for application layer security, an argument i’d say applies to wired and wireless network alike. point: wireless is exposing holes that have existed in our network security all along, but patching those holes will secure everything, including wireless without spending loads of money on expensive aps and proprietary clients. wireless vulnerabilities related to my review of wireless security landscape is this review of threats to wireless security. passive sniffing “the same information in a probe response frame is available in the beacon frames that every . network is required to transmit (even closed networks). so, we just listen for these frames in monitor mode instead.” extreme tech’s guide to exploiting and protecting wifi networks “airsnort can determine the wep key in seconds…” the wireless security landscape the view from the trenches fall below is an email i sent to maclabmanagers mail list in late september . our discussions of wireless security had just begun at that time. the wireless landscape has changed a lot since then, but the responses have information that remains valid and useful to us today. howdy, we’re using wireless in many locations here, but somebody just got scared about security. until now we haven’t been using wep, nor have we cloaking the network name for wireless base stations that serve mobile classrooms on campus. wired mag’s commandments of programming wired magazine has an interesting article on “extreme programming.” supposedly, the solo programmer pulling all-nighters on excessive caffein is out. in are hour work weeks, group coordination, and two people per computer. but what about productivity cry the managers. according to the article, coders do more, do it faster, and do it with fewer bugs this way. summary page for music industry wackiness i’ve posted a number of stories and links related to the music industry and p p and such. here’s a short summary of them. first was a story about how music swappers actually buy more music. then came a story about the decline of the album format, and why it’s a good thing for listeners. i followed that up with something about copying is theft, and other legal myths. and just now i posted a story about the real reasons for the decline in the music biz. perfect for the church social hey, so what about the local sports team and their player that’s excelling with that thing that he does? some people like to argue so much they run out of material. or, maybe it’s like what rob gordon says in high fidelity: “it’s not what you’re like, it’s what you like.” so maybe arguments erupt as we try to establish and defend our identity (evidence: teenagers). if true, and our identity is made up of the pop-culture elements that we consume, then what are the key traits we must evaluate? street lights…and other things that don’t work the way they should it’s probably due to my color blindness, but i have the darndest time seeing streetlights (the red/yellow/green things at controlled intersections) at night. i’ve had to explain it a million times, but nobody seems to understand. finally i’ve discovered a sympathetic friend, sort of. michael darnell writes about his complaints with street lights and other things that don’t work well or aren’t designed well. time wasters i found myself waiting. a cd quietly burned in the combo-drive, a computer slowly reboot after a system update, and a large file was drifting across the ether[net] between my laptop and sever. clearly this was the time to surf over to ilovebacon.com and waste some time. i was in luck right away. ask snoop isn’t quite as funny as old unix jive, but it’s good for some quick laughs. music this, music that continuing the recent music and copyright theme…. it turns out that i wasn’t the only one who thought the buymusic.com ads looked a little familiar. rob walker wrote about the new apple clones for slate.com. “…i kept re-watching the buymusic ads to try and figure out what i was missing. is there a hidden critique here? a satire? not really. they’re just knockoffs. it’s as if, by borrowing the look and feel of apple’s ads, buymusic is explicitly interested in underscoring that its service is a copycat. website spotlight i just added arstechnica to the list of websites i check daily. i’ve been reading technical articles there for years, but two articles today clinched it: “the social complexities of the f-word” and “your cheating heart’s been clickin’ her buttons. both are well worth reading for anybody who cares about the social aspects of technology. well, the first one doesn’t really have anything to do with technology, it’s just funny. copying is theft – and other legal myths music has been an issue for me lately. what with my previous stories about the “decline of the album format” (and why i think it’s a good thing) and how music swappers apparently buy more music, you’d think i’d gotten the matter out of my system. no. copying is theft – and other legal myths is an article that everybody who’s ever heard of mp s should read. no matter what you’ve come to believe (or how much the riaa pays you), the title is real. usb hacking so i’d like to get this old usb video capture device working in os x, but the vendor has quit the business and no os x drivers are around for it. a little searching on the web netted the following how-to on making one vendor’s usb device drivers work with another vendor’s products. the details relate to usb wifi adapters, but we can generalize. with the tips in that story in mind, we can face down the next question: are there any drivers that might be made to work with my usb device? whiney sell-outs charles haddad writes in business week online about musicians making a stand for the “integrity of the album format.” fortunately, he gets it right: this isn’t about artists looking after their art, this is about the end of a business strategy where a few good tracks are mingled with a pile of chaff and called an ‘album.’ what’s really important here is that you can buy what you want, rather than just what labels and the bands have decided you should have. no longer do you have take the fat with the meat — and pay $ or more for a cd that has only three songs you like. …this doesn’t necessarily mean the death of album rock, just bad album rock. a package of great songs that work together will still sell. just look at the evergreen appeal of the who’s tommy or miles davis’ kind of blue . the labels may be forced to change. if filler no longer sells, will the music industry continue to compel bands to produce it? maybe, just maybe, bands and labels will start improving the overall quality of pop music. music labels have heads up asses a story on bbc news (file swappers ‘buy more music’) reports on a study that claims those who download music using p p services (old napster, gnutella, etc) actually buy more music. it should make sense to anybody with a hair of marketing experience: try before you buy. yummy shit karen pointed out an article about scary-but-common food ingredients at fortune.com stupid os x server hint os x server is great, but it doesn’t respond well when you change its ip number. the resulting fiasco will make you think working a fast food job is worth it. here are some links that won’t make it easier but will at least give you a bootable machine: a little how-to support discussion more discussion even more discussion update august , : apple has finally done something, just a little something, to address this problem. dvrs are cool i don’t watch much tv and i don’t own a tivo, but i love the idea. so i’m glad to read about open source folks building their own dvrs. apollo archive the apollo archive boasts a wealth of content covering the moon landing. good stuff. google-watch google has been raved about since it first appeared on the search engine scene four years ago. now that it’s trounced all the other, however, people are getting concerned about the effects of the monopoly. google-watch is leading the charge. their claim? they say that google’s pagerank means only that the rich get richer, and they’re concerned about close ties between google and government snoops. hmmm. you are being lied to i found a collection of three books by the disinformation company on a shelf in city lights. i’d already picked out my book (toothpicks & logos) when i saw abuse your illusions, everything you know is wrong, and you are being lied to lined up and beckoning to me. i’ll have to take another look at them, but at least the publisher has an interesting story. good liberal rabble rousing it’s a pleasure to read the many pages of molotov cocktail for the soul. iug : library portal integration elaine allard and i will be presenting on library portal integration at the iug in san jose, ca. two sessions have been scheduled for sunday, april th: am and : pm. our description, in the program guide: like many colleges, plymouth state college is working to consolidate its online resources inside a portal. within this single point of service students can register for classes and check their grades, faculty can review their rosters and post grades, and staff can review benefits and vacation time. tinkerer’s joy while reading up on the slimp network mp player i came across some mention of dallas semiconductor and their line of wonderfully hackable tini ics. these little things have ethernet interfaces, java runtime engines, and webservers built-in, and are ideal for making non-networked devices internet ready. as if we don’t now have enough internet connected light switches and soda machines. a nice overview of tini is available. ohh, film music pornorchestra: the pornorchestra is an attempt to radically reinterpret the soundtrack to pornographic film. this complicated genre has taken its share of scorn: from adult film producers who refuse to pay it any mind to legions of consumers who instinctively snap the sound off after pressing play. performing live improvised and composed scores to pornographic film, the pornorchestra invigorates the mysterious experience of the voyeur-cum-auditeur. the equivalent of a circus band with its collective eye on the trapeze artist: the pornorchestra teases out the thrill, amplifying the collective gasp at pornographic triumph — and tragedy — using the most eclectic and creative musical minds working in the bay area today. the promise of wireless wired has a story about the effect of wireless on agriculture, theme parks, health care, and conferences. so speaketh o’reilly’s rael dornfest about a recent conference with ubiquitous wifi access: “people weren’t disappearing back to their rooms to check email between sessions. they’d just sit down in one of the common areas and log on. because everyone was gathering in the same place, there was a lot more spontaneous discussion. also, the sessions themselves became more interactive.” cool fonts font diner offers some darn cool fonts. go visit their site for freebies too. a farmer’s job i don’t know who gets the worse end of this stick, but it’s really sad that chemists can’t tell the difference between banned nerve agents and agricultural pesticides. conflict management how to talk down your adversary: “there is no reproach between me and you except the stabbing of kidneys and the chopping of heads.” damn cool site plumb design’s visual thesaurus may be the coolest thing ever. psychoanalysis word of the day ego dystonic elvis vs. nixon a friend forwarded a link that reveals the following story (as quoted from the website): on december , , elvis presley paid a visit to president richard m. nixon at the white house in washington, d.c. the meeting was initiated by presley, who wrote nixon a six-page letter requesting a visit with the president and suggesting that he be made a “federal agent-at-large” in the bureau of narcotics and dangerous drugs. tom bihn bags the story is that tom bihn designs and makes bags for laptops and other stuff. or, at least that’s what tom says at his site. tom bihn has been designing and making bags for well over twenty years. daypacks he made when he was years old are still in use, and in santa cruz, california, where tom grew up, his laptop cases and book bags are almost legendary. conferencing in dc i’m in washington d.c. at the computers in libraries conference. it’s a good lineup of presenters and good programs, but i’m sad to know that i’ll be missing a peace rally this saturday. where to eat and drink: old dominion brewery is in virginia not far from dc. it’s in an industrial park and you’ll doubt that you’re in the right place, but the food and local brews are good. short quiz for discussion: world history mid-term exam this test consists of one ( ) multiple-choice question (so you better get it right!) here’s a list of the countries that the u.s. has bombed since the end of world war ii, compiled by historian william blum: china: - korea: - china: - guatemala: indonesia: cuba: - guatemala: congo: peru: laos: - vietnam: - cambodia: - guatemala: - bi bam bab in cambridge the korea garden is on pearl street somewhere behind the middle east in cambridge, mass. it’s the sort of place that attracts local asians and very few white boys (like me). so it’s hard to say what they must have thought when cliff and i walked in one night this winter. an argument broke out in the kitchen as the waitress presented our order. we joked and smiled among ourselves about it, but my smile fell as my dinner was delivered. counterscript telemarketers may be people too, but this script will ease the pain of their next call. take a look at egbg’s counterscript. and if you’re looking for serious anti-telemarketing resources, look at junkbusters’s resources. warren redstone brochure available! i found a brochure about the warren redstone and present it here for your enjoyment in pdf form. it features the story of how and why it came to warren, written by ted asselin, the man who brought it here. it also has information about the progress of the rocketry in the s. the brochure was originally in tri-fold form, but is presented here as a two page pdf file. enjoy. yum! email received today: nothing starts a monday off like kippered seafood snacks, deviled ham, with a side of spam, potted meat food product, followed by vienna sausage, all washed down with some icey cold clam juice. now i am ready to face the day. yours meatily, dr. meaty mcmeat meatofski meatovich hamkowsky-beafeau porkson justin and the sled dogs the season for running sled dogs is almost at its end. here’s a short video of justin racing for the finish of one of his last races of . click the link to watch justin’s big finish. ashcroft’s biggest boob in the way emails thread their way from one person to another i came across the text of a speech about antics in the us justice department. it was titled “an open letter to john ashcroft” and came with this preface: the following is a letter read by claire braz-valentine, author at this year’s in celebration of the muse, cabrillo college. it is worth knowing that the author is a woman of + years, conservatively dressed and obviously quite talented. marketing artifacts each of us deals with a lot of stuff unique to our jobs or life context, stuff that outsiders never see. now and then it’s fun to see that other stuff. here’s some: silly marketing materials. more commercialism! people have asked about this whole t-shirt thing. click the banner to see how it works. sign up! update: i just found a similar service for video distribution. you might want to check out customflix.com. state of the union? it’s not real, but it may be more accurate. watch the state of the union speech here. thanks to my sister for pointing me to this. [update]: the link above may be down, the speech is mirrored here. where have all the updates gone, long time passing? since this website is such an important and valued news source for so many people, i’ve received many dire complaints about the scarcity of updates over the past month. here’s the story: january is a busy, busy month at work. students are gone, computers must be updated. work also includes many large changes to the lamson library website, and more updates are due shortly. daytime work is one thing, but i’ve also been pursuing my side business more actively. common sense revisited? this may not be news to somebody who hadn’t swallowed the school approved version of american history whole, but there are a few important things to note: before , colonists paid less in taxes than britons in their homeland did. while the colonies were not represented in parliament, neither were big british cities such as liverpool or manchester. meanwhile the colonists enjoyed a free press, voted for local representation, ate better, lived in larger houses, and were generally better educated than their british cousins (the literacy rate in massachusetts was more than twice that in britain). bryson on language speaking on language patterns around the time of the american civil war, bill bryson states: …no nineteenth century journalist with any self-respect would ever write that a house had burned down, but must instead say that a great conflagration consumed the edifice.’ –bill bryson quoting (in part) kenneth cmiel’s democratic eloquence in, made in america, an informal history of the english language in the united states. mitnick off parole he’ll be on parole of a long time, and he’s facing a number of additional restrictions, but kevin mitnick is finally free! maison bisson’s winter drink the holidays are long since past, here’s a drink to carry you through ’till spring. rusty nail parts scotch part drambuie serve over ice in an old fashioned glass. please enjoy it responsibly. the light i’ve found it. it’s here! newswatch: foreign secrets: bad; domestic secrets: good. the news of the day is government secrecy. npr’s all things considered ran two stories about the matter today. one story about general secrecy, and another story about admiral poindexter (formerly of the iran-contra scandal). previously, npr ran a capsule biography about henry kissinger. of note is the discussion about kissinger’s disbelief in open government. that story was followed by analysis by daniel schorr which may suggest why kissinger was chosen to head up the independent panel to investigate the attacks of september th, then another story about his resignation from that panel. trickle down voodoo it seems clear that trickle down economics is back with new tax breaks for the rich, new spending on the security-industrial complex, and our first dip into deficit spending in years. while some call it it voodo economics, faith in trickle down economics seems to be based upon the oft repeated line that anytime you put money into the economy, it benefits everybody. when pressed about rising executive salaries, believers embrace that too as eventually benefitting the economy. i found myself in an argument about these matters recently, and had to take a moment to assemble my thoughts about it. new books i used to read magazines — i find it difficult to commit to things and magazines let me off easy, but i’ve been feeling unfulfilled by magazines lately (those who know me might also point out that i was somehow able to commit to marriage, and i’m still married over four years now). so i’ve been reading books left and right. now, after the holidays, i’ve got a pile more. bowling for columbine highlights meaningless ideology there’s a small battle being fought in the comments of my previous entry about bowling for columbine. it should be no surprise that gun rights are a very serious matter for many people. nonetheless, guns are involved in a huge number of homicides in the us each year. and so those who would seek to prevent or limit those murders find themselves battling gun owners who would rather ignore them. road rage while the state argues with environmentalists about needed environmental abatement in the project to widen i , we should all take a moment to consider the social implications of the plan. wider roads will inevitably lead more people to commute greater distances to work each day. whatever the causes of road rage, we can all acknowledge that time spent in the car is not quality time. incidents of road rage are at their highest in areas where commuting times are the greatest — think of la and washington dc. ipod links: ipod ipodhacks.com ipods around the world newtons around the world ipoding.com podnews wired news’ cult of mac wired ipod hacking story water world water is the primary ingredient in every liquid soap, body wash, shampoo, and conditioner product in my bathroom. some even boast “purified water.” ebn videos online ebn, emergency broadcast network, was a band of media jammers from the days of the gulf war (the one back in ). they disappeared from the scene a few years ago, but you can find some of their old videos over at guerrillanews.com. and, as long as we’re talking about media jamming, i should throw this book at you: jamming the media by gareth branwyn. edit: the links here go nowhere, but a few videos are in youtube: movie: bowling for columbine a friend of mine recently pointed out what i should have seen for myself: conservatives won’t change. so, while bowling for columbine is great entertainment for open-minded folks, it won’t make an impact on the folks who most need to see it. if you’re lucky you may still be able to catch this film in theaters, but everybody should take a moment to view this clip of the cartoon that appears in the film: a brief history. turn of century bridge jumpers had wide field of opportunity the opening of a new bridge in the early th century attracted a lot of attention. it was at that time that materials and engineering skill finally allowed cities to bridge rivers that had formerly required a ferry to cross. new york, with its many islands and rivers, was exceptional in this regard. new yorkers eagerly followed news of the design and construction of bridges. bridge openings where celebrated with days of events and fireworks attended by presidents and luminaries. wnyc’s _on the media_ does sex show. on the media’s recent show on november th and a piece in all things considered explored the relationship between technology and pornography. this is familiar territory for some—wired magazine reports on it regularly… click the links above and listen for yourself. booklist: nickel and dimed when i first found barbara ehrenreich’s nickel and dimed while waiting for someone or something, i picked it up and started reading in the middle. i found myself immediately taken in to her story and her writing, and was more than a little remise to give it up. not many non-fiction books about social issues are call page-turners. but this is one. ehrenreich attempts three low-wage jobs in three cities for a month each, trying to find housing and food within the budget allowed by such work. apple and the future of intelectual property macintouch pointed me to a blog entry at plasticbag.org related to the role of computers in the war over digital intellectual property rights. the author believes apple has already staked out its territory in this matter. after a series of examples, he explains the following: the reasons for all this, of course, are that – for good or ill – at the moment copyrighted material and intellectual property are endangered and cornered beasts anyway. marching toward privatization republicans and business leaders have been pushing privatization (and deregulation) for decades. now, the results of this effort are becoming clear. even as the bush administration announces plans to privatize nearly a million federal jobs, reports of the costs and failures of such privatization roll in. mother jones reports this month on the growth in privatization of municipal water systems. the result in cities like atlanta has been water boiling alerts do to dangerous bacteria levels, and poor service do to a workforce slashed by cost cutting. activist art art is not, or does not have to be, cheery. it turns out that people become troubled and conflicted when they see pictures of the hungry and the homeless just weeks before thanksgiving and the start of the holidays. the nashua telegraph takes up the story here: a new exhibit in the town hall gallery, designed to raise awareness of and funds for the open cupboard food pantry, has gathered some complaints from residents and prompted the board of selectmen to suggest that it be removed. the exhibit consists of a selection of black- and-white photographs taken by resident preston heller of urban street scenes and various people he describes as being at the bottom of the social ladder.’ update nov- - : nhpr reported on this story today, and linked to the photographer’s online gallery. in mother jones: a confederacy of cronies readers can trust mother jones to shine liberal light on conservatives. in a confederacy of cronies george packer tells us how difficult it can be to play america’s ceo, where regular americans really stand. great movie criticism it’s hard to explain why or how i just stumbled across a year old roger ebert movie review, so i won’t. i will try to explain why i found the review so real. i actually saw this movie, and it’s really every bit as bad as the review suggests. ebert questions how movies stereotype baddies. ebert doesn’t get too controversial, so this is as much as we’ll get out of him. mac geeks have more fun thanks to the folks at macos x hints, i’ve been pointed to the most useless thing ever: a tool that allows you to view any quicktime file in your terminal window as ascii text. yes, it is absolutely useless. understanding marijuana liane hansen of npr’s weekend edition sunday interviewed dr. mitch earleywine about his recent book, understanding marijuana: a new look at the scientific evidence this weekend. earleywine has the credentials to look at this seriously and be taken seriously. but he probably won’t be. there’s no shortage of books on this subject, and the drug war marches on. but as long as we’re slinging books, let me throw michael pollan’s botany of desire at you. framethief animation toolbox framethief is a toolbox for capturing hand-drawn frames and assembling them as animation. image sources can include video camera — the old standby, and digital still camera — a new twist that allows animators to work in hdtv resolutions. one component, framesplicer, can be used to turn any quicktime compatible video file into a dv stream that can be used in imovie. political-economic conspiracy? marektplace comentator james galbraith explains in tuesday’s show how this will be a longer and deeper recession than previously thought and many economic indicators may have been manipulated to hide the recession’s true nature prior to november . galbraith reminds us that things were rather similar years ago, when unemployment rose over % and democrats took control of congress from a far-right conservative president. history did not repeat itself, yet. mile markers matt frondorf’s american mile markers takes us on a tour from new york to san francisco, one photo per mile. it’s a fine concept — inspiring, really, but the pictures are quite a mishmash. matt calls his mile marker project statistical photography.’ a lot of photography tends to be anecdotal and heavily edited,’ he says. and it doesn’t present what is really there — every picture from beginning to end.’ yahoo! pen twirling! pen twirling takes great skill that can be achieved only by hard practice and determination. though promoted by stars as famous as miss iyo matsumoto, it can be difficult to find pen twirling masters capable of teaching the sport. hideaki kondoh, who’s interest in pen twirling was sparked by a tv appearance by iyo matsumoto, struggled to learn: “i couldn’t help admiring her excellent performance, but i didn’t think i would try to spin a pen myself. hops n’ things it was a few years ago now that jon at hops n’ things put us on track to brew our first big batch of cider. knowledge comes from books, but a guy like jon can give you know-how. today he introduced us to distillers’ active dry yeast, or dady. our last batch of cider went to proof with epernay champaign yeast, dady might get us to proof! more importantly, he was kind enough to help us fix a co leak in our keg system — and he stayed open late to get it done. redstonebrewery.com online! after months of lost time, redstonebrewery.com is finally online. there’s not much there, but you wait baby. you just wait and see. or. um. well, we’ll see what happens there next. raspberry jelly i usually try to keep this blog above trivial things like this, but not today. i enjoy penut butter and jelly sandwiches, but usually with raspberry preserves — the stuff with fruit chunks a seeds in it. so i was rather surprised when i found i’d accidentally bought hannaford brand red raspberry jelly. it mostly tastes like raspberry, but it’s been pureed smooth like jello. i tried it, the product doesn’t spread well and the texture is all wrong. modern drunkard magazine this little ‘zine just scored distribution with borders book stores. but if you can’t find it there, take a look at modern drunkard magazine online. take a look at their wino wisdom section where you’ll find gems like “the secret of being a good drunk is not to try to hard. to me, it just comes naturally. you might even say it’s effortless.” and “i don’t smoke filtered cigarettes for the same reason i don’t drink whiskey through a bar rag. megahertz gap? so the project to crack a -bit rc encryption key is over. some computer in japan figured it out in july, but everybody was too busy to notice until last week. the real news here isn’t that -bit rc is crackable (everybody knew it could be done, eventually), the real news is that they compiled efficiency statistics on the various computer platforms that did the job. here’s the quote, straight from their press release: “our peak rate of , , kkeys/sec is equivalent to , mhz apple powerbook g laptops or , ghz amd athlon xp machines…” was capitalism the only difference? <a href="http://www.cera.com/commandingheights/” title=”commanding heights“>_commanding heights_ authors daniel yergen and joseph stanislaw tell us that workers in communist russia were not motivated to work simply because the government controlled economy offered no rewards for innovation. this they use as the basis for their argument that communism/government controlled economies were bad and capitalism was good. and what’s truly amazing is that in this obvious comparison between the usa and communist russia, they find the most significant difference to be economic. mc hawking drops some science you the opening to this site announces “yo! this site is your ultimate resource for information about stephen hawking the gangsta rapper.” and if that isn’t enough to make you go look there right now, then i suppose you feel bad for the poor guy and don’t like jerks who wish to make fun of him. anyway. just now he’s got a link up that points out one more sport i’ve never heard of or imagined: cup stacking. the first law of assignation the person [closest to the act/holding the instrument of the act], no matter how qualified or culpable is first to be assigned [credit/blame] for the act. natalie jeremijenko and the interaction between humans and technology it’s not for nothing that the mit technology review named natalie jeremijenko “one of the top one hundred young innovators.” anybody who bothers to read this blog should run out and look over her portfolio now. weeds and flowers weeds and flowers alike seek the sunlight — nobody can fault them for that — but some of them learn do it with beauty and grace. human-intoface: face=identity? from the artist’s statement: “images of faces hold little ability to communicate the totality of a personality. the essence of a personality is not something that is stored in a static two dimentional array of dots, grains, or pixels. rather, what is stored are subtle cues which signify base personality traits, such as a curl of a lip, squint of an eye, or pursing of the lips. these can work in series or combinations to suggest complexity of description, but ultimately, amount only to a caricature. hungry-man xxl! the marketers and designers for this product found their audience, and know how to speak to them. just look at the pictures. “i know what i like, and i like a lot of it” reads the text next to the over-weight, blue-collared white boy on the back. in bold yellow type at the bottom, it reads “it’s good to be full.” with . pounds of food, this preprocessed meal delivers calories, % of your recommended fat intake ( % of saturated fats), and % of your recommended sodium. book list: flight of passage i’m all wrapped up by flight of passage, rinker buck’s tale of his journey cross country with his brother in an old piper cub. as much as it’s a tale of flying, it’s a tale of teenage angst. both subjects that i identify with (but aren’t we supposed to grow out of teenage angst?). american tyranny the worst forms of tyranny are those so subtle, so deeply ingrained, so thoroughly controlling as not even to be consciously experienced. so there are americans who are afraid to entertain contrary notions for fear of jeopardizing their jobs, but who still think they are “free.” –michael parenti’s democracy for the few. corn flakes, mccarthy, and flag wavers this story would be more appropriate for early july — that’s probably when this flag-printed box of kellogg’s corn flakes was put on the shelf — but it was just last weekend when i came across it at our warren village market. of course, in early july, everything including corn chips and cat litter was available in patriotic red, white, and blue, so it really wouldn’t have stood out then. dreams. what do they mean? years ago, i used to wake up with a start. i’d be trying to sit up with my hands outstretched in front of me. i’d wake up thinking i’d been falling. now. i find that i wake up thinking i’d stubbed my toe or hit my head. somewhat unrelated: i’ve gotten no end of laughs and amusement from dion mcgregor dreams again, a collection of sleep talking from dion mcgregor, an apparently famous “somniloquist. casey’s sky diving adventure i made my one and only parachute jump back in the fall of . about a year ago i re-edited the video of that event. casey’s skydiving adventure o’reilly offers macos x conference the o’reilly folks aren’t the only old unix geeks who’ve been looking at mac os x with hungry eyes. mac os x is cool enough to get its own section on slash dot. and, of course, apple is pushing it’s ‘switch‘ campaign toward windows users. but as much as the o’reilly folks love mac os x, they probably wouldn’t be planning a conference about it if it wasn’t clear there were hordes of like-minded geeks willing to shell out the $ or so it costs to attend. vegas guide, part : introduction las vegas may be the most thoroughly american city. no other town has been so shaped by the singular desire to make a buck. churches and strip clubs coexist in close proximity. each competes for the hard luck — but not broke — gamblers seeking refuge from their losses. if capitalism works, it works in vegas. vegas is america’s liver. the worst of pop culture eventually finds a home someplace in las vegas or the surrounding clark county. vegas guide, part : peyote most of nevada’s land is under federal control. the pentagon, department of energy, and bureau of land management claim a total area of about % of the state. it’s mostly desert, and the desert dois best left alone, so few people seem to care. some towns, mostly old silver mining camps, persist amid the desert. horses graze free on the school ball field in blue diamond, nevada. the town sits on a spring in red rock canyon. vegas guide, part : nukes and moon hoaxes on a map, mercury sits a little northwest of las vegas. there is nothing to suggest that the town is inaccessible to the average tourist, but it is in fact a part of the nevada test site — a nuclear bomb testing facility. the site was formed in from land originally granted to the shoshone indians. nearly one thousand nuclear devices have been detonated there between its formation and , when president bush imposed a moratorium on tests that has been extended by succeeding administrations. vegas guide, part : flesh prostitution in vegas is illegal, but that’s okay. for a little jiggle, you can check out the innumerable gentlemen’s clubs and strip shows. even many of the ritzy hotels often have their own “tantalizing topless revues.” freemont street, the heart of old vegas and one of the city’s largest attractions, is home to more than one strip club. but a short drive will get you more than jiggle. fifty miles west of las vegas on highway , just accross the clark county line in nye county you’ll find the sleepy town of pahrump — “heart of the new old west” according to the welcome sign at the town line. morse museum mummy unmasked this isn’t current news by any stretch. the story was reported in the boston globe when it happened in , and can be found on the web at maine antique digest. it goes like this: the contents of the morse museum were auctioned off in the early s. among the spoils were two egyptian mummies. one of them landed in the hands of a maine antiques dealer. the egyptian government learned of the mummy, which was advertised as a ‘princess. redstone brewery’s product labels brewing cider takes a long time. …and most of it is just waiting. so while we wait, i draw up new labels. click for maison bisson’s summer drink hot weather demands cool drinks. lemonade is fine for the kids, but adults need a pitcher of something more entertaining. give it a try: vicker’s delight: part vodka parts lemonade dash lime juice dash orange juice prepare in a pitcher with ice and share. adjust quantities to taste. enjoy safely. the old scooter yes. the scooter was a thing of ridicule for most people, but i loved it. riding the scooter was like ‘playing bikes’ when i was ten. it was just fun, and i didn’t need an excuse to do it. i named her trixie, but most people just called her scooter. but the scooter is sold now. it went first to cliffy, then to chuck. did cliffy appreciate it like i did? airplane safety it may be a little bit cliche after being ridiculed in fight club (the line was “look at their faces, as calm as hindu cows.”), but i’ve always loved airplane safety guides. click for warren’s morse museum it’s hard to say which is more memorable: warren’s rocket or our morse museum. for larger picture, click who are these dorks? what a motley crew who work for its. click for pictures. newton: best pda ever just as i’m about to retire my old newton, just as i’m exporting the contacts and calendar entries, i rediscovered why the newton was — and still is — the best pda ever. the newton had a rough start back in the early s when the first model was released. i’ve never used an older model, but it’s clear that the handwriting recognition was bad enough to be ridiculed in comics and the simpsons. now even the conservatives agree: supporting the drug war supports terrorists this may be old news (it was published on may th, ) but, david r. henderson’s essay on how the drug war effects the war against terrorism is a must read for everybody. conservatives tell truth about drug war. why do i say the hoover institution is a pack of conservatives? because eric alterman says so. cape cod dining: ay caramba cafe sandee and i stumbled into the ay caramba cafe on main street in harwich at just the right time. we were starving and desperate for something other than fried sea food. diners can help themselves to chips and three varieties of homemade salsa. each is rather unique, and far more complex than the mild, medium, and hot descriptions we typically use to describe salsas. sandee and i both had the pork tomales that were on special — cheese tomales were also offered. cape cod our friends troy and karen were kind enough to invite us to cape cod to visit them. we lazed around on the beach, took in a show at the wellfleet drive-in, and twice gorged ourselves on fried seafood at arnold’s restaurant. geeks may take interest in cape cod’s involvment in the history of trans-atlantic communications. nauset light beach was a former terminus for many undersea telegraph cables. friendy links: see troy here, here, and here. doonesbury’s middle age slump a feature story by jesse walker in reason magazine’s july issue confirms something i’ve been worried about for a while: doonesbury isn’t what it used to be. walker gives us examples detailing trudeau’s mild conservitive shift, and his more unfortunate shift toward irrelevence. i’m too young to know the strip from its beginnings in the early s (or earlier), but we can all compare old and new cartoons online in the doonsebury retrospective. the incident the front the shocks and coil springs slowed the downward thrust of the front suspension as inertia, stable just moments before, pitches the vehicle forward. a small, unconscious rightward twitch of the steering wheel is amplified by tires which, at this moment, have greater than normal mechanical advantage. the turn, though slight, moves the center of gravity even farther forward and now to the left. the rear of the vehicle, lightly weighted under normal conditions, is riding at the full extension of the rear leaf springs. now playing at maison bisson while mainstream (commercial) pop music producers are anxiously introducing ever younger children to ever more sexualised music, they might be giants are busy making music for kids of all ages. their new album, no!, might sound fluffy and sacharine compared to the band’s earlier work, but so what. like so many of their songs, you’ll quickly be singing along. besides, sandee says “it’s just good music.” lustworthy: honda silver wing and reflex sure, italian scooters look great, but where do you get them serviced? motostrada in maryland has a great selection of new and vintage european scooters, but that’s the nearest dealership and service center. it’s a great shop, don’t get me wrong, but it’s not really a solution for people in northern new hampshire. so if i don’t trust biff at the local cycle shop to work on an european import scooter, what would i trust biff to work on? learning unix macos x’s unix underpinnings have had mac users asking the same question for a while now: “how can i learn unix?” and for those who really want to learn unix, i point them to ?leenfrisch’s <a href="http://www.oreilly.com/catalog/esa /” title="essential system administration, rd edition“>essential system administration, rd edition. it’s direct and concise, yet thorough. it was the book i turned to for an introduction to unix, and it’s a book i keep on my shelf as a reference when i need it. frozen mud slides — from scratch who wouldn’t enjoy a frozen mud slide on a hot summer day? typical recipes call for crushed ice and cream or ice cream. for some reason, we decided to try making them from ice cream, from scratch. the maisonbisson frozen mud slide this recipe requires an ice cream maker, we used the deni scoop factory. . cups heavy cream cup milk cup sugar . cups bailey’s dash vanilla mix ingredients in bowl, then pour into ice cream maker’s freezer container. the plan what we need is a van. a black van with red alloy wheels and a diamond bubble window. yeah. get on the jazz, sucka. streamripper saves mp radio to disk i must be an idiot not to have found streamripper sooner. in the days before walkmans i used to record radio broadcasts on an early portable cassette recorder so i could listen later. this is how i discovered “angel in a centerfold” and many other great cultural landmarks from the early s. of course, things have changed since then. my taste hasn’t improved so much as commercial radio has fallen. internet radio, thankfully, may rescue me. story review: derryl murphy’s last call one: i discovered fictionwise.com, a source all types of fiction in ebook formats. two: here’s the assignment that lead me to look for fictionwise in the first place. click for pdf. [update] it’s funny how things circulate on the web. i’ve googled myself enough to know how i show up in odd places, so i can understand how derryl murphy might have wondered how a review of one of his many stories appeared here. my new favorite pop i found a bottle of ibc cream soda at our famous warren village market and it quickly reminded me of why i love cream soda. but now, no other cream soda tastes as good. i’ve tried a few; they just make me sad. now i need to speak with the folks at the market to get a case of the good stuff. it can also be ordered from popsoda.com. pictures of the warren rocket warren is blessed with a rocket. it was once a intermediate range ballistic missile, but it’s basically the same rocket that launched america’s first astronauts allen b. shepherd and gus grissom into sub-orbital space. it’s enough to be proud of, anyway. roadsideamerica.com has a story on our rocket, but it’s based on reader reports and it seems people just don’t know what town they’re in when they see the thing. redstone brewery’s first steps in the fall of cliff convinced me that i needed to brew hard cider. in turn, i convinced him that we needed to brew lots of it. we soon bought barrels that had been used for cherry coke concentrate and found an orchard that would sell us bulk sweet cider. after siphoning the gallons from two barrels in my truck into two barrels prepared in my basement, adding sugar and other flavors, and pitching the yeast, we waited. color theory my overwhelming interest in earth-tones and browns leads me to look for them and define them numerically. a lousy overview of color models, especially the hsv model. originally written for one of my classes. click here for pdf. [update]: i’d like to point out a later story about color blindness and streetlights. tempo cameras need regular testing. don’t they? view tempo at .mac theater. originally put together to demonstrate synchronization of music and images. look for daria and her silly monkey, and a short appearance by travis. hammernode dynodns services hammernode dynamic dns services couldn’t be better. well, what could be better than a free, high quality service? headshots our new camera equipment arrived one day in august . obviously, it needed testing. this is the result.view headshots at .mac theater. that’s me looking like an idiot. and cliff too. sorry, this one isn’t “fast start.” you’ll have to wait until it loads all mbs. iug : houston officially i’m here to attend the innovative users’ group conference, but there’s a lot more to do in texas and i took a few extra days to do it. my brother lives just north of austin, and just north of that is waco. being so close, i had to go visit. …and while there, i couldn’t help but look for the branch davidian compound. houston is an interesting city, but two landmarks particularly interested me. looking at waco **texas stories** i had a chance to visit waco in april . here are some links that i gathered from that time. eventually i’ll post a story to go with them. dr. pepper museum waco visitor bureau red men museum texas ranger museum branch davidians contrasting houston texas stories the beer can house on the northwest side of town was built by john milkovisch starting in . over the next years he drank a six-pack per day to furnish and adorn the house with almost , cans. meanwhile, on the southeast side of town, cleveland turner looked to god to help get him off the sauce. as thanks for his salvation and sobriety, he gathered up all the trash in his neighborhood, painted it, and arranged it to look like flowers. galveston’s seawolf park **texas stories** while in texas i had an oportunity to see galveston and visit seawolf park. seawolf park is home to a wwii sub and an escort cruiser. it pleased me to no end that i was able to climb all over inside and outside both boats. i took more pictures there than anywhere else during my texas adventure. [](http://homepage.mac.com/misterbisson/seawolfpark/ _ .jpg) cavalla’s diving controls <td align="center" valign="middle"> [<img src="http://homepage. visiting the branch davidian compound texas stories work brought me to texas in april , but morbid curiosity brought me to waco. i found a story by dan tobias about the branch davidian compound and its remains. following his directions, i found my way to the site and later emailed dan with the changes i found since he last visited. my email to him is included in the body of this story, but i recommend you read dan’s story about the branch davidians first. quicktime embed tags apple’s docs on embedding qt media in web pages. it’s here mostly as a bookmark for me. click here for the docs. search from https://gist.github.com/eddiewebb/ feb f f ddd ae a cb ae, which continues: this file exists solely to respond to /search url with the related search layout template. no content shown here is rendered, all content is based in the template $theme/layouts/page/search.html setting a very low sitemap priority will tell search engines this is not important content. this implementation uses fusejs, jquery and mark.js the full details can be found in https://gist.github.com/eddiewebb/ feb f f ddd ae a cb ae. you should never see this content! home - libraries at vassar college skip to main content apply visit give about academics admissions campus life athletics arts students families employees alumnae/i page menu libraries today's hours: see all library hours » ask a librarian faq home find, borrow, request research libraries & collections using the libraries about ask a librarian feedback my account vassar college libraries now has access to overdrive: https://vassar.overdrive.com/ . for more information click here. provide selections upfront (robust) - this is the one we use search discover limit to: books articles audio video score advanced» only show items from the vassar catalog use the features of discover to search the classic vassar library catalog other ways to search: library catalog databases course reserves journals research guides worldcat connectny illiad digital library oa which option should i choose? ▶ search options discover - search books, articles, media, and more - all at the same time. this is a quick way of searching for an article by its title. choose this option for searching streamed audio databases. you can only show items that are also in the catalog. note that there are no articles in the catalog, so this option will be grayed out if you have chosen to search for articles. vassar catalog - search books, journals, and media but not articles. maximizes use of subject headings. browse shelves by call number. databases - see a list of all databases that provide access to articles, newspapers, encyclopedias, indexes, streaming audio and visual recordings, primary source collections, musical and artistic works, and more. course reserves - find material that your professor has placed on reserve for a class. search by instructor or course. journals - search for a specific journal by journal title. for articles, use discover or databases. research guides - find recommended resources and research tips by librarians on a wide variety of subjects and courses. more search tips ▶ library services for spring services for spring faculty: request materials for your courses faculty: request library instruction work remotely with library resources and get research help how to connect from off campus research help ask a librarian search course reserves more information » connect from off-campus forms faq info for: faculty | students find, borrow, request find, borrow, request find items you need in the library and online, get maps and call numbers, and ask a librarian for any questions you have. find, borrow, request using the libraries using the libraries learn more about the spaces and services that you can find here, such as information about accessibility, computers & printing, classrooms and teaching facilities, and more. using the libraries libraries & collections libraries & collections from rare materials to zines, from more than a million books to thousands of online journals: learn more about our libraries and collections. libraries & collections events calendar announcements: art library exhibit: s.m.s. (shit must stop) – - on view through the end of the fall semester vassar libraries support the black caucus of the ala statement condemning increased violence and racism towards black americans and people of color (click here)vassar college libraries join hathitrust! learn morevotes for women! online exhibition: view now powered by springshare. all rights reserved. login to libapps title × loading... close facebook twitter libraries find, borrow, request books & more / search ask a librarian maps and call numbers circulation and borrowing materials course reserves connectny and interlibrary loan video collection recent acquisitions search this site about the libraries fast facts mission statement strategic directions library administration staff directory research & instruction meet your librarians & ask your questions instruction citing sources open access information for... faculty students my accounts log in libraries and collections archives & special collections library art library digital library main (thompson) library music library government information zines collection development & management preservation feedback send feedback using the libraries accessibility visiting the libraries using spaces and services computers, laptops, and printing classrooms & teaching facilities library classroom forms connecting from off-campus senior lockers and carrels faq view our faq vassar college libraries raymond avenue, box poughkeepsie, ny - - librarysitemgr@vassar.edu vassar college digital collections skip to content ohio state nav bar skip to main content the ohio state university help buckeyelink map find people webmail search ohio state digital collectionsuniversity libraries home about help contact login search go digital collections billy ireland cartoon library & museum the billy ireland cartoon library & museum houses the world’s largest collection of materials related to cartoons and comics, including original art, books, magazines, journals, comic books, graphic novels, archival materials, and newspaper comic strip pages and clippings. byrd polar and climate research center archival program named in honor of one of america's most famous explorers, admiral richard e. byrd, the byrd polar and climate research center of the ohio state university is recognized internationally as a leader in polar and alpine research. the center's research programs are conducted throughout the world. jerome lawrence and robert e. lee theatre research institute the jerome lawrence and robert e. lee theatre research institute serves as an archive for performers, playwrights, choreographers, designers, producing organizations, and theatre and dance companies, among others, and advances the study and inspiration of the performing arts. music & dance library the music & dance library supports the teaching, research, and performance areas of the osu school of music and the department of dance. in addition to a large circulating collection of books, scores and audiovisual media, the library houses several special collections including sheet music, facsimiles of music manuscripts, and sound recordings. ohio public policy archives the ohio public policy archives is a collaborative effort of the ohio state university libraries special collections and the john glenn college of public affairs. its mission is to collect, preserve, and provide access to collections of historical documents relating to the function and history of policymaking in the federal branches of the u.s. government and informed by individuals and groups from the state of ohio. rare books and manuscripts library the rare books and manuscripts library serves as a laboratory for the study of “the book” in all its forms and includes collections spanning -years’ worth of literary and artistic production, from babylonian cuneiform tablets and medieval manuscripts to modern literary archives, avant poetry, and historical photographs. university archives since , university archives has served as the official memory of the ohio state university. with more than , cubic feet of manuscripts materials and approximately million images, university archives provides researchers, students, and life-long learners access to original documents, photographs, and objects to create their own interpretation of ohio state university history. geology library & map room this collection contains digitized items from ohio state’s geology library and map room. these locations combined contain a majority of ohio state’s over . million print maps holdings. maps include local, state, national, and international locations. on facebook on twitter on instagram on linkedin on youtube © the ohio state university - university libraries neil avenue mall | columbus, ohio | - -osul ( ) request an alternate format of this page | accessibility | privacy policy | contact us copyright information | details and exceptions digital preservation conference dlf about overview calendar faq foundational principles leadership strategic plan membership join the ndsa member orientation members groups overview interest groups content infrastructure standards and practices active working groups communications and publications conference program innovation awards levels of preservation ndsa agenda publications overview levels of preservation ndsa agenda osf repository conference digipres conference past digipres conferences news digital preservation conference about the ndsa and digital preservation the ndsa is a consortium of more than organizations committed to the long-term preservation and stewardship of digital information and cultural heritage, for the benefit of present and future generations. digital preservation (#digipres ) will be a crucial venue for intellectual exchange, community-building, development of good practices, and national-level agenda-setting in the field, helping to chart future directions for both the ndsa and digital stewardship. the ndsa strives to create a safe, accessible, welcoming, and inclusive event, and operates under the dlf forum’s code of conduct. conference materials and recordings held november , , the program, slide decks, and recordings of the presentations for digital preservation : get active with digital preservation are now all available. browse the program. slide decks are available on osf. recordings are available as a playlist on the ndsa youtube channel. we are so grateful to our digital preservation sponsors! keynote: jennifer ferretti jennifer ferretti will open this year’s digital preservation conference. jennifer a. ferretti (she/her) is an artist and digital initiatives librarian at the maryland institute college of art on piscataway land (baltimore, maryland). she is a first-generation american latina/mestiza whose librarianship is guided by critical perspectives, not neutrality. with a firm belief that art is information, she is interested in the research methodologies of artists and non-western forms of knowledge making and sharing. jennifer is a library journal mover and shaker, and founder and principal of we here,™️ a community dedicated to supporting folks who identify as black, indigenous, and people of color in libraries and archives. digipres organizing committee thank you to our wonderful planning committee! courtney c. mumma, texas digital library (chair) tricia patterson, harvard university (vice chair) heather barnes, wake forest university suzanne chase, georgetown university david cirella, yale university library greg colati, university of connecticut stefan elnabli, ucsd libraries elizabeth england, us national archives and records administration siobhan hagan, dc public library deirdre joyce, syracuse university julia kim, library of congress alex kinnaman, virginia tech university libraries kathryn michaelis, emory university jes neal, hampshire college dan noonan, the ohio state university krista oldham, clemson university libraries margo padilla, ny historical society museum & library thomas pulhamus, university of delaware aliya reich, clir/dlf amy rudersdorf, avp abby stambach, college of the holy cross lance stuchell, university of michigan library paige walker, boston college kristen weischedel, illinois institute of technology frederick zarndt, digital divide data calendar and past meetings future events: for the latest on upcoming events, see our ndsa calendar. more events relevant to the ndsa’s mission are to be found on the dlf community calendar. past meetings: an archive of digital preservation meetings from - can be found on the past digital preservation conference page. ndsa about members groups calendar social twitter itunes youtube news linkedin contact ndsa c/o clir+dlf south clark street, arlington, va e: ndsa@diglib.org ndsa the ndsa is proudly hosted by the digital library federation at clir. all content on this site is available for re-use under a cc by-sa . international license. dlf solrcloud | apache solr reference guide . . apache solr reference guide about this guide getting started solr tutorial a quick overview solr system requirements installing solr deployment and operations solr control script reference solr configuration files taking solr to production making and restoring backups running solr on hdfs solrcloud on aws ec upgrading a solr cluster indexupgradertool solr upgrade notes major changes in solr major changes in solr major changes from solr to solr using the solr administration user interface overview of the solr admin ui logging cloud screens collections / core admin java properties thread dump suggestions screen collection-specific tools analysis screen dataimport screen documents screen files screen query screen stream screen schema browser screen core-specific tools ping plugins & stats screen replication screen segments info documents, fields, and schema design overview of documents, fields, and schema design solr field types field type definitions and properties field types included with solr working with currencies and exchange rates working with dates working with enum fields working with external files and processes field properties by use case defining fields copying fields dynamic fields other schema elements schema api putting the pieces together docvalues schemaless mode understanding analyzers, tokenizers, and filters analyzers about tokenizers about filters tokenizers filter descriptions charfilterfactories language analysis phonetic matching running your analyzer indexing and basic data operations introduction to solr indexing post tool uploading data with index handlers transforming and indexing custom json indexing nested child documents uploading data with solr cell using apache tika uploading structured data store data with the data import handler updating parts of documents detecting languages during indexing de-duplication content streams reindexing searching overview of searching in solr velocity search ui relevance query syntax and parsing common query parameters the standard query parser the dismax query parser the extended dismax (edismax) query parser function queries local parameters in queries other parsers json request api json query dsl json facet api json faceting domain changes faceting blockjoin faceting highlighting spell checking query re-ranking learning to rank transforming result documents searching nested child documents suggester morelikethis pagination of results collapse and expand results result grouping result clustering spatial search the terms component the term vector component the stats component the query elevation component the tagger handler response writers velocity response writer near real time searching realtime get exporting result sets parallel sql interface solr jdbc - dbvisualizer solr jdbc - squirrel sql solr jdbc - apache zeppelin solr jdbc - python/jython solr jdbc - r analytics component analytics expression sources analytics mapping functions analytics reduction functions streaming expressions stream source reference stream decorator reference stream evaluator reference math expressions scalar math vector math variables matrices and matrix math streams and vectorization text analysis and term vectors statistics probability distributions monte carlo simulations time series linear regression interpolation, derivatives and integrals curve fitting digital signal processing machine learning computational geometry graph traversal stream request handler api solrcloud getting started with solrcloud how solrcloud works shards and indexing data in solrcloud distributed requests aliases solrcloud resilience solrcloud recoveries and write tolerance solrcloud query routing and read tolerance solrcloud configuration and parameters setting up an external zookeeper ensemble using zookeeper to manage configuration files collections api cluster and node managment commands collection management commands collection aliasing shard management commands replica management commands parameter reference command line utilities solrcloud with legacy configuration files configsets api rule-based replica placement cross data center replication (cdcr) cdcr architecture cdcr configuration cross data center replication operations cdcr api solrcloud autoscaling overview of solrcloud autoscaling autoscaling policy and preferences solrcloud autoscaling triggers solrcloud autoscaling trigger actions solrcloud autoscaling listeners solrcloud autoscaling automatically adding replicas solrcloud autoscaling fault tolerance autoscaling api migrating rule-based replica rules to autoscaling policies colocating collections legacy scaling and distribution introduction to scaling and distribution distributed search with index sharding index replication combining distribution and replication merging indexes solr plugins lib directories and directives package management package manager internals adding custom plugins in solrcloud mode the well-configured solr instance configuring solrconfig.xml datadir and directoryfactory in solrconfig schema factory definition in solrconfig indexconfig in solrconfig requesthandlers and searchcomponents in solrconfig initparams in solrconfig updatehandlers in solrconfig query settings in solrconfig requestdispatcher in solrconfig update request processors codec factory solr cores and solr.xml format of solr.xml defining core.properties coreadmin api configsets resource loading configuration apis blob store api config api request parameters api managed resources implicit requesthandlers jvm settings v api monitoring solr metrics reporting metrics history mbean request handler configuring logging using jmx with solr monitoring solr with prometheus and grafana performance statistics reference distributed solr tracing securing solr configuring authentication, authorization and audit logging basic authentication plugin hadoop authentication plugin kerberos authentication plugin rule-based authorization plugins jwt authentication plugin enabling ssl audit logging zookeeper access control client apis introduction to client apis choosing an output format using solrj using javascript using python using solr from ruby other clients further assistance solr glossary errata how to contribute to solr documentation solr . solr website other formats archived pdfs other versions online solr resources solr javadocs lucene/solr source code solr community links solrcloud apache solr includes the ability to set up a cluster of solr servers that combines fault tolerance and high availability. called solrcloud, these capabilities provide distributed indexing and search capabilities, supporting the following features: central configuration for the entire cluster automatic load balancing and fail-over for queries zookeeper integration for cluster coordination and configuration. solrcloud is flexible distributed search and indexing, without a master node to allocate nodes, shards and replicas. instead, solr uses zookeeper to manage these locations, depending on configuration files and schemas. queries and updates can be sent to any server. solr will use the information in the zookeeper database to figure out which servers need to handle the request. in this section, we’ll cover everything you need to know about using solr in solrcloud mode. we’ve split up the details into the following topics: getting started with solrcloud how solrcloud works shards and indexing data in solrcloud distributed requests solrcloud resilience solrcloud recoveries and write tolerance solrcloud query routing and read tolerance solrcloud configuration and parameters setting up an external zookeeper ensemble using zookeeper to manage configuration files zookeeper access control collections api parameter reference command line utilities solrcloud with legacy configuration files configsets api rule-based replica placement cross data center replication (cdcr) solrcloud autoscaling colocating collections together stream request handler api getting started with solrcloud © apache software foundation. all rights reserved. site version: . . site last generated: - - the code lib journal mission editorial committee process and structure code lib issue , - - editorial: for pandemic times such as this peter murray a pandemic changes the world and changes libraries. open source tools for scaling data curation at qdr nicholas weber, sebastian karcher, and james myers this paper describes the development of services and tools for scaling data curation services at the qualitative data repository (qdr). through a set of open-source tools, semi-automated workflows, and extensions to the dataverse platform, our team has built services for curators to efficiently and effectively publish collections of qualitatively derived data. the contributions we seek to make in this paper are as follows: . we describe ‘human-in-the-loop’ curation and the tools that facilitate this model at qdr; . we provide an in-depth discussion of the design and implementation of these tools, including applications specific to the dataverse software repository, as well as standalone archiving tools written in r; and . we highlight the role of providing a service layer for data discovery and accessibility of qualitative data. keywords: data curation; open-source; qualitative data from text to map: combing named entity recognition and geographic information systems charlie harper and r. benjamin gorham this tutorial shows readers how to leverage the power of named entity recognition (ner) and geographic information systems (gis) to extract place names from text, geocode them, and create a public-facing map. this process is highly useful across disciplines. for example, it can be used to generate maps from historical primary sources, works of literature set in the real world, and corpora of academic scholarship. in order to lead the reader through this process, the authors work with a article sample of the covid- open research dataset challenge (cord- ) dataset. as of the date of writing, cord- includes , full-text articles with metadata. using this sample, the authors demonstrate how to extract locations from the full-text with the spacy library in python, highlight methods to clean up the extracted data with the pandas library, and finally teach the reader how to create an interactive map of the places using arcgis online. the processes and code are described in a manner that is reusable for any corpus of text using integrated library systems and open data to analyze library cardholders greg sohanchyk and dan briem the harrison public library in westchester county, new york operates two library buildings in harrison: the richard e. halperin memorial library building (the library’s main building, located in downtown harrison) and a west harrison branch location. as part of its latest three-year strategic plan, the library sought to use existing resources to improve understanding of its cardholders at both locations. to do so, we needed to link the circulation data in our integrated library system, evergreen, to geographic data and demographic data. we decided to build a geodemographic heatmap that incorporated all three aforementioned types of data. using evergreen, american community survey (acs) data, and google maps, we plotted each cardholder’s residence on a map, added census boundaries (called tracts) and our town’s borders to the map, and produced summary statistics for each tract detailing its demographics and the library card usage of its residents. in this article, we describe how we acquired the necessary data and built the heatmap. we also touch on how we safeguarded the data while building the heatmap, which is an internal tool available only to select authorized staff members. finally, we discuss what we learned from the heatmap and how libraries can use open data to benefit their communities. update oclc holdings without paying additional fees: a patchwork approach nicole wood and scott shumate accurate oclc holdings are vital for interlibrary loan transactions. however, over time weeding projects, replacing lost or damaged materials, and human error can leave a library with a catalog that is no longer reflected through oclc. while oclc offers reclamation services to bring poorly maintained collections up-to-date, the associated fee may be cost prohibitive for libraries with limited budgets. this article will describe the process used at austin peay state university to identify, isolate, and update holdings using oclc collection manager queries, marcedit, excel, and python. some portions of this process are completed using basic coding; however, troubleshooting techniques will be included for those with limited previous experience. data reuse in linked data projects: a comparison of alma and share-vde bibframe networks jim hahn this article presents an analysis of the enrichment, transformation, and clustering used by vendors casalini libri/@cult and ex libris for their respective conversions of marc data to bibframe. the analysis considers the source marc data used by alma then the enrichment and transformation of marc data from share-vde partner libraries. the clustering of linked data into a bibframe network is a key outcome of data reuse in linked data projects and fundamental to the improvement of the discovery of library collections on the web and within search systems. collectionbuilder-contentdm: developing a static web ‘skin’ for contentdm-based digital collections devin becker, evan williamson, olivia wikle unsatisfied with customization options for contentdm, librarians at university of idaho library have been using a modern static web approach to creating digital exhibit websites that sit in front of the digital repository. this “skin” is designed to provide users with new pathways to discover and explore collection content and context. this article describes the concepts behind the approach and how it has developed into an open source, data-driven tool called collectionbuilider-contentdm. the authors outline the design decisions and principles guiding the development of collectionbuilder, and detail how a version is used at the university of idaho library to collaboratively build digital collections and digital scholarship projects. automated collections workflows in gobi: using python to scrape for purchase options katharine frazier the nc state university libraries has developed a tool for querying gobi, our print and ebook ordering vendor platform, to automate monthly collections reports. these reports detail purchase options for missing or long-overdue items, as well as popular items with multiple holds. gobi does not offer an api, forcing staff to conduct manual title-by-title searches that previously took up to hours per month. to make this process more efficient, we wrote a python script that automates title searches and the extraction of key data (price, date of publication, binding type) from gobi. this tool can gather data for hundreds of titles in half an hour or less, freeing up time for other projects. this article will describe the process of creating this script, as well as how it finds and selects data in gobi. it will also discuss how these results are paired with nc state’s holdings data to create reports for collection managers. lastly, the article will examine obstacles that were experienced in the creation of the tool and offer recommendations for other organizations seeking to automate collections workflows. testing remote access to e-resource with codeceptjs ralf weber at the badische landesbibliothek karlsruhe (blb) we offer a variety of e-resources with different access requirements. on the one hand, there is free access to open access material, no matter where you are. on the other hand, there are e-resources that you can only access when you are in the rooms of the blb. we also offer e-resources that you can access from anywhere, but you must have a library account for authentication to gain access. to test the functionality of these access methods, we have created a project to automatically test the entire process from searching our catalogue, selecting a hit, logging in to the provider’s site and checking the results. for this we use the end end testing framework codeceptjs. issn - current issue issue , - - previous issues issue , - - issue , - - issue , - - issue , - - older issues for authors call for submissions article guidelines log in this work is licensed under a creative commons attribution . united states license. catmandu catmandu about download tutorial may , catmandu . on may th , nicolas steenlant (our main developer and guru of catmandu) released version . of our catmandu toolkit with some very interesting new features. the main addition is a brand new way how catmandu fix-es can be implemented using the new catmandu::path implementation. this coding by nicolas will make it much easier and straightforward to implement any kind of fixes in perl. in the previous versions of catmandu there were only two options to create new fixes: create a perl package in the catmandu::fix namespace which implements a fix method. this was very easy: update the $data hash you got as first argument, return the updated $data and you were done. then disadvantage was that accessing fields in a deeply nested record was tricky and slow to code. create a perl package in the catmandu::fix namespace which implemented emit functions. these were functions that generate perl code on the fly. using emit functions it was easier to get fast access to deeply nested data. but, to create fix packages was pretty complex. in catmandu . there is now support for a third and easy way to create new fixes using the catmandu::fix::builder and catmandu::fix::path class. let me give an simple example of a skeleton fix that does nothing: package catmandu::fix::rot ; use catmandu::sane; use moo; use catmandu::util::path qw(as_path); use catmandu::fix::has; with 'catmandu::fix::builder'; has path => (fix_arg => ); sub _build_fixer { my ($self) = @_; sub { my $data = $_[ ]; # ..do some magic here ... $data; } } ; in the code above we start implementing a rot (path) fix that should read a string on a json path and encrypt it using the rot algorithm. this fix is only the skeleton which doesn’t do anything. what we have is: we import the as_path method be able to easily access data on json paths/ we import catmandu::fix::has to be able to use has path constructs to read in arguments for our fix. we import catmandu::fix::builder to use the new catmandu . builder class provides a _build_fixermethod. the builder is nothing more than a closure that reads the data, does some action on the data and return the data. we can use this skeleton builder to implement our rot algorithm. add these lines instead of the # do some magic part: # on the path update the string value... as_path($self->path)->updater( if_string => sub { my $value = shift; $value =~ tr{n-za-mn-za-m}{a-za-z}; $value; }, )->($data); the as_path method receives a json path string an creates an object which you can use to manipulate data on that path. one can update the values found with the updater method, or read data at that path with the getter method or create a new path with the creator method. in our example, we update the string found at the json path using if_string condition. the updaterhas many conditions: if_string needs a closure what should happen when a string is found on the json path. if_array_ref needs a closure what should happen when an array is found on the json path. if_hash_refneeds a closure what should happen when a hash is found on the json path. in our case we are only interested in transforming strings using our rot (path) fix. the rot algorithm is very easy and only switched the order of some characters. when we execute this fix on some sample data we get this result: $ catmandu -i lib convert null to yaml --fix 'add_field(demo,hello);rot v (demo)' --- demo: uryyb ... in this case the fix can be written much shorter when we know that every catmandu::path method return a closure (hint: look at the ->($data) in the code. the complete fix can look like: package catmandu::fix::rot ; use catmandu::sane; use moo; use catmandu::util::path qw(as_path); use catmandu::fix::has; with 'catmandu::fix::builder'; has path => (fix_arg => ); sub _build_fixer { my ($self) = @_; # on the path update the string value... as_path($self->path)->updater( if_string => sub { my $value = shift; $value =~ tr{n-za-mn-za-m}{a-za-z}; $value; }, ); } ; this is as easy as it can get to manipulate deeply nested data with your own perl tools. all the code is in perl, there is no limit on the number of external cpan packages one can include in these builder fixes. we can’t wait what catmandu extensions you will create. written by hochstenbach leave a comment posted in advanced, updates tagged with catmandu, fix language, perl april , lpw : “contrarian perl” – tom hukins at : , tom hukins shares his enthusiasm for catmandu! written by hochstenbach leave a comment posted in uncategorized june , introducing filestores catmandu is always our tool of choice when working with structured data. using the elasticsearch or mongodb catmandu::store-s it is quite trivial to store and retrieve metadata records. storing and retrieving a yaml, json (and by extension xml, marc, csv,…) files can be as easy as the commands below: $ catmandu import yaml to database < input.yml $ catmandu import json to database < input.json $ catmandu import marc to database < marc.data $ catmandu export database to yaml > output.yml a catmandu.yml configuration file is required with the connection parameters to the database: $ cat catmandu.yml --- store: database: package: elasticsearch options: client: ' _ ::direct' index_name: catmandu ... given these tools to import and export and even transform structured data, can this be extended to unstructured data? in institutional repositories like librecat we would like to manage metadata records and binary content (for example pdf files related to the metadata). catmandu . introduces the catmandu::filestore as an extension to the already existing catmandu::store to manage binary content. a catmandu::filestore is a catmandu::store where each catmandu::bag acts as a “container” or a “folder” that can contain zero or more records describing file content. the files records themselves contain pointers to a backend storage implementation capable of serialising and streaming binary files. out of the box, one catmandu::filestore implementation is available catmandu::store::file::simple, or short file::simple, which stores files in a directory. some examples. to add a file to a filestore, the stream command needs to be executed: $ catmandu stream /tmp/myfile.pdf to file::simple --root /data --bag --id myfile.pdf in the command above: /tmp/myfile.pdf is the file up be uploaded to the file::store. file::simple is the name of the file::store implementation which requires one mandatory parameter, --root /data which is the root directory where all files are stored. the--bag is the “container” or “folder” which contains the uploaded files (with a numeric identifier ). and the --id myfile.pdf is the identifier for the new created file record. to download the file from the file::store, the stream command needs to be executed in opposite order: $ catmandu stream file::simple --root /data --bag --id myfile.pdf to /tmp/file.pdf or $ catmandu stream file::simple --root /data --bag --id myfile.pdf > /tmp/file.pdf on the file system the files are stored in some deep nested structure to be able to spread out the file::store over many disks: /data `--/ `--/ `--/ `--/myfile.pdf a listing of all “containers” can be retreived by requesting an export of the default (index) bag of the file::store: $ catmandu export file::simple --root /data to yaml _id: ... a listing of all files in the container “ ” can be done by adding the bag name to the export command: $ catmandu export file::simple --root /data --bag to yaml _id: myfile.pdf _stream: !!perl/code '{ "dummy" }' content_type: application/pdf created: md : '' modified: size: ... each file::store implementation supports at least the fields presented above: _id: the name of the file _stream: a callback function to retrieve the content of the file (requires an io::handle as input) content_type: the mime-type of the file created: a timestamp when the file was created modified: a timestamp when the file was last modified size: the byte length of the file md : optional a md checksum we envision in catmandu that many implementations of filestores can be created to be able to store files in github, bagits, fedora commons and more backends. using the catmandu::plugin::sidecar catmandu::filestore-s and catmandu::store-s can be combined as one endpoint. using catmandu::store::multi and catmandu::store::file::multi many different implementations of stores and filestores can be combined. this is a short introduction, but i hope you will experiment a bit with the new functionality and provide feedback to our project. written by hochstenbach leave a comment posted in uncategorized march , catmandu . catmandu . has been released to with some nice new features. there are some new fix routines that were asked by our community: error the “error” fix stops immediately the execution of the fix script and throws an error. use this to abort the processing of a data stream: $ cat myfix.fix unless exists(id) error("no id found?!") end $ catmandu convert json --fix myfix.fix < data.json valid the “valid” fix condition can be used to validate a record (or part of a record) against a jsonschema. for instance we can select only the valid records from a stream: $ catmandu convert json --fix 'select valid('', jsonschema, schema:myschema.json)' < data.json or, create some logging: $ cat myfix.fix unless valid(author, jsonschema, schema:authors.json) log("errors in the author field") end $ catmandu convert json --fix myfix.fix < data.json rename the “rename” fix can be used to recursively change the names of fields in your documents. for example, when you have this json input: { "foo.bar": " ", "my.name": "patrick" } you can transform all periods (.) in the key names to underscores with this fix: rename('','\.','_') the first parameter is the fields “rename” should work on (in our case it is an empty string, meaning the complete record). the second and third parameters are the regex search and replace parameters. the result of this fix is: { "foo_bar": " ", "my_name": "patrick" } the “rename” fix will only work on the keys of json paths. for example, given the following path: my.deep.path.x.y.z the keys are: my deep path x y z the second and third argument search and replaces these seperate keys. when you want to change the paths as a whole take a look at the “collapse()” and “expand()” fixes in combination with the “rename” fix: collapse() rename('',"my\.deep","my.very.very.deep") expand() now the generated path will be: my.very.very.deep.path.x.y.z of course the example above could be written more simple as “move_field(my.deep,my.very.very.deep)”, but it serves as an example that powerful renaming is possible. import_from_string this fix is a generalisation of the “from_json” fix. it can transform a serialised string field in your data into an array of data. for instance, take the following yaml record: --- foo: '{"name":"patrick"}' ... the field ‘foo’ contains a json fragment. you can transform this json into real data using the following fix: import_from_string(foo,json) which creates a ‘foo’ array containing the deserialised json: --- foo: - name: patrick the “import_from_string” look very much like the “from_json” string, but you can use any catmandu::importer. it always created an array of hashes. for instance, given the following yaml record: --- foo: "name;hobby\nnicolas;drawing\npatrick;music" you can transform the csv fragment in the ‘foo’ field into data by using this fix: import_from_string(foo,csv,sep_char:";") which gives as result: --- foo: - hobby: drawing name: nicolas - hobby: music name: patrick ... i the same way it can process marc, xml, rdf, yaml or any other format supported by catmandu. export_to_string the fix “export_to_string” is the opposite of “import_from_string” and is the generalisation of the “to_json” fix. given the yaml from the previous example: --- foo: - hobby: drawing name: nicolas - hobby: music name: patrick ... you can create a csv fragment in the ‘foo’ field with the following fix: export_to_string(foo,csv,sep_char:";") which gives as result: --- foo: "name;hobby\nnicolas;drawing\npatrick;music" search_in_store the fix “search_in_store” is a generalisation of the “lookup_in_store” fix. the latter is used to query the “_id” field in a catmandu::store and return the first hit. the former, “search_in_store” can query any field in a store and return all (or a subset) of the results. for instance, given the yaml record: --- foo: "(title:abc or author:dave) and not year: " ... then the following fix will replace the ‘foo’ field with the result of the query in a solr index: search_in_store('foo', store:solr, url: 'http://localhost: /solr/catalog') as a result, the document will be updated like: --- foo: start: , limit: , hits: [...], total: ... where start: the starting index of the search result limit: the number of result per page hits: an array containing the data from the result page total: the total number of search results every catmandu::solr can have another layout of the result page. look at the documentation of the catmandu::solr implementations for the specific details. thanks for all your support for catmandu and keep on data converting 🙂 written by hochstenbach leave a comment posted in uncategorized june , metadata analysis at the command-line i was last week at the elag conference in copenhagen and attended the excellent workshop by christina harlow of cornell university on migrating digital collections metadata to rdf and fedora . one of the important steps required to migrate and model data to rdf is understanding what your data is about. probably old systems need to be converted for which little or no documentation is available. instead of manually processing large xml or marc dumps, tools like metadata breakers can be used to find out which fields are available in the legacy system and how they are used. mark phillips of the university of north texas wrote recently in code lib a very inspiring article how this could be done in python. in this blog post i’ll demonstrate how this can be done using a new catmandu tool: catmandu::breaker. to follow the examples below, you need to have a system with catmandu installed. the catmandu::breaker tools can then be installed with the command: $ sudo cpan catmandu::breaker a breaker is a command that transforms data into a line format that can be easily processed with unix command line tools such as grep, sort, uniq, cut and many more. if you need an introduction into unix tools for data processing please follow the examples johan rolschewski of berlin state library and i presented as an elag bootcamp. as a simple example lets create a yaml file and demonstrate how this file can be analysed using catmandu::breaker: $ cat test.yaml --- name: john colors: - black - yellow - red institution: name: acme years: - - - - this example has a combination of simple name/value pairs a list of colors and a deeply nested field. to transform this data into the breaker format execute the command: $ catmandu convert yaml to breaker < test.yaml colors[] black colors[] yellow colors[] red institution.name acme institution.years[] institution.years[] institution.years[] institution.years[] name john the breaker format is a tab-delimited output with three columns: an record identifier: read from the _id field in the input data, or a counter when no such field is present. a field name. nested fields are seperated by dots (.) and list are indicated by the square brackets ([]) a field value when you have a very large json or yaml field and need to find all the values of a deeply nested field you could do something like: $ catmandu convert yaml to breaker < data.yaml | grep "institution.years" using catmandu you can do this analysis on input formats such as json, yaml, xml, csv, xls (excell). just replace the yaml by any of these formats and run the breaker command. catmandu can also connect to oai-pmh, z . or databases such as mongodb, elasticsearch, solr or even relational databases such as mysql, postgres and oracle. for instance to get a breaker format for an oai-pmh repository issue a command like: $ catmandu convert oai --url http://lib.ugent.be/oai to breaker if your data is in a database you could issue an sql query like: $ catmandu convert dbi --dsn 'dbi:oracle' --query 'select * from table where ...' --user 'user/password' to breaker some formats, such as marc, doesn’t provide a great breaker format. in catmandu, marc files are parsed into a list of list. running a breaker on a marc input you get this: $ catmandu convert marc to breaker < t/camel.usmarc | head fol record[][] ldr fol record[][] _ fol record[][] cam a fol record[][] fol record[][] _ fol record[][] fol fol record[][] fol record[][] fol record[][] fol record[][] a the marc fields are part of the data, not part of the field name. this can be fixed by adding a special ‘marc’ handler to the breaker command: $ catmandu convert marc to breaker --handler marc < t/camel.usmarc | head fol ldr cam a fol fol fol imchf fol . fol s nyua eng fol a fol a (paper/cd-rom : alk. paper) fol a dlc fol c dlc fol d dlc now all the marc subfields are visible in the output. you can use this format to find, for instance, all unique values in a marc file. lets try to find all unique values: $ catmandu convert marc to breaker --handler marc < camel.usmarc | grep "\t " | cut -f | sort -u s nyua eng s mau eng s njua eng s cau b eng s caua eng s mau eng s mau eng s mau eng s mau eng s cau eng nam a catmandu::breaker doesn’t only break input data in a easy format for command line processing, it can also do a statistical analysis on the breaker output. first process some data into the breaker format and save the result in a file: $ catmandu convert marc to breaker --handler marc < t/camel.usmarc > result.breaker now, use this file as input for the ‘catmandu breaker’ command: $ catmandu breaker result.breaker | name | count | zeros | zeros% | min | max | mean | median | mode | variance | stdev | uniq | entropy | |------|-------|-------|--------|-----|-----|------|--------|--------|----------|-------|------|---------| | | | | . | | | | | | | | | . / . | | | | | . | | | | | | | | | . / . | | | | | . | | | | | | | | | . / . | | | | | . | | | | | | | | | . / . | | a | | | . | | | | | | | | | . / . | | a | | | . | | | . | | | . | . | | . / . | | a | | | . | | | | | | | | | . / . | | c | | | . | | | | | | | | | . / . | | d | | | . | | | . | . | [ , ] | . | . | | . / . | | a | | | . | | | | | | | | | . / . | | a | | | . | | | | | | | | | . / . | | b | | | . | | | | | | | | | . / . | | | | | . | | | | | | | | | . / . | | a | | | . | | | | | | | | | . / . | | a | | | . | | | . | | | . | . | | . / . | | d | | | . | | | . | | | . | . | | . / . | | q | | | . | | | . | | | . | . | | . / . | | a | | | . | | | . | | | . | . | | . / . | | c | | | . | | | . | | | . | . | | . / . | | d | | | . | | | . | | | . | . | | . / . | | a | | | . | | | | | | | | | . / . | | b | | | . | | | . | | | . | . | | . / . | | c | | | . | | | . | | | . | . | | . / . | | a | | | . | | | . | | | . | . | | . / . | | a | | | . | | | | | | | | | . / . | | b | | | . | | | | | | | | | . / . | | c | | | . | | | | | | | | | . / . | | a | | | . | | | . | | | . | . | | . / . | | a | | | . | | | | | | | | | . / . | | b | | | . | | | . | | | . | . | | . / . | | c | | | . | | | . | | | . | . | | . / . | | e | | | . | | | . | | | . | . | | . / . | | a | | | . | | | . | | | . | . | | . / . | | a | | | . | | | . | | | . | . | | . / . | | a | | | . | | | . | | | . | . | | . / . | | a | | | . | | | . | | | . | . | | . / . | | v | | | . | | | . | | | . | . | | . / . | | a | | | . | | | . | | | . | . | | . / . | | ldr | | | . | | | | | | | | | . / . as a result you get a table listing the usage of subfields in all the input records. from this output we can learn: the ‘ ’ field is available in records (see: count) one record doesn’t contain a ‘ a’ subfield (see: zeros) the ‘ a’ is available in all records at least once at most times (see: min, max) only out of ‘ a’ subfields have unique values (see: uniq) the last column ‘entropy’ provides a number how interesting the field is for search engines. the higher the entropy, the more uniq content can be found. i hope this tools are of some use in your projects! written by hochstenbach comments posted in uncategorized may , catmandu . catmandu . has been released today. there has been some speed improvements processing fixes due to switching from the data::util to the ref::util package which has better a support on many perl platforms. for the command line there is now support for preprocessing fix scripts. this means, one can read in variables from the command line into a fix script. for instance, when processing data you might want to keep some provenance data about your data sources in the output. this can be done with the following commands: $ catmandu convert marc --fix myfixes.fix --var source=publisher --var date= - < data.mrc with a myfixes.fix like: add_field(my_source,{{source}}) add_field(my_data,{{date}}) marc_field( ,title) marc_field( ,issn) . . . etc . . your json output will now contain the clean ‘title’ and ‘issn’ fields but also for each record a ‘my_source’ with value ‘publisher ’ and a ‘my_date’ with value ‘ - ’. by using the text::hogan compiler full support of the mustache language is available. in this new catmandu version there have been also some new fix functions you might want to try out, see our fixes cheat sheet for a full overview. written by hochstenbach leave a comment posted in updates april , parallel processing with catmandu in this blog post i’ll show a technique to scale out your data processing with catmandu. all catmandu scripts use a single process, in a single thread. this means that if you need to process times as much data , you need times at much time. running a catmandu convert command with the -v option will show you the speed of a typical conversion: $ catmandu convert -v marc to json --fix heavy_load.fix < input.marc > output.json added ( /sec) added ( /sec) added ( /sec) added ( /sec) added ( /sec) added ( /sec) added ( /sec) added ( /sec) added ( /sec) added ( /sec) in the example above we process an ‘input.marc’ marc file into a ‘output.json’ json file with some difficult data cleaning in the ‘heave_load.fix’ fix script. using a single process we can reach about records per second. it would take . hours to process one million records and hours to process ten million records. can we make this any faster? when you buy a computer they are all equipped with multiple processors. using a single process, only one of these processors are used for calculations. one would get much ‘bang for the buck’ if all the processors could be used. one technique to do that is called ‘parallel processing’. to check the amount of processors available on your machine use the file ‘/proc/cpuinfo’: on your linux system: $ cat /proc/cpuinfo | grep processor processor : processor : the example above shows two lines: i have two cores available to do processing on my laptop. in my library we have servers which contain , , or more processors. this means that if we could do our calculations in a smart way then our processing could be , , or times as fast (in principle). to check if your computer is using all that calculating power, use the ‘uptime’ command: $ uptime : : up days, : , users, load average: . , . , . in the example above i ran did ‘uptime’ on one of our servers with processors. it shows a load average of about . to . . this means that in the last minutes between and processors where being used and the other two did nothing. if the load average is less than the number of cores ( in our case) it means: the server is waiting for input. if the load average is equal to the number of cores it means: the server is using all the cpu power available. if the load is bigger than the number of cores, then there is more work available than can be executed by the machine, some processes need to wait. now you know some unix commands we can start using the processing power available on your machine. in my examples i’m going to use a unix tool called ‘gnu parallel’ to run catmandu scripts on all the processors in my machine in the most efficient way possible. to do this you need to install gnu parallel: sudo yum install parallel the second ingredient we need is a way to cut our input data into many parts. for instance if we have a processor machine we would like to create equal chunks of data to process in parallel. there are very many ways to cut your data in to many parts. i’ll show you a trick we use in at ghent university library with help of a mongodb installation. first install, mongodb and the mongodb catmandu plugins (these examples are taken from our centos documentation): $ sudo cat > /etc/yum.repos.d/mongodb.repo < part $ catmandu export mongodb --database_name -q '{"part.rand ": }' > part we are going to use these catmandu commands in a bash script which makes use of gnu parallel run many conversions simultaneously. #!/bin/bash # file: parallel.sh cpu=$ if [ "${cpu}" == "" ]; then /usr/bin/parallel -u $ {} < result.${cpu}.json fi this example script above shows how a conversion process could run on a -processor machine. the lines with ‘/usr/bin/parallel’ show how gnu parallel is used to call this script with two arguments ‘ ’ and ‘ ’ (for the -processor example). in the lines with ‘catmandu export’ shows how chunks of data are read from the database and processed with the ‘heavy_load.fix’ fix script. if you have a -processor machine, you would need to provide parallel an input which contains the numbers , , to and change the query to ‘part.rand ’. gnu parallel is a very powerfull command. it gives the opportunity to run many processes in parallel and even to spread out the load over many machines if you have a cluster. when all these machines have access to your mongodb database then all can receive chunks of data to be processed. the only task left is to combine all results which can be as easy as a simple ‘cat’ command: $ cat result.*.json > final_result.json written by hochstenbach comments posted in advanced tagged with catmandu, json path, library, linux, marc, parallel procesing, perl february , catmandu . after years of programming, minor releases we are finally there: the release of catmandu . ! we have pushed the test coverage of the code to . % and added and cleaned a lot of our documentation. for the new features read our changes file. a few important changes should be noted. by default catmandu will read and write valid json files. in previous versions the default input format was (new)line delimited json records as in: {"record":" "} {"record":" "} {"record":" "} instead of the valid json array format: [{"record":" "},{"record":" "},{"record":" "}] the old format can still be used as input but will be read much faster when using the –line_delimited option on the command line. thus, write: # fast $ catmandu convert json --line_delimited < lines.json.txt instead of: # slow $ catmandu convert json < lines.json.txt by default catmandu will export in the valid json-array format. if you still need to use the old format, then provide the –line_delimited option on the command line: $ catmandu convert yaml to json --line_delimited < data.yaml we thank all contributors for these wonderful four years of open source coding and we wish you all four new hacking years. our thanks goes to: nicolas steenlant christian pietsch dave sherohman dries moreels friedrich summann jakob voss johann rolschewski jonas smedegaard jörgen eriksson magnus enger maria hedberg mathias lösch najko jahn nicolas franck patrick hochstenbach petra kohorst robin sheat snorri briem upasana shukla vitali peil deutsche forschungsgemeinschaft for providing us the travel funds lund university library , ghent university library and bielefeld university library to provide us a very welcome environment for open source collaboration. written by hochstenbach leave a comment posted in uncategorized june , catmandu chat on friday june : cest, we’ll provide a one hour introduction/demo into processing data with catmandu. if you are interested, join us on the event page: https://plus.google.com/hangouts/_/event/c jcknos egjlthk m btha o more instructions on the exact google hangout coordinates for this chat will follow on this web page at friday june : . to enter the chat session, a working version of the catmandu virtualbox needs to be running on your system: https://librecatproject.wordpress.com/get-catmandu/ written by hochstenbach leave a comment posted in events june , matching authors against viaf identities at ghent university library we enrich catalog records with viaf identities to enhance the search experience in the catalog. when searching for all the books about ‘chekov’ we want to match all name variants of this author. consult viaf http://viaf.org/viaf/ /#chekhov,_anton_pavlovich,_ - and you will see many of them. chekhov Čehov tsjechof txékhov etc any of the these names variants can be available in the catalog data if authority control is not in place (or not maintained). searching any of these names should result in results for all the variants. in the past it was a labor intensive, manual job for catalogers to maintain an authority file. using results from linked data fragments research by ruben verborgh (iminds) and the catmandu-rdf tools created by jakob voss (gbv) and rdf-ldf by patrick hochstenbach, ghent university started an experiment to automatically enrich authors with viaf identities. in this blog post we will report on the setup and results of this experiment which will also be reported at elag . context three ingredients are needed to create a web of data: a scalable way to produce data. the infrastructure to publish data. clients accessing the data and reusing them in new contexts. on the production site there doesn’t seem to be any problem creating huge datasets by libraries. any transformation of library data to linked data will quickly generate an enormous number of rdf triples. we see this in the size of public available datasets: ugent academic bibliography: . . triples libris catalog: . . triples gallica: . . triples dbpedia: . . triples viaf: . . triples europeana: . . triples the european library: . . . triples pubchem: . . . triples also for accessing data, from a consumers perspective the “easy” part seems to be covered. instead of thousands of apis available and many documents formats for any dataset, sparql and rdf provide the programmer a single protocol and document model. the claim of the linked data fragments researchers is that on the publication side, reliable queryable access to public linked data datasets largely remains problematic due to the low availability percentages of public sparql endpoints [ref]. this is confirmed by the study by researchers from pontificia universidad católica in chili and national university of ireland where more than half of the public sparql endpoints seem to be offline . days per month. this gives an availability rate of less than % [ref]. the source of this high rate of inavailability can be traced back to the service model of linked data where two extremes exists to publish data (see image below). from: http://www.slideshare.net/rubenverborgh/dbpedias-triple-pattern-fragments at one side, data dumps (or dereferencing of urls) can be made available which requires a simple http server and lots of processing power on the client side. at the other side, an open sparql endpoint can be provided which requires a lot of processing power (hence, hardware investment) on the serverside. with sparql endpoints, clients can demand the execution of arbitrarily complicated queries. furthermore, since each client requests unique, highly specific queries, regular caching mechanisms are ineffective, since they can only optimized for repeated identical requests. this situation can be compared with providing a database sql dump to endusers or open database connection on which any possible sql statement can be executed. to a lesser extent libraries are well aware of the different modes of operation between running oai-pmh services and z . /sru services. linked data fragment researchers provide a third way, triple pattern fragments, to publish data which tries to provide the best of both worlds: access to a full dump of datasets while providing a queryable and cachable interface. for more information on the scalability of this solution i refer to the report presented at the th international usewod workshop. the experiment viaf doesn’t provide a public sparql endpoint, but a complete dump of the data is available at http://viaf.org/viaf/data/. in our experiments we used the viaf (virtual international authority file), which is made available under the odc attribution license. from this dump we created a hdt database. hdt provides a very efficient format to compress rdf data while maintaining browser and search functionality. using command line tools rdf/xml, turtle and ntriples can be compressed into a hdt file with an index. this standalone file can be used to without the need of a database to query huge datasets. a viaf conversion to hdt results in a gb file and a gb index. using the linked data fragments server by ruben verborgh, available at https://github.com/linkeddatafragments/server.js, this hdt file can be published as a nodejs application. for a demonstration of this server visit the iminds experimental setup at: http://data.linkeddatafragments.org/viaf using triple pattern fragments a simple rest protocol is available to query this dataset. for instance it is possible to download the complete dataset using this query: $ curl -h "accept: text/turtle" http://data.linkeddatafragments.org/viaf if we only want the triples concerning chekhov (http://viaf.org/viaf/ ) we can provide a query parameter: $ curl -h "accept: text/turtle" http://data.linkeddatafragments.org/viaf?subject=http://viaf.org/viaf/ likewise, using the predicate and object query any combination of triples can be requested from the server. $ curl -h "accept: text/turtle" http://data.linkeddatafragments.org/viaf?object="chekhov" the memory requirements of this server are small enough to run a copy of the viaf database on a macbook air laptop with gb ram. using specialised triple pattern fragments clients, sparql queries can be executed against this server. for the catmandu project we created a perl client rdf::ldf which is integrated into catmandu-rdf. to request all triples from the endpoint use: $ catmandu convert rdf --url http://data.linkeddatafragments.org/viaf --sparql 'select * {?s ?p ?o}' or, only those triples that are about “chekhov”: $ catmandu convert rdf --url http://data.linkeddatafragments.org/viaf --sparql 'select * {?s ?p "chekhov"}' in the ghent university experiment a more direct approach was taken to match authors to viaf. first, as input a marc dump from the catalog is being streamed into a perl program using a catmandu iterator. then, we extract the and fields which contain $a (name) and $d (date) subfields. these two fields are combined in a search query, as if we would search: chekhov, anton pavlovich, - if there is exactly one hit in our local viaf copy, then the result is reported. a complete script to process marc files this way is available at a github gist. to run the program against a marc dump execute the import_viaf.pl command: $ ./import_viaf.pl --type usmarc file.mrc - l $$aedwards, everett eugene,$$d - http://viaf.org/viaf/ - l $$aclelland, marjorie bolton,$$d - http://viaf.org/viaf/ - l $$aschein, edgar h. - l $$akilbridge, maurice d.,$$d - http://viaf.org/viaf/ - l $$awiseman, frederick. - l $$amiller, wilhelm,$$d - http://viaf.org/viaf/ - l $$ahazlett, thomas c.,$$d - http://viaf.org/viaf/ [edit: - - an updated version of the code is available as a git project https://github.com/librecat/marc rdf ] all the authors in the marc dump will be exported. if there is exactly one single match against viaf it will be added to the author field. we ran this command for one night in a single thread against . authors containing a date and found . exact matches in viaf (= %). in a quite recent follow up of our experiments, we investigated how ldf clients can be used in a federated setup. when combining in the ldf algorithm the triples result from many ldf servers, one sparql query can be run over many machines. these results are demonstrated at the iminds demo site where a single sparql query can be executed over the combined viaf and dbpedia datasets. a perl implementation of this federated search is available in the latest version of rdf-ldf at github. we strongly believe in the success of this setup and the scalability of this solution as demonstrated by ruben verborgh at the usewod workshop. using linked data fragments a range of solutions are available to publish data on the web. from simple data dumps to a full sparql endpoint any service level can be provided given the resources available. for more than a half year dbpedia has been running an ldf server with . % availability on a cpu , gb ram amazon server with . million requests. scaling out, services such has the lod laundromat cleans . datasets and provides access to them using a single fat ldf server ( gb ram). for more information on the federated searches with linked data fragments visit the blog post of ruben verborgh at: http://ruben.verborgh.org/blog/ / / /federated-sparql-queries-in-your-browser/ written by hochstenbach leave a comment posted in advanced tagged with ldf, linked data, marc, perl, rdf, sparql, triple pattern fragments, viaf older posts recent posts catmandu . lpw : “contrarian perl” – tom hukins introducing filestores catmandu . metadata analysis at the command-line catmandu . parallel processing with catmandu catmandu . catmandu chat matching authors against viaf identities preprocessing catmandu fixes earthquake in kathmandu importing files from a hotfolder directory librecat/memento hackathon day : merry christmas! day : exporting rdf data with catmandu day : importing rdf data with catmandu day : marc to dublin core day : set up your own oai data service day : harvest data with oai-pmh day : index your data with elasticsearch day : store your data in mongodb day : working with csv and excel files day : processing marc with catmandu day : processing json data from webservices day : catmandu json paths day : introduction into catmandu day : editing text with nano day : grep, less and wc day : bash basics create a free website or blog at wordpress.com. catmandu create a free website or blog at wordpress.com. add your thoughts here... (optional) post to cancel privacy & cookies: this site uses cookies. by continuing to use this website, you agree to their use. to find out more, including how to control cookies, see here: cookie policy . . updated: / / * enhancement: marcvalidator -- added a new option to determine if a mnemonic record is too long * enhancement: command-line -- batch process switch has been added to the tasks processing function * bug fix: when doing first run, wizard not showing in some cases. * bug fix: rda helper -- $e processing * bug fix: rda helper -- punctuation in the $e * bug fix: marceditor -- page processing correct to handle invalid formatted data better * bug fix: marcvalidator fails when attempting to process a .mrk file from outside the marceditor * bug fix: linked data processing: when processing services with multiple redirects -- process may stop pre-maturely. (example: lc's id.loc.gov xx processing) * bug fix: edit field -- find fields with just spaces are trimmed, causing the field data to process improperly. * bug fix: rda helper will fail if ldr length is incorrect when attempting to determine character encoding * bug fix: command-line: error is thrown when using batch process switch. this has been corrected. * bug fix: marcengine: batch process function -- correction when dealing with tmp data . . updated: / / * enhancement: add/delete function: added the ability to add a second value when doing dedup processing. * enhancement: add/delete function (task): added the ability to add a second value when doing dedup processing. . . updated: / / * update: updated the marcedit installer to switch from the embedded powershell helper to a signed module. this was due to issues with some antivirus tools * update: updated oclc api process for oclc holdings codes with special characters. * update: updated oclc api due to a change in a url to be case-sensitive. * update: command-line program updates * update: marceditor closing code updates to remove redundant messages. * various bug fixes * ui updates related to new windows ui changes. . . updated: / / * bug fix: custom reports: when running the spine label template, and error would be generated when authors were missing. new data validate has been added to correct the issue. * enhancement: marcvalidator results window: updated the results window so that it loads faster. * enhancement: installer: i've updated the custom action (a powershell script) that runs prior to installation to ensure that no conflicting versions of marcedit are installed. the script has been signed to limit potential issues related to local group policies. . . updated: / / * enhancement: custom reports: added templates functionality with templates for spine label printing. * enhancement: validator: added warning for invalid utf- characters * enhancement: marceditor delete records: added option to remove records with invalid utf characters. * enhancement: marceditor edit subfields: added option to remove subfields with invalid utf characters. * enhancement: marceditor edit fields: added option to remove fields with invalid utf characters * bug fix: added some error trapping validation . . updated: / / * enhancement: updated saxon to . * enhancement: updated dotnetrdf . * enhancement: added error checking around tab delimited export * enhancement: added lite-weight validation to subfield/position element in export tab delimited export. . . updated: / / * bug fix: oai harvester: tool attempts to identify server type to set correct accept header. on some servers, accept header being set was malformed causing harvest to fail. * bug fix: oai harvester: cleaned up a hard to find issue where files were sometimes failing to transform to marc causing harvest to fail. * enhancement: marcmerge: added a shortcut to take the merged file into the marceditor directly from the results window. * enhancement: removed dependency to system.web within the core application. . . updated: / / * enhancement: ui changes on the main windows and marc tools window * update: network adaptor changes affecting assemblies * bug fix: marceditor: memory leak that occurs when opening .mrc files directly in the editor. * bug fix: marceditor: corrected classname collision issue that didn't affect marcedit, but could impact plugin writers. * enhancement: sru display updated to improve results display when the user works with non-marc data. * behavior change: validate headings -- check subfield a behavior has changed when working with x and fields. fields now utilize xx rules to test the name heading. * bug fix: validate headings -- not sure if this is related to a change at the source or a bug in the code, but variant updates were starting to result in multiple $a$a pairs. added validation code to ensure proper variant updates. . . updated: / / * behavior change: z . batch tool now creates a log -- this file is used when doing sru or xml conversions and logs if the process fails for debugging. * bug fix: ils integration option: added code to ensure that changes are refreshed within the interface. * behavior change: ils integration option: added code check so if the integrations.xml is missing, a template is generated. . . updated: / / * bug fix: lc changed their urls for the id.loc.gov. updated rules file and validate headings links. . . updated: / / * bug fix: build new field: when using a field without a subfield definition or grouping definition, an error was being generated. . . updated: / / * enhancement: build new field: enhanced the function to enable the development of multiple fields. . . updated: / / * enhancement: integration with task scheduler (including a template) * enhancement: performance improvement on save * enhancement: build new field - change isn't visible -- staging code for an upcoming change. . . updated: / / * bug fix: watcher folder -- updated the service timer. * enhancement: rules files updated . . updated: / / * enhancement: added latin yiddish transliteration * enhancement: updated installer pre-checks to address issues in some older versions of windows. . . updated: / / * enhancement: added bibframe marc translation to the marc tools. have enhanced the process so that it can work on single or multiple work/instance pairs. * enhancement: added the json processing functions to the batch processing tool. * enhancement: updated some language elements. . . updated: / / * enhancement: added serbian transliteration to the marceditor * enhancement: updated installer to provide a check to prevent installation mismatch and to accommodate a change in windows that seems to be forcing reflection unless specifically turned off (which is causing trouble with the automatic updater). . . updated: . . * enhancement: added classical greek transliteration * enhancement: updated the marccompare html template to provide entries to just show changed records. * bug fix: updated the ui that was preventing new ils integration entries from showing up in the ils integrations list. . . updated: . . * enhancement: added belorussian transliteration * update: transliteration optimizations. . . updated: . . * bug fix: marceditor: conditional replace: regex would sometimes not process due to a switch statement falling back to simple in-string. * enhancement: transliteration additions: russian, ukrainian, bulgarian * enhancement: transliteration dialog -- process entire file * enhancement: options: transliteration processing rules * behavior change: transliterations can now be run on a whole file and have rules applied. . . updated: . . * new feature: xml editor -- new editor added to marcedit to edit xml documents. * enhancement: options -- updates to the main window buttons * bug fix: options -- validation needed corrected * behavior change -- main window, tools menu -- consolidated some menu entries * update: dependency updates . . updated: / / * update: dependency updates * enhancement: marceditor: show line numbers. . . updated: / / * bug fix: clustering tools -- the clustering tool wasn't working because a dependency was missing. this has been corrected. . . updated: / / * enhancement: marceditor (edit shortcuts): insert missing ldr function * enhancement: command-line tool - updated the printout functions to make silent more silent. * behavior change: dedup tool -- updated the dedup tool when run outside of the marceditor so that it doesn't require multiple files. * update: task manager debugger -- updated ui slightly * update: preferences: small updates with the preferences ui * update: xml profile wizard -- updated the text to make sense. * bug fix: classify tool -- fields over get an extra added to the field. this has been fix. . . updated: / / * enhancement: updated the ils integration -- first step in adding validation as part of the process. * enhancement: added application error log . . updated: / / * bug fix: marceditor task menu -- grouped task note showing * enhancement: rda helper -- $e should now be part of the task processing * bug fix: rda helper -- removed the space between the ", " when putting in the $e * enhancement: updated language files * enhancement: build links - updated rules file, small tweaks to abbreviation normalizations. . . updated: / / * bug fix: com object -- realized i hadn't changed the file version number. this may cause the com object to fail so i reset it. . . updated: / / * bug fix: marcvalidator -- finishing updates related to normalization. * enhancement: marcvalidator -- new options to set fields for testing dedups (or disabling dedups) * enhancement: linked data -- updated rules file * enhancement: various ui updates to bring it into compliance with . . * enhancement: all -- .net framework has been updated to . . . . updated: / / * bug fix: marcvalidator: ui issue -- messages weren't updating due to a thread boundary issue. this didn't affect functionality * bug fix/behavior change: normalization: when encountering data that cannot be normalized via the strict normalization functions, the tool will introduce a new line into the record so that it invalidates on processing. this can be confusing and hard to debug. switching to using the .net fallback function that allows normalization to fix problems that throw errors in strict/standard mode. * enhancement: preferences: new preference available to auto update rules files. this allows changes to be made without updating software. . . updated: / / * enhancement: mengine -- added error checking around the xml streaming function. * enhancement: oclc_api component -- added some addition error checking * enhancement: oclc_api component -- added holdings limiter to search api * enhancement: updated remove isbd punctuation tables * enhancement: added remove isbd punctuation to the taks manager. * enhancement: updated the language around the normalization code. * enhancement: oclc downloader enhancements (allowing download by library holdings) * enhancement: linked data rules file update (adding normalization form to the translation) . . updated: / / * enhancement: generate lcsh -- automatically generate lcsh subjects utilizing an lcsh call number. * enhancement: replace all task -- added the ability to use files in task lists. . . updated: / / * bug fix: find/replace conditional -- specific conditionals could result in records dropping. this was related to the recent changes adding and/or to the conditional language. * enhancement: marcvalidator -- added links to the results pages to allow users to jump to specific records within the editor. this is a work in progress. . . updated: / / * bug fix: find/replace conditional - conditionals not including and/or/not were not being processed. this has been corrected. * enhancement: find/replace conditional - new keywords: or/and added to the conditional processing. * ui update: marc tools window was chopping text if font size or font type set spacing beyond specific window thresholds. * update: marcrules file updated * enhancement: marcvalidator -- safe array typing added to rules file processing to prevent formatting errors from causing a stopping error. * enhancement: when noautoupdate.txt is added to the program directory, in addition to disabling autoupdate, it will disable the manual check update menu entries. . . updated: / / * enhancement: oclc integration -- oclc integration has been transitioned to utilizing oauth . . this will require you to revalidate your oclc keys and likely have oclc update your keys. * enhancement: oclc integration -- all oclc and ils integrations have been moved to their own configuration area. * enhancement: integration export/import: new tool to allow for export and import of integration settings * enhancement: z . -- updated overwrite/append dialog * enhancement: z . -- updated ui window (sizable and scrollable) * ui fix: find all results -- corrected a label * enhancement: all housecleaning code has been moved out of start up and into an asynchronous thread. * enhancement: validate headings -- small tweak to ensure all files are being closed. * enhancement: marceditor: added more error checking and trapping to the save file process. . . updated: / / * enhancement: merge records tool: added support for multiple match points. use a pipe "|" between multiple points. * enhancement: merge records tool: allowing ldr to be overlaid. * enhancement: updated saxon and linked data components * bug fix: oclc api search: the publisher search wasn't resolving correctly because it wasn't seeing the selection. * bug fix: oclc api search: if the query failed due and the object returned null, and error could be thrown when the tool generated the debugging message in specific instances. * bug fix: bibframe functionality -- missing an updated xslt file in the current installer. * bug fix: marceditor: when restoring from a previous version - if the user didn't use save as, the restored version may not save back to the current file. this has been corrected. . . updated: / / * bug fix: add/delete - found that in cases where the config value wasn't defined -- this would flag the new filter sort on insertion (shouldn't have). this would result in the field being inserted in an unexpected position and require a sort to correct. * bug fix: delimited text translator - updated the auto-generate arguments code. when using this, blank cells could sometimes be skipped in correctly. the work around had been to have stand-in data in the cells. this has been corrected. . . updated: / / *** please note -- . . may not select the correct auto update installer if you are using the *** admin installation prior to . . . you may need to download the update directly from *** https://marcedit.reeset.net * bug fix: add/delete field function -- when using add if not present, and error would occur. this was related to the new filter sort on field insert. . . updated: / / *** please note -- . . may not select the correct auto update installer if you are using the *** admin installation. you may need to download the update directly from *** https://marcedit.reeset.net * enhancement: recover from backup settings * enhancement: delete record(s) command in marceditor * enhancement: add sort by command to task * enhancement: optional field insertion/sort rules * enhancement: z . /sru query expansions * enhancement: z . /sru auto convert to marc options * enhancement: copy task actions from global edit tools * enhancement: updated task copy clipboard * enhancement: replace function ui updates * enhancement: saxon.net, newtonsoft.json component updates * bug fix: batch processing tool not processing all xml data files * bug fix: updates to the command-line harvester options * bug fix: installer -- installation may not select correct user type for download. this should be corrected. . . updated: / / * update: z . /sru client -- on download, added a check to ensure that appended records are added using the proper mnemonic format. * bug fix: marceditor -- when no recent files are present in the list, and error would occur. it didn't impact functionality. * update: check urls -- allowed proxy option set in updates to apply across all items. * update: task window: replace all -- updated the interface since the replace button needs to be disabled since it's not applicable in this space. . . updated: / / * update: oai harvester -- added the ability to utilize a the system proxy while harvesting. * update: z . /sru client -- added the ability to utilize the system proxy while searching. * update: alma integration -- added the ability to utilize the system proxy while searching. * update: marc tools -- program will remember the character conversion selections for file types. * update: character conversion -- updated code to ensure ldr is always correct, even if ldr encoding is incorrect prior to processing. * marceditor -- powershell integration. * marceditor -- recovery point function * bug fix: custom report error being generated while files are in use. * enhancement: installer -- new installer is being used to simplify build process and dependency integration. * enhancement: installer -- new installer will automatically build native images when the computer is idol after update. this will improve speed and memory management within the program. . . updated: / / * update: tab delimited export -- support for the xx syntax, i.e. -- use xx to extract all - fields. * update: linked data rules file -- changes to update viaf and fast resolution to json * update: linked data framework -- expanding the regular expression processing rules to pattern matching * update: json component -- version upgrade. * update: saxon component -- version upgrade * update: com version change. . . updated: / / * bug fix: edit / : if wordwrap is enabled, the function may not work. the tool will disable wordwrap automatically if this function is used. * bug fix: sru/z . general cleanup ** fixed a batch download problem ** fixed an issue with alma integration from batch ** fixed an encoding communication issue * update: gui -- updated some gui items around the program where text was being truncated if fonts were scaled. * update: console -- added some error checking around the file verification. . . updated: / / * bug fix: custom reports: gui issue in some cases where fonts are scaled. * enhancement: z . /sru: updating 'get next' link when working with multiple resources. . . updated: / / * bug fix: alma integration - corrected download of batch searches with alma * bug fix: alma integration - corrected download of data from context menus. . . updated: / / * enhancement: custom reports - new custom reporting feature enabling user defined reporting structures. * bug fix: com object -- when using a streaming function, there are cases where the temporary file creation will for internal processing may be null. * enhancement: sru query -- updated the sru processing to better recognize marcxml data. * enhancement: sru query -- added code to correct most invalid utf characters. . . updated: / / * enhancement: marceditor global edit date function: new formats supported. see documentation for new supported types. * enhancement: extract selected records: updating find all in record to support regular expressions and exact work search * enhancement: batch process records: added new option to allow users to specify the output file type. * enhancement: options: new option to allow for normalizations to work differently when working with an paired set of data. * enhancement: marceditor transliterations: added option so if a field is provided, the output will include the format for an pair. * enhancement: oclc worldcat api: report has been updated to include the oclc record number if available. * bug fix: marceditor ux update: updated the cuttering menu title * bug fix: {lcub}/{rcub} fix: in some cases, this was catching data that shouldn't and causing records to not compile. * bug fix: marceditor find: after successful find alls, sometimes find would report an error. this should be corrected. . . updated: / / * bug fix: com object looks like i might not have updated all the necessary registry keys on the fall back object. * update: marceditor: edit / -- im missing a value in the gui, so updating as appropriate. * enhancement: oai batch processing allowing batch jobs to harvest raw metadata rather than only converting to marc * update: rda helper some work around the $e in the xx to ensure that its not duplicating another $e and maybe work on determining if punctuation needs corrected (this wont be perfect, but better than now) * update: json object refresh updating json dependencies * enhancement: exposing the xml => json function (this is an internal function i use for my research work) . . updated: / / * enhancement: custom report writer: new report writer to evaluate all fields within a record. . . updated: / / * bug fix: alma sru search -- had to update the profile reader to accommodate how and where alma's xml defines the record schema. * enhancement: custom reporting: create your own reports against your marc data. . . updated: / / * bug fix: z . -- when using dedup or preferred record download, if the resulting searches created a lot of duplicates, gaps could sometimes form in the result set. * enhancement: marceditor -- edit shortcuts -- added function that will automatically replace html entities to utf characters * enhancement: task manager -- added function to automatically replace html entities with utf characters * updated: linked data rdf components (improves performance) * updated: main window -- generate reports . . updated: / / * bug fix: oclc integration -- removed an old link to a depricated component * bug fix: marceditor -- autosave was initiating repaging when it shouldn't . . updated: / / * enhancement: marceditor -- updating editor to allow support for mixing manual and global edits. * enhancement: automatic update notifications -- added options to specify when prompt should occur * enhancement: watch folders -- added ftp/sftp support * enhancement: watch folders -- added new watcher actions added (join/marcxml) * enhancement: marc validator â�� added ldr trap when doing non-structure validation so records where an ldr doesnâ��t appear as the first field or appears in the middle of a record will generate a critical reported error. update the dedup records â�� lamda expression to allow for last record seems to have issues when there are imperfections in the data. * enhancement: z . â�� if database config is deleted â�� code will now put it back. * updated validator to clean up counting so blank lines donâ��t artificially raise the number of record numbers reported processed (donâ��t impact the actual log file though) . . * bug fix: marcengine -- corrects the process reseting the ldr * bug fix: marceditor -- / options were not working -- this has been corrected. * enhancement: marceditor -- / -- added an option to right click on the field and process it. * change note: marceditor -- / -- flexed where the cursor can be put for processing. this behavior could change in the future -- but i'll try to preserve this. . . * update: console/terminal program - build links -- you can start process as mrc or mrk and end as mrc or mrk * bug fix: console/terminal build links -- when processing with limited options, fields not processed may get cleared. * update: marcengine -- performance tuning * update: oclc downloader -- updated return messages to make results clearer. * bug fix: oclc downloader -- if using a number over billion, a parse error will occur. * update: oclc search -- when downloading local bibliographic data, updated return messages to the response is clearer. . . * enhancement: command-line clean up * enhancement: watcher folder initial implementation completed. * enhancement: marceditor -- improved page reading/processing * enhancement: preferences -- added option to enable legacy paging mode * enhancement: ui enhancements to support windows dark mode . . * update: classify about url update * update: added code to ensure that if a user deletes the last line on a page, the save will ensure that records are not merged together * update: added load/save settings to merge file * update: re-enabled the marc pop up when the ime isn't enabled. * update: ux fixes to prevent windows dark mode settings from overriding user themes. * general performance updates. . . * enhancement: task manager -- automatic unlock option if lock files get stuck * enhancement: task manager -- added option in the dropdown to rename task [mostly to address some common confusion] * enhancement: global unhandled exceptions handler -- small updates to pick up a few errors that occur deeper in the .net stack. * performance update: tweaking the task broker around regex evaluation . . * this is primarily a maintenance fix including fixes related to the .net framework. * enhancement: validate headings tools -- variant process will work with subjects. * enhancement: validate headings tools -- added fields (like the $e) will be carried over when variants are replaced. * bug fix: select extracted records -- when clicking the link to find fields missing a value, it doesn't find text due to the match case option. this is fixed. . . * enhancement/bug fix: marcedit startup performance improvements -- i spent a lot of time looking at what is included in the startup and pulling as much of the processing out of startup. first cold start will still be the slowest, but in testing, i'm getting about - % improvement. * enhancement: marceditor -- to ensure all changes were catch, i made the save process a little more noisy in . . . i've pulled this back. * enhancement/bug fix: select extracted marc records/delete extracted marc records -- when processing via file, the exact word match has never been enabled. turning this on. * enhancement: ui changes -- i've noticed in some setups, windows will try to display outside of the screen boundaries. i added a new function that evaluates the primary screen and sets startup appropriately. . . * bug fix: corrected error causing save problems in . . . * bug fix: extract selected: ui issue resolved. . . * behavior change: disabled line highlighting due to some performance issues. * enhancement: command-line tool: added batch process to character encoding * enhancement: watch folders -- paired down some of the notifications * enhancement: z . batch log -- pushed the zero results list forward * enhancement: linked data tool -- updated work id extraction * enhancement: oclc integration -- search includes limiters * enhancement: extract selected records: add match case option * bug fix: highlighting function and mnemonic popup was interrupting the ime replacements for some languages. disabling both till i can figure out why ime functions are being interrupted. . . * enhancement: folder watcher -- testing version of the new marcedit watcher service * bug fix: file garbage collection -- under very specific conditions (which i cannot always recreate), it appears garbage collection is happening on temporary files before it should. * enhancement: thread management improvements * enhancement: implemented .net jit multi-core performance profiler -- this should allow startup to get faster as .net is updated. * enhancement: performance improvement allowing jump to records to move faster within a page. * bug fix: find/replace: when using find and making edits on a file edited through a global update, the special undo function wasn't be updated to reflect user changes via manual edit through find. the program now forces repagination if an edit is detected. * enhancement: program exit -- added sweeper to remove temporary files with a last write date of greater than days. * update: dedup records -- i came across a record where the control number was so large, it expanded beyond the current control number scope (which max around billion. expanded the function so current size can now be , , , , , , . . . * bug fix: oclc integration profiles -- when switching profiles after being established, sometimes profiles wouldn't clear or reset causing data to save incorrectly * enhancement: oclc integration -- expanded debugging information * enhancement: rda helper -- additional option to automatically add $eauthor to the field. * enhancement: marceditor -- experimental option to normalize isbd punctuation * bug fix: task manager -- control flow was stopping when working with small task sets. * enhancement: main window -- button will show if no unicode font is present . . * enhancement: marceditor -- updated program to recognize remaining supplimental plane unicode characters * enhancement: character conversion -- this is more of a restoration of a behavior from marcedit , when converting characters, required you to validate mnemonic data before doing character conversions because unvalidated data was causing issues. i've looped validation into the conversion process, and included mnemonic validation functions to ensure format is correct prior to processing characters. * bug fix: task management -- moving tasks up or down fails when the task isn't in a group. * bug fix: oclc integration -- for whatever reason, some windows instances were not updating the dependencies. i flushed version numbers; this should correct the issue. * enhancement: folder watching functionality -- you will see these options appearing in the preferences and watching functionality has been embedded into the application. this is disabled currently. process will be open for testing after the next update. * behavior change: marceditor - field edit. when editing control fields, you had to include data to find and replace. if find is blank, it will now replace data regardless of content. this is inline with the edit subfield function behavior. * bug fix: marceditor - line highlighting -- line height was correct on large fonts. this should be correct (or at least, much better). . . * new marcengine options ** replace smart quotes or typesetters marks with dumb characters. the characters addressed: *** \u , en dash *** \u , em dash *** \u , horizontal bar *** \u , double low line *** \u , left single quotation mark *** \u , right single quotation mark *** \u a, single low- quotation mark *** \u b, single high-reversed- quotation mark *** \u c, left double quotation mark *** \u d, right double quotation mark *** \u e, double low- quotation mark *** \u , horizontal ellipsis *** \u , prime *** \u , double prime ** new marcengine option to fix incorrect mnemonics on compile that incorrectly have {lcub}[mnemonic value]{rcub} syntax * new debug messages being surfaced from the engine if compilation fails. most importantly, the tool will try to provide a record number of the failure. * oclc api new debugging messages and a new validation option to ensure users have access to the registry api. this is now required, and im running into folks that are having trouble generating authentication tokens because their keys only have access to search and metadata. * oclc api more debugging information captured. when things do fail, hopefully the additional debugging information will help identify where that is occurring. ** (*) com updates *** new properties: **** correctsmartquotes [(default)true|false] **** correctmaskedmnemonics [(default)true|false] * (*) file associations ** reset button added this will reset file associations back to install (if you have correct permissions). it does the following: *** removes all user defined extensions *** removes older marcedit values coded in the hkroot registry key (if you have permission) *** updates the hk_local_machine [on marcedit admin install] or hk_current_user [on marcedit user install] values to reset .mrc, .mrk, .mrk associations ** more debugging messages available ** new runs as administrator button added to the upper right if the installation is an admin install, and thus, needs admin permissions to reset .mrc, .mrk, .mrk extensions * performance update the jit compiler has a profiler which enables the program to run faster on most current generation systems after first run. * new knowledge-base topics new topics for: ** resetting file associations ** necessary permissions from oclc for oclc integration (since these changes july ) * command-line tool ** updating command list help command to include supported switches for: *** marccount provides count of # of records in either a .mrc or .mrk file *** fieldcount generates the defined field_count report *** help pages outputs usage and manual information for specific command **** example: cmarcedit.exe -split -help * bug fix: delimited text translator -- updated code so formulas that result in numbers process correctly . . * bug fix: extract/delete selected records: you can now select multiple items when sorted * update: xml processing now can use the xsl:strip-whitespace element -- even when compiled. * enhancement: build uris -- function now can process excel and tab delimited files, in addition to marc files. . . * enhancement: task manager -- introduction of control flow techniques. * bug fix: task editor -- if editing a multi-line replacement, the options wouldn't read correctly if the action is edited. * enhancement: bibframe testbed -- bibframe xslt files updated. * enhancement: saxon components updates. . . * bug fix: delimited text translator -- when processing xls, data is duplicated. this has been fixed. * bug fix: oclc integration -- a change to the oclc api has broken marcedit's oclc integration. the code has been updated, though the change on the oclc side requires that users have access to the search api, the metadata api, and the registry api (used to find registry ids and oclcholdings codes). * bug fix: oclc integration: due to a change to the api, the get locations link in the integration was broken. this has been corrected. * enhancement: oclc integration: marcedit has always required that users enter their oclc code, but oclc is moving towards using registry ids for most operations. you can now enter either and marcedit will use the registry api to get the correct information based on data passed. * enhancement: update linked data/validate headings rules file . . * bug fix: edit subfield: when no options were selected and the replace text is used, the outcome isn't as expected. * bug fix: z . window: if the window coordinates are outside of the screen bounds, the screen isn't visible. this is corrected. * enhancement: marceditor ** preferences: a new preference has been added to the marceditor section that enables line highlighting ** marceditor: when line highlighting is enabled, the current line is highlighted to the user defined color. . . * update: task management ** create groups ** assign groups ** rename groups ** remove groups * bug fix: batch process marcxml=>marc -- when the native processing option is selected, this process will error. this is no longer the case. * update: installer -- bit/ bit installers have been updated to correct some com problems when installing a -bit syversion on a bit system. * update: installer -- improved selection of the installed version for automatic updates. this was specifically problematic when installing a -bit version on a -bit system. * bug fix: marceditor -- font size/type was sticking. this has been corrected. * update: language files have been refreshed * update: rules file updates (this are only picked up if your system allows for automatic rules file updates) * update: z . updates to handle some z . configurations. * update: uri updates within the program to point to https resources . . * update: rda helper -- updating sound recording processing * bug fix: marccompare -- updated template . . * bug fix: linked data processing in tasks -- when processing via a task, it's possible that if the task is older, you may end up using an older rules file. this has been corrected. . . * enhancement: transliteration beta -- inclusion of from and to arabic * enhancement: virtual keyboard * enhancement: us international keyboard standard support in the marceditor * enhancement: build new field -- updates to the pattern matching when using sub-functions * enhancement: build new field -- updated ui * enhancement: rdf helper -- updates to the processing * enhancement: dependencies -- components related to rdf processing have been updated . . * bug fix: linked data processing -- a recent change was preventing the xx data from getting a url. * bug fix: edit subfield processing â�� when working with the tool, data exposed through the regex is all data from the subfield code to the end of the subfield. however, for normal find and replace, the subfield code shouldnâ��t be visible. iâ��ve found that it is. this will be corrected so that regex will continue to see the subfield code and all subfield data, but the find/replace non-regex options will see data minus the subfield (as intended). * bug fix: oai harvester â�� when harvesting data using the new getrecord batch process, if an item fails, the process stops. this shouldnâ��t be the case â�� it should log the file and move forward. * bug fix: task editor â�� dynamic task list creation â�� a debug message didnâ��t get turned off with the last update. . . * enhancement: task editor batch edit changes which includes: ** batch delete ** batch move ** dynamic task list creation * enhancement: batch z . query -- add multiple get record ids to a file and the program will batch query them. . . * bug fix: replace all function: multi-line replace option stopped working after last update due to changes to expand conditional processing. this should be corrected. * enhancement: task editor: ability to delete batch task actions. * enhancement: task editor: first stage of code added to start adding some new batch functionality to task editor management. . . * bug fix: delimited text translator option -- fix bug in certain instances of trying to run legacy settings * enhancement: delimited text translator -- support for forumlas in excel * enhancement: add/delete field tool -- added new option to add field if a field is not present. . . * enhancement: validator -- added ability to provide different outputs * enhancement: validator -- if you add userdefined -- you can specify your own data for output * enhancement: delimited text translator -- new excel processing component. the tool no longer uses excel directly. . . * share settings ** enhancement: share integration settings -- allow users to share ils integration and oclc integration settings as a profile * enhancement: oai token handling: finding some servers are requiring special encoding. i've added enhanced logic to the error processing/retry. * enhancement: tab delimited tool can be resized horizontally (grow only) * enhancement: new program icon (cleaned up) . . * enhancement: xml profiling wizard: json profiling has been added to the wizard. * enhancement: oai harvester: added the ability to modify the user agent string to bypass bepress filtering. . . * linked data work ** enhancement: dbpedia collection profiles completed ** enhancement: new collection options added to rules file -- regex and headers ** enhancement: updated rules file ** enhancement: atom pub support added to marceditor ** enhancement: sru component -- updated to determine main entry by record type. ** enhancement: options: added atompub option. . . * linked data work ** enhancement: thread pool has been expanded to ** enhancement: new collections for dbpedia and isni added. ** enhancement: viaf process shifted to json, and updated to capture forms of names. * task processing ** enhancement: logging has been added so that when a task fails because the format is incorrect (user adds incorrect data) â�� the information is logged. in previous versions, these kinds of errors were suppressed and hidden * z /sru ** enhancement: updated the sru properties window ** enhancement: update z properties so batch results are logged (you can set the path â�� otherwise they are saved to userpath\documents\marcedit\batch_logs\file.txt * enhancement: new file extension (mrcx) for marcxml (apparently this is a real mime-type) . . * bug fix: marceditor: when working with large files, the tool would lose track of the last modified file. this could result in changes being lost. * enhancement: verify urls: you now have threading options...default threads are set to . * enhancement: verify urls: updated the report format. * enhancement: build links: this service is now threaded. this should result in noticeable speed improvements. * enhancement: replace function: when working with external files, the tool now filters away the bom. * enhancement: task management: users can now turn off the task broker evaluation and fall back to the old processing method. * enhancement: regular expression store -- function has been enhanced significantly. . . * bug fix: swap field validation error corrected * enhancement: when closing marcedit, if the editor is open, it flags on close so you don't lose data * bug fix/enhancement: task broker -- couple changes to how control data is processed to support older formatted tasks. * behavior change: extract selected records -- updated the process so i believe it will be faster. . . * bug fix: linked data/validate headings: corrected json binding. . . * update: updated installer that i believe will correct instances where the json object wasn't updated on install causing issues with the validate headings tool. * update: updated validator results window so new lines are operating system away so copy and paste work better * update: installer -- i've changed the titles so that it's easy to tell which version of marcedit is installed . . * update: saxon component update -- saxon bug fixes and enhancements * update: rdf components * update: json components * update: marcengine -- due to the changes around the dependences, the marcedit marcengine had to be updated as well. * enhancement: clustering support for non-marc data * enhancement: improved file handling when processing global updates in the marceditor * bug fix: linked data gui editor * gui enhancements * install wizard: adding normalization enforcement to the gui * enhancement: updating the unhandled error trapper. should prevent the program from shutting down on most crashes * bug fix: updated the automatic save tool editor * update: installer clean up * bug fix: replace all -- using the conditional has element may cause the process to match too broadly. . . * enhancement: marceditor - added an expression store, or a place to store any text data. * enhancement: additional updates related to ordinal case. . . * enhancement: ordinal case match in the find/find all window * update: build links: generate field moved from the $ to $ to match pcc recommendations * enhancement: clustering tools -- added fingerprint tokenization to the toolset. . . * enhancement: oclc downloader -- continues processing if encountering an bad oclc number -- reports bad elements. * bug fix: marcvalidator -- fixes the validator link from within the marceditor under the reports menu * enhancment: console program - the xml switch will test file type to allow the consolve version to work more like the gui. * enhancement: ui changes. . . * update: unimarc xslt -- updated to include isbn conversion * update: marceditor -- took a shot at updating the editor to make the automatic word selection like previous versions of windows. * update: xml conversion -- when working in the marceditor, the exclude or include namespaces wasn't being respect (namespaces were always included). * update: ui updates to clean up layout . . * bug fix: marceditor ui -- changed shortcut codes from editing functions as they were overlapping the constant data keys * enhancement: z . options -- added option so that users can set a global download file * enhancement: z . options -- ui changes in the preferences window * enhancement: z . options -- adding the global save file required updating the prompt/overwrite settings * bug fix: z . options -- found that in some cases, the session variables were being dropped, which resulted in multiple data prompts * bug fix: add fast headings -- updates required to ensure data coming from classify api are normalized according to new data normalizaiton rules. . . * bug fix: hexeditor throws an error when find is run more than once. * bug fix: task manager: exporting executable tasks works better with paths that include spaces. * bug fix: ui fix - find/replace window -- when themes are present, highlighting now works better. * bug fix: validator -- when a subfield group defines length, but not all the fields are defined, a trapped exception for that record may occur. this is corrected. . . * bug fix/enhancement: marceditor/tasks (normalizations): when running tasks which process by file, the entire file may not be normalized. this will mean some values may be missed on edit, if normalization values are different. this has been updated. . . * enhancement: marceditor - new setting added to provide unicode normalization standardization * enhancement: hex editor -- added into the marceditor * enhancement: task manager -- added the ability to export tasks as executables. * enhancement: executables and libraries have been code-signed (this is in addition to the msi) * miscellaneous performance changes . . * bug fix: export settings: export was capturing both marcedit .x and marcedit .x data. * enhancement: task management: added some continued refinements to improve speed and processing * bug fix: oclc integration: corrected an issue occuring when trying to post bib records using previous profiles. * enhancement: linked data xml rules file editor completed * enhancement: linked data framework: formal support for local linked data triple stores for resolution . . * enhancement: marc tools -- when a zip file is found as part of a web download, the program automatically extracts the file from the zip. this has been added to the basic functionality (if you point to a zip file, it will try to extract the data from the zip file) * bug fix: rda helper -- when editing from the task manager, the task options are not displayed. rather, the last run state is displayed. * behavior change: rda helper: within a task, the rda helper shouldn't be run multiple times. this restriction has been lifted. * bug fix: rda helper: if the user selects the $b, but doesn't select a language, the default text isn't captured (but is blanked). * behavior change: rda helper: per an earlier request, the and were loosely paired together. since these remained separate options on the form, this was confusing. i've severed their relationship and left construction to the user. . . * bug fix: dedup tool -- null error occurs under certain circumstances when cross-references match across multiple records. * bug fix: copy field -- match position was being evaluated when it should have been. the side affect of this was some operations wouldn't run. * bug fix: ui changes -- i've noticed a number of fields seemed to thin (i.e., bottoms of 'g's or 'p's were being cut. this should be corrected. * bug fix: dedup field -- realized that i had missed the "on" prefix for larger oclc numbers. . . * bug fix: task broker -- updated the task broker to enable compatibility with the mono stack. this has been done to support the marcedit mac work * bug fix: oclc api code -- updated the oclc api code to complete the token conversion and make a couple changes to enable compatibility with the mono stack. this has been done to support the marcedit mac work. * bug fix: sparql browser -- updated the sparql processing code to set the default httpencoder. this was done to enable compatibility with the mono stack. this has been done to support the marcedit mac work. * bug fix: clustering library -- updates specifically to enable compatibility with the mono stack. this has been done to support the marcedit mac work. * bug fix: task broker -- updated some code in the parse editfield and parse subfield elements to ensure that all elements are run. * bug fix: task broker -- if an error is thrown (due to invalid format) in the edit field function, the task broker suppresses the error and skips processing other tasks. this has been changed so that the error is logged, and the other tasks continue. . . * bug fix: add/delete function (delete duplicate records): some items are missed when you have lots of duplicates across multiple field groups. * enhancements: updates to the ui for linux * enhancements: updates to clustering library for macos work * enhancements: updates to the marceditor engine for macos work . . * enhancement: clustering tools: added the ability to extract records via the clustering tooling * enhancement: clustering tools: added the ability to search within clusters * enhancement: linux build created * bug fix: clustering tools: numbering at the top wasn't always correct * bug fix: task manager: processing number count wouldn't reset when run * enhancement: task broker: various updates to improve performance and address some outlier formats * bug fix: find/replace task processing: task editor was incorrectly always check the conditional option. this shouldn't affect run, but it was messy. * enhancement: copy field: added a new field feature * enhancement: startup wizard -- added tools to simplify migration of data from marcedit to marcedit . . * bug fix(?): add/delete window -- i added a status message to this operation to create a template which will be added to other batch operations. i can't recreate the problem, but i'm getting some reports of errors related to accessing a disposed object. i believe what is happening, is that the object is being garbage collected and the order isn't regular (i.e., it's system dependent). i added some code to make this process regular. if my guess is correct, this should fix the problem. . . * enhancement: task broker -- added additional preprocessing functions to improve processing (specifically, not elements in the replace task, updates to the edit field) * enhancement: task broker -- updated the process that selects the by record or by file approach to utilize file size in selecting the record buffer to process. * enhancement: new option -- added an option to offer preview mode if the file is too large. * enhancement: results window -- added an option to turn on word wrapping within the window. * enhancement: main window -- more content allowed to be added to the most recent programs run list ** added a shortcut (ctrl+p) to immediately open the recent programs list * bug fix: constant data elements -- if the source file isn't present, and error is thrown rather than automatically generating a new file. this has been corrected. * update: task debugger -- updated the task debugger to make stepping through tasks easier. * bug fix: task processor -- commented tasks were sometimes running -- this has been corrected in both the marceditor and console program. * enhancement: status messages been added to many of the batch edit functions in the marceditor to provide more user feedback. * enhancement: added check so that if you use "x" in places where the tool allows for field select with "x" or "*", the selection is case insensitive (that has not been the default, though it's worked that way in marcedit but this was technically not a supported format * updated installer: .net requirements set to . as the minimum. this was done because there are a few problems when running only against . . . . * refinement: rda helper -- when selecting the , the ldr position changes to "i" regardless of if the is present ($e additions still only happen if a is present) * enhancement: ui feedback added to the extract select/delete selected windows. * bug fix: ui refresh has been added to the invert selections (this wasn't refreshing until the scroll bar was moved). * enhancement: task broker -- added preprocessor/optimization support for dealing with replace all tasks using conditional arguments * enhancement: task broker -- updates related to logging of data * enhancement: main window -- added most recently run programs list * enhancement: z . preferences; added save file options/behaviors * enhancement: z . -- added a managed databases option * enhancement: z . -- remembers if the options box is left open * bug fix: z . -- if a database isn't selected, the database queried won't be filled in after selection till restarting the z . client. * bug fix: z . -- an odd error wasn't being trapped causing the tool to crash * enhancement: clustering; new option added to extract records from a selected cluster; rather than editing a cluster. * bug fix: enhanced log file management ** enabled field selection ** fixed bug when file was not selected ** corrected parser due to change in enhanced log format. ** updated log file viewing to enable viewing of large log files without causing the application to lockup . . * bug fix: task broker -- updates related to the edit subfield preprocessing code. the default behavior has shifted from not run a task to run a task if unclear. * ui changes in the z . client * enhancements: task broker -- code optimizations -- run by record approach now processing at tasks with operations or greater. * enhancements: log file enhancements (more data added) . . * enhancement: delete field: dedup field data -- added new elements for debugging and logging. updated to expand functionality * behavior change: clustering tools: cluster tools have been updated, with new features set for this weekend. . . * bug fix: cluster data -- when working with data outside of the marceditor, the clustering tool throws an error. * bug fix: edit indicators - when in a task, the process and indicator tasks are stacked, the tool is processing data too quickly causing issues. this has been corrected. * bug fix: add/delete field: deduping fields; this includes artifacts in narrow cases. * enhancement: handful of ui clean up tasks . . * bug fix: build new field -- inside the task broker, the internal field type for one of the parameters was incorrect. * enhancement: z . window remembers last searched index * behavior change: marcengine marcxml => marc -- when reading empty elements, the tool will try to create valid marc fields. this can cause problems downstream, so this accommodation has been removed. * bug fix: copy field -- when using regular expressions, evaluation may result in operations not being run in the task by record method (tasks with over items) . . * enhancement: ui updates ** connected export tab delimited drop down box so all fields are available in dropdown ** open dialog not showing when clicking button in edit xml functions window ** updated the z . window -- text might cover the search box if too many databases are selected. ** updated oclc integration to add profiles *** updated all oclc windows to select profiles, not values * behavior change: edit indicator - if the format of the file is invalid, the function may eat the last line of a record. this will correct that. * enhancement: oclc api integration -- added support for affiliates * enhancement: oclc api integration -- support for oclc's access tokens * enhancement: performance update -- updating some internal loops in the marc engine -- testing shows a small, but noticeable speed gain. * bug fix: onix to marcxml transformation -- updated the transformation provided via the installer . . * bug fix: add/remove field: dedup field option -- an error would occur when the dedup field was left blank. this is directly related to the new functionality to allow preference in field order when deduping data. * enhancement: ease of access -- new option to enable form transparencies when a form owner is set. this is primarily used in the marceditor. example -- the add/delete field window can become transparent and allow the data underneath the form to be read * enhancement: added new sounds to the sounds folder . . * marcedit production release * update: connected updates to marcedit's updating tooling . . * bug fix: task management: adding case conversion statements weren't working * enhancement: com object -- new functions added * enhancement: marcengine: temp file management updated to reflect same management implemented in the task management . . * enhancement: edit shortcuts in the marceditor * bug fix: delete field: remove duplicate fields -- updated when handling multiple dedups across multiple field groups. * enhancement: z . custom field addition * enhancement: z . smartquote filter * enhancement: marc sql explorer updates * enhancement: replace all function: not keyword added to the preform if options * enhancement: rda helper: added code to allow for custom $b data * enhancement: edit shortcuts -- generate digit isbn * enhancement: script maker -- updating for the new com object * enhancement: support for plugins re-enabled. . . * bug fix -- ui issues with the inputbox dialog, losing string data when fonts are larger. updated inputbox dialog and connected it to the form sizing/font sizing event pump * bug fix-- ui issues with the inputbox dialog. this dialog is not connected to the theming event pump. it has now been connected. the only content that will now not be themed, are messages generated by the operating system. * bug fix -- logging enhancements -- process by record changed the way records are counted internally, which caused issues with the log management. updates to ensure that this works correctly. [way i could do this -- add a value to meedit that is static (so able to be pulled by the various functions -- and is set each time the value is called into the function. it is only set when run as a file. that would allow me to set record processing number -- remember, number starts at zero (i thinkâ�¦need to confirm that) * bug fix: select/delete extracted records -- sorting by checkboxes in the virtual list has been corrected * bug fix: delete field: remove dedups -- when using a delimited field list, the order was not always preserved. this should be corrected. . . * bug fix: edit subfield: delete duplicate in the new task process for by record was functioning incorrectly * enhancement: delete field - added new functionality so that delete by duplicate allows preference to be set by the tag list * bug fix: task editor: edit subfield task wouldn't select delete duplicates when editing the task (though data was saved correctly) * enhancement: check urls -- http status codes for waiting have been added * enhancement: saxon: saxon library has been updated * enhancement: extract/delete selected marc records -- list loads much faster (using a virtual list). . . * bug fix: edit : removed debugging message * bug fix: task editor: edit field not selecting the match case when editing the task. * enhancement: delete field: delete by position was requires a very specific format that wasn't very intuitive. updated to just make this a comma delimited list (when deleting field groups). * enhancement: linked data platform -- added support for local rdf endpoints * enhancement: linked data platform -- add code to support editing of the linked data rules file. this will be public on the next build. * bug fix: swap field function: in a task, this process wasn't completing. * bug fix: linked data platform: updated to recognize oclc numbers with the "on" prefix. . . * bug fix: task broker: edit subfield task switcher was throwing an error if the field defined uses the wildcard syntax of ** and finds nothing if using xx. * enhancement: rda helper â�� abbreviations syncing file is now active. * enhancement: sparql browser â�� local rdf files can be used as sparql endpoints. . . * bug fix: task broker: replace function task switcher is over-processing when working in the by record mode and under processing when searching regular expressions in the by record mode. * bug fix: task processing: when running a task by record, the count was off (i'd changed the buffer for optimized work. . . * bug fix: task broker: indicator tasks with multiple wildcards were being missed. . . * bug fix/behavior change: edit subfield (remove data): removed convenience feature that automatically would force regular expressions to ignore case. * bug fix: task processing: process task by record -- character encoding/file encoding switching correction (results in some task elements not processing when they should) * bug fix: task broker -- replace all task wasn't processing if the conditional option was selected. . . * enhancement: new hex editor tool added to marcedit (replacing marc spy). this tool is more robust * enhancement: theme builder completed * enhancement: preferences updated to support the theme builder * enhancement: clustering tooling: dice coefficient algorithm added * enhancement: enabled sorting of results by a-z and - . * enhancement: accessibility -- keyboard shortcuts implemented for the main window and marc tools window * enhancement: batch editing library optimizations (specifically related to logging changes, counting changes, and regular expression processing) * bug fix: copy field data - processing error in task processing when the task broker shifts record processing to by record. * enhancement: edit indicators: count returns # of actual changes (not number of indicators potentially touched) * enhancement: delimited text translator has been updated and themes enabled. * enhancement: sparql browser has been refreshed with examples and help * enhancement: linked data platform enhancement to provide functions to read and write to local triple stores. * behavior change: old chm help file has been removed in favor of linking out to the website. . . * bug fix: export tab delimited records (open file and save file buttons not working) * enhancement: xml crosswalk wizard -- enabled root element processing * bug fix: xml crosswalk wizard -- some elements not being picked up, all descendants should now be accounted for * bug fix: batch process function - file collisions in subfolders would result in overwritten results. * enhancement: batch processing function: task processing uses the new task manager * enhancement: batch processing function: tasks can be processed as subdirectories . . * installer change: i bumped the minimum requirements to .net . . on the previous installer build; after some research, it appears . will suffice. * enhancement: support regular expression option with the delete field/duplication field data question. * enhancement: support for multiple field processing using the delete field functions * bug fix: sru search not picking up custom entries when record number or keyword search selected. * enhancement: preview marc file link added back to right click on .mrc file . . * enhancement: all processes: updated temp file management * bug fix: plugin manager failing because it's missing a column for marcedit version (note, none of the current plugins will work with marcedit ) * enhancement: added new languages for croatian, estonian, indonesian, hungarian, and vietnamese * enhancement: offer download into private font collection the noto fonts when no unicode font is present. this will make the fonts *only* available for use with marcedit. * when editing a task list -- could the list not refresh? this occurs when you have a theme defined. * bug fix: update all the z . /sru databases (specifically -- the lc databases point to the old voyager endpoint that i believe is turned off) * bug fix: working with saxon, xslt transformations that link to files with spaces or special characters fail * bug fix: clustering tool -- selecting a top level cluster would include # of records in the cluster, not just the data to copy * enhancement: clustering tools -- add to the main window as a stand-alone tool * bug fix: on install, the file types are not associated * enhancement: new font's dialog to support private fonts collections * bug fix: fonts not sticking when using the startup wizard * enhancement: added unicode font download to help * bug fix: z . /sru downloads were only downloading as .mrk formatted data, not as binary marc. the tool has been updated to select download type by extension. * enhancement: updated the icon a bit so that itâ��s not so transparent on the desktop. * enhancement: both the and bit user installers have been signed. other binaries are not signed yet. . . * bug fix: main window -- only able to size window larger, not smaller * bug fix: task processing: tasks that collected user input stop in debugging * bug fix: task processing: deletion menu not functioning when working with the ils post and pre-processing steps * bug fix: task processing: adding a subfield edit task throws an error on save * bug fix: task processing: information wasn't making it to the log file * enhancement: task processing: benchmarking information added to the output results * enhancement: task processing: ui freezing minimized * enhancement: task processing: tasks and files are being evaluated by the broker and selecting the best task processing methodology for performance [simple criteria -- task with more than (is it or ?) items will move to the by record option, tasks under go to by file] * enhancement: task processing: optimized loops and string processing in the code. * enhancement: delimited text translator: defined generic is replaced if an is defined in a spreadsheet * bug fix: translations: some placeholder fields were translated that shouldn't have been resulting in odd data (like undefined on the main screen). * enhancement: setup wizard -- proactively purges some marcedit settings that are not applicable or may cause issues in marcedit * bug fix: setup wizard -- couple labels were truncated * bug fix: setup wizard: some paths not corrected when importing data from marcedit * enhancement: translations: new translations added (czech, bulgarian, catalan, danish, bangla) * enhancement: quick links -- couple more links enabled * bug fix: marcsplit -- splitting by # of files causes and error * bug fix: z . /sru: imported lists from marcedit might not merge correctly causing an error * enhancement: marcengine: general optimizations * bug fix: task processing: updated temp file management * enhancement: all processes: updated temp file management * enhancement: linked data: new profiled databases * enhancement: enabling automated update functionality (update checking, not automatic update installation yet) * enhancement: support for unc paths when setting network locations with tasks * enhancement: task processing -- preprocessing of tasks in process by records. . . * initial alpha release leadership group dlf about overview calendar faq foundational principles leadership strategic plan membership join the ndsa member orientation members groups overview interest groups content infrastructure standards and practices active working groups communications and publications conference program innovation awards levels of preservation ndsa agenda publications overview levels of preservation ndsa agenda osf repository conference digipres conference past digipres conferences news leadership group leadership team the ndsa leadership consists of the elected coordinating committee, and the chairs/co-chairs of the interest and working groups, and a representative from the host organization. together, the coordinating committee and the interest and working group chairs work to articulate a long-term, strategic vision for ndsa. the leadership group meets once a month online and in person once during the digital preservation annual conference. select activities of the leadership group include: approving new ndsa member applications. creating and reviewing ndsa publications (e.g. the ndsa agenda). evaluating the effectiveness of the interest and working groups and providing guidance and assistance to the group chairs as appropriate. this can include recommending the creation, consolidation, or disbanding of interest or working groups and working to eliminate unnecessary duplication of effort. coordinating with dlf on administrative management of the ndsa. working with international partners to extend digital preservation advocacy and awareness work with parent organization to plan the digipres conference creating and reviewing the annual roadmap for the ndsa for items that need to be voted on, the interest and working group chairs are considered to be ex officio members; they do not vote and their presence is not counted as part of a quorum. only the elected cc members may vote. coordinating committee members may also pursue becoming a member of the coordinating committee (cc). members of the coordinating committee serve a three-year elected position that works with the chairs of the interest and working groups on the strategic goals of the ndsa. details about the purpose, charge, and expectations of the committee are recorded in the coordinating committee information document. the primary responsibilities and expectations of individual cc members include: approving and/or participating in interest groups and working groups as needed or required. actively promoting and representing the work of the ndsa in their own professional communities. actively engaging in the ongoing work of the cc, with an expected % attendance record for monthly cc meetings. communicating clearly, respectfully, and in a timely fashion to support active participation by all members of the project team, especially when leading or participating in cc projects. the ndsa leadership group is comprised of the coordinating committee, the interest group and working group co-chairs, and the host organization representatives, which in collaboration provide strategic leadership for the organization. committee members serve staggered terms of three years. host organization ndsa derives its administrative and financial support through a “host organization”. the host organization: provides a membership mechanism, coordination, and support for the ndsa organization. provides outreach and communication frameworks to ndsa leadership, which may be used to inform the broader digital preservation community about ndsa activities, events, and products. represents the ndsa organization with a distinct and branded web presence. supports the work of the ndsa coordinating committee and provides one voting member of the leadership in conjunction with elected members of the cc. supports and coordinates the execution of an annual ndsa conference. commits to a -year (renewable) term as ndsa host organization. coordinating committee members daniel noonan, chair dan ( st term, - ) is an associate professor and the digital preservation librarian for the ohio state university libraries (osul). reporting to the associate director for distinctive collections and digital programs, dan plays a key role in developing a trusted digital preservation ethos and infrastructure at osul. this position contributes strategy and expertise, and provides leadership through close collaboration with faculty, staff, and other leaders in osul’s digital programs, preservation and digitization, distinctive collections, content and access, archival description and access, and publishing and repository services groups. previously, he was osul’s electronic records/digital resources archivist and electronic records manager/archivist. simultaneously, dan was an adjunct faculty member for kent state university, teaching an archives foundations course. prior to joining osul, he was the supervisor for electronic records management for the state of new jersey, and the digital documents librarian for the new jersey institute of technology. dan has an extensive service record including co-chairing ndsa's levels of preservation revision work group, teaching for the society of american archivists' digital archives specialist program, and serving both as a faculty member ( - teaching digital strategies) and on the steering committee ( - ) of the archives leadership institute. stephen abrams stephen abrams ( st term, - ; innovation award working group co-chair) is head of digital preservation at the harvard library, with responsibility for policy, strategy, and innovation regarding long-term stewardship of harvard’s rich digital collections. he was project leader and editor for the iso pdf/a standard, project manager for the jhove and jhove format characterization systems, and principal investigator for the california state government web archive, cobweb collaborative collection development, and make data count data metric projects. his research interests are in cost and business models for sustainable digital library services, new modes of post-custodial curation, and metrics for evaluating digital preservation efficacy. mr. abrams was previously associate director of the uc curation center at the california digital library. he holds a ba in mathematics from boston university, an alm in the history of art and architecture from harvard university, and is pursuing a phd in information science from queensland university of technology. elizabeth england elizabeth england ( st term, - ) is a digital preservation specialist at the u.s. national archives and records administration, where she participates in strategic and operational initiatives and services for the preservation of born-digital and digitized records of the federal government. in addition, she teaches an introduction to digital preservation continuing education course for the university of wisconsin ischool. prior to joining nara, elizabeth was the digital archivist and a national digital stewardship resident (ndsr) at johns hopkins university. elizabeth currently serves on the ndsa communications and publications working group and has previously served on the ivy plus libraries consortium web collecting advisory group and ndsr advisory group. salwa ismail salwa ismail ( st term, - ) is the head of library technologies at georgetown university library. prior to joining georgetown university, she was the head of the digital library at fau. her portfolio includes library servers, systems, and applications; computing infrastructure; web services; digital initiatives and services; digital preservation; digital scholarship; and ils & discovery services. in , she was listed by e-campus news as “ leaders shaping the future of higher education”. in , she was named a library mover and shaker by library journal for being a digital driver. she is currently the chair of the dspace leadership group and in - was a mentor to a national digital stewardship residency (ndsr) program resident, where her project led to georgetown university library being selected as an ndsr host institution. she is very passionate about how libraries, through innovation of library technology and digital services, can play a role as agents of research and scholarship in institutions of higher education. she earned her b.s. in computer engineering and mba from florida atlantic university and her mslis degree from florida state university. she is currently a ph.d. candidate in the computational social science program at george mason university. courtney mumma courtney c. mumma ( st term, - ), is a an archivist and a librarian. she is the deputy director of the texas digital library consortium, a collective of university libraries working towards open, sustainable, and secure digital heritage and scholarly communications. she has over a decade of experience in open source software development and maintenance, infrastructure support and digital preservation good practice and education. jessica neal jessica c. neal ( st term, - ; digipres vice chair) is an archivist, records manager, and memory worker. she is currently the sterling a. brown archivist at williams college and records management program assistant at the massachusetts institute of technology. jes’ work centers archives, preservation, data management, and developing ethical frameworks to better steward digital collections and projects that specifically focus on black led and created social movements, oral histories, and literary history and culture. jes received her b.a. in african world studies from dillard university and her m.l.i.s in archival studies from the university of wisconsin-milwaukee. linda tadic linda tadic ( nd term, - ) is founder and ceo of digital bedrock, a managed digital preservation service provider that serves any type of organization, and even individuals. she has years’ experience in leading preservation, metadata, and digital production operations at organizations such as artstor, hbo, the media archives and peabody awards collection at the university of georgia, and the getty research institute. currently an adjunct professor in ucla’s graduate school of education and information studies teaching digital asset management, she was previously an adjunct professor in nyu’s moving image archiving and preservation program (courses in collection management and cataloging and metadata). she consults and lectures on digital asset management, audiovisual and digital preservation, metadata, and copyright, with clients as diverse as wnet/thirteen, the academy of motion picture arts and sciences, sbs (australia), dunhuang research academy (china), espn, and the missouri history museum. she is a founding member and former president of the association of moving image archivists (amia). nathan tallman, vice chair nathan tallman ( st term, - ) is digital preservation librarian at penn state university libraries where he coordinates policies, workflows and best practices to ensure the long-term preservation and accessibility of psu libraries’ born-digital and digitized collections. he also advises on equipment, infrastructure, and vendors for penn state digital content. nathan serves on the aptrust governing board and metaarchive cooperative steering committee. prior to his arrival at penn state, nathan was the digital content strategist at the university of cincinnati and associate archivist at the american jewish archives. interest and working group chairs brenda burk brenda burk (content interest group co-chair) joined clemson university libraries in as the head of special collections. as part of the clemson university libraries, the special collections and archives houses the university archives, records management, manuscript collections, and rare books. previously she was the philanthropic studies archivist at iupui university library and public records archivist at the wisconsin historical society. in her current position, she continues to build a premier research collection that supports the university and creates an environment encouraging scholarly inquiry, creative thinking, and lifelong learning. her research interests include information seeking behaviors of users, course-integrated instruction, public awareness and perceptions of archives. brenda graduated from the university of wisconsin-madison with a ba in history and a ma in library and information studies with an emphasis in archival administration. bradley daigle bradley daigle (levels of digital preservation co-chair) is content and strategic expert for the academic preservation trust and other external partnerships at the university of virginia library. he also works on copyright issues related to digital collections. currently he is also chair of the virginia heritage governance team. having been in the library profession for over fifteen years, he has published and presented on a wide range of topics including mass digitization, digital curation and stewardship, sustaining digital scholarship, intellectual property issues, mentoring in libraries, and digital preservation. in addition to his professional field, his research interests also include the history of the book, natural history, and early modern british literature. he received his ma in literature from the university of montreal and an mls from catholic university. felicity dykas felicity dykas (standards and practices interest group co-chair) is head of digital services at the university of missouri (mu). in this role she oversees the work of mospace, the university of missouri institutional repository, the mu digital library, and digitization work. prior roles include cataloger and head of the catalog department. her current work interests include providing online access to information resources (now and into the future) by focusing on the development and use of international, national, and local standards for digital objects and metadata that enhances discoverability. collaborating with others on digital projects also is a priority. as a mantra, felicity frequently quotes the ifla library reference model user tasks: our goal is to help our users find, identify, select, obtain, and explore information resources. felicity is active in ala, with current membership on the association of library collections and technical services continuing education committee. carol kussmann carol kussmann (communications and publications working group chair) is the digital preservation analyst at the university of minnesota libraries. in this role, she works across many departments within the libraries, as well as outside the libraries including through the statewide minnesota digital library program. she addresses current and future requirements for the long-term preservation of electronic records in the areas of archives and special collections, information and data repositories, and journal publishing. as co-chair of the libraries electronic records task force her efforts focus on developing and implementing workflows for ingesting, processing, and providing access to incoming electronic materials that are part of the archives and special collections units. as an inaugural digital preservation outreach and education (dpoe) trainer, she works with minitex to provide digital preservation training in the region on a regular basis. after completing the initial implementation work for the council of state archivists’ (cosa) electronic records resource center she remains a member of cosa’s tools and resources subcommittee. other current activities include serving on the steering committee of the electronic records section of the society of american archivists (saa) and teaching digital archives specialist courses for saa. eric lopatin eric lopatin (infrastructure interest group co-chair) is product manager for the california digital library's digital preservation initiatives, including the merritt repository that preserves library special collection content from libraries across all ten university of california campuses, as well as escholarship publications, etds, and datasets submitted to dryad from organizations worldwide. in this role, he leads the product development of merritt and its integration with cdl systems, while also directly supporting campus-specific preservation efforts. through ongoing uc-systemwide initiatives, he helps promote the adoption of digital preservation best practices and associated technologies. prior to joining cdl, eric worked at the public library of science where he was product owner for a development team focused on bolstering editorial process efficiency across journal operations. his recent work in open access publishing, as well as a string of years spent at adobe systems enabling cross-application workflows and shared technology, have all contributed to his interests in the realms of preservation, publishing and software development. krista oldham krista oldham (innovations award working group co-chair) is the university archivist at clemson university, where her responsibilities include overseeing the acquisition, description, and preservation of university records, as well as supporting and promoting their use. additionally, krista is responsible for assisting in developing and managing a comprehensive, institution-wide records management program. she earned a m.i.s. from the university of tennessee, knoxville and earned both a m.a. in history and a b.a. in history from the university of arkansas, fayetteville. prior to starting her position at clemson, krista worked at haverford college as the college archivist/records manager for quaker and special collections and at the university of arkansas, fayetteville special collections as the senior archivist and the senior archives manager. in addition to her archival work, krista served as co-director of the arkansas delta oral history project, an initiative led by the endowed brown chair in english literacy. she is a co-author of the arkansas delta oral history project: culture, place, and authenticity, which was published in by syracuse university press. tricia patterson tricia patterson ( digipres chair) is a digital preservation analyst at harvard library, where she champions communication with the future by ensuring long-term stewardship and usability of harvard’s digital historical assets. centrally positioned, she supports programmatic activities for the digital repository, web and email archiving, digital forensics, and other related enterprises across the library. prior to joining harvard university, she was a national digital stewardship resident (ndsr) at mit libraries, where she researched and documented digital preservation workflows. tricia has served as a coordinator for the saa research forum, an inaugural ndsr advisory group member, and she co-developed and instructs an saa das course on email archiving. leah prescott leah prescott (infrastructure interest group co-chair) is the associate director for digital initiatives and special collections at georgetown university law library, where she is responsible for the digital curation lifecycle, from the development of a production digitization program for all types of library media, to providing access to digital materials in multiple repositories, to the implementation of a preservation strategy that is in-line with best practices. she is also responsible for conservation staff at the library, and for the special collections department with collections that include manuscripts, archives, rare books and the specialized national equal justice library. previously leah was the digital projects coordinator at the getty research institute in los angeles, where she participated in research and development of digital solutions, including digital repositories for access and preservation, mass digitization projects, and initiatives to create new avenues for scholarly collaboration in art history. she has a b.a. degree in history from the university of connecticut, an m.l.s. degree from syracuse university, and has been certified as an archivist since . linda reynolds linda reynolds (standards and practices interest group co-chair) is the director of the east texas research center at stephen f. austin state university (sfa) in nacogdoches, texas. she is responsible for overseeing all aspects of archival work from paper processing to digital preservation. since there is no digital archivist at sfa, she has taken on that responsibility. she is a one woman digital show and has thrown herself into the deep end of the digital pool to bring sfa’s chaotic digital assets into a manageable system. she advises and works with many cultural heritage institutions and community members in the east texas area on processing, preservation, and providing access of digital and non-digital material. deb verhoff deb verhoff (content interest group co-chair) is the digital collections manager for nyu libraries with responsibility for planning repository services. in her role within the digital library technology services team, she guides digital preservation activities for born digital and digitized content. prior to joining nyu, deb worked as an arts librarian and led a digital library project for robert wilson's watermill center. she holds a masters in fine arts from the school of the museum of fine arts, boston and a masters in library and information science from simmons college. host organization trevor owens dr. trevor owens (coordinating committee, dlf representative) is a librarian, researcher, policy maker, and educator working on digital infrastructure for libraries. owens serves as the first head of digital content management for library services at the library of congress. in addition, he teaches graduate seminars in digital history for american university’s history department and graduate seminars and digital preservation for the university of maryland’s college of information, where he is also a research affiliate with the digital curation innovation center. owens is the author of three books, including the theory and craft of digital preservation and designing online communities: how designers, developers, community managers, and software structure discourse and knowledge production on the web. his research and writing has been featured in: curator: the museum journal, digital humanities quarterly, the journal of digital humanities, d-lib, simulation & gaming, science communication, new directions in folklore, and american libraries. he serves as a dlf advisory committee member. aliya reich aliya reich (clir/dlf representative) is clir/dlf’s program manager for conferences and events, where she leads the planning team that puts on the dlf forum and its affiliated events, including support for ndsa’s digital preservation conference. she also works closely with iiif, another clir affiliate, and their event management team. aliya’s academic background is in art history and museums, and she’s held positions at historic annapolis, the phillips collection, the national building museum, the walters art museum, and the national gallery of art. in her spare time she coaches running with charm city run, leads pacing for parkinson’s, a charity running team at the baltimore running festival, and is a big sister through big brothers big sisters at the y of central maryland. she earned her ma in art history at washington university in st. louis and her ba in art history and french from the university of virginia. ndsa about members groups calendar social twitter itunes youtube news linkedin contact ndsa c/o clir+dlf south clark street, arlington, va e: ndsa@diglib.org ndsa the ndsa is proudly hosted by the digital library federation at clir. all content on this site is available for re-use under a cc by-sa . international license. dlf literary machines literary machines digital libraries, books, archives archiviiify a short guide to download digitized books from internet archive and rehost on your own infrastructure using iiif with full-text search. pywb . - docker quickstart four years have passed since i first wrote of pywb: it was a young tool at the time, but already usable and extremely simple to deploy. since then a lot of works has been done by ilya kreymer (and others), resulting in all the new features available with the . release. also, some very big webarchiving initiatives have moved and used pywb in these years: webrecorder itself, rhizome, perma, arquivo pt in portugal, the italian national library in florence (italy), (others i’m missing). anonymous webarchiving webarchiving activities, as any other activity where an http client is involved, leave marks of their steps: the web server you are visiting or crawling will save your ip address in its logs (or even worse it can decide to ban your ip). this is usually not a problem, there are plenty of good reasons for a webserver to keep logs of its visitors. but sometimes you may need to protect your own identity when you are visiting or saving something from a website, and there a lot of sensitive careers that need this protection: activists, journalist, political dissidents. tor has been invented for this, and today offer a good protection to browse anonymously the web. can we also archive the web through tor? open bni il maggio viene annunciato il rilascio libero della bibliografia nazionale italiana (bni). viene apprezzata l’apertura di questo catalogo (anche se con i limiti dei soli pdf), e da profano di biblioteconomia faccio anche una domanda sull’effettivo caso d’uso della bni. il agosto viene annunciato il rilascio delle annate e anche in formato unimarc e marcxml. incuriosito dal catalogo inizio ad esplorarlo, per pensare a possibili trasformazioni (triple rdf) o arricchimenti con/verso altri dati (wikidata). epub linkrot linkrot also affects epub files (who would have thought! :)). how to check the health of external links in epub books (required tools: a shell, atool, pup, gnu parallel). skos nuovo soggettario, api e autocomplete come creare una api per un form con autocompletamento usando i termini del nuovo soggettario, con i sorted sets di redis e nginx+lua. serve deepzoom images from a zip archive with openseadragon vips is a fast image processing system. version higher than . can generate static tiles of big images in deepzoom format, saving them directly into a zip archive. a wayback machine (pywb) on a cheap, shared host for a long time the only free (i’m unaware of commercial ones) implementation of a web archival replay software has been the wayback machine (now openwayback). it’s a stable and mature software, with a strong community behind. to use it you need to be confident with the deploy of a java web application; not so difficult, and documentation is exaustive. but there is a new player in the game, pywb, developed by ilya kramer, a former internet archive developer. built in python, relatively simpler than wayback, and now used in a pro archiving project at rhizome. opendata dell’anagrafe biblioteche come usare gli opendata dell’anagrafe delle biblioteche italiane e disegnare su una mappa web gli indirizzi delle biblioteche. api json dell’opac sbn alcuni mesi fa è stata rilasciata da iccu una app mobile per consultare l’opac sbn. anche se graficamente poco accattivante l’app funziona bene, e trovo molto utili le funzioni di ricerca di un libro scansionando il codice a barre con la camera del telefonino, e la possibilità di bookmarkare dei preferiti. incuriosito dal funzionamento ho pensato di analizzarne il traffico http. maisonbisson menu close home search subscribe ☰menu maisonbisson a bunch of stuff i would have emailed you about scroll downpage of older posts → every journalist ryu spaeth on the dirty job of journalism: [e]very journalist […] at some point will have to face the morally indefensible way we go about our business: namely, using other people to tell a story about the world. not everyone dupes their subjects into trusting them, but absolutely everyone robs other people of their stories to tell their own. every journalist knows this flushed feeling, a mix of triumph and guilt, of securing the story that will redound glory unto them, not the subject. some subjects who have no outlet, who are voiceless, approve of this arrangement, since they have no other way of getting their story heard. but even they will not wholly recognize their own depiction in the newspaper, by virtue of the fact that it was told by someone else with their own agenda. this is what jonathan franzen has called the “inescapable shame of being a storyteller”—that it involves stealing from another person, much in the way some people believe a photograph steals a bit of the sitter’s soul. casey bisson on #journalism, #reporting, #storytelling, dec the three tribes of the internet authors primavera de filippi, juan ortiz freuler, and joshua tan outline three competing narratives that have shaped the internet: libertarian, corporate, and nationalist. “[these narratives] emerged from a community of shared interests; each calls for a set of institutional arrangements; each endures in today’s politics.” » about words casey bisson on #internet, #hyperspace, #law, #governance, #libertarian, #corporate, #nationalist, #berkman klein center, #harvard berkman center, nov happy d.b. cooper day d.b. cooper day is celebrated on this day, the saturday following thanksgiving, every year. casey bisson on #agent smith, #aircraft hijacking, #aviation accidents and incidents, #d.b. cooper, #fbi, #federal bureau of investigation, #festival, #hijackers, #hijacking, #mysteries, #skyjacking, nov vitaminwater's #nophoneforayear contest back in the before times, vitaminwater invited applicants to a contest to go a full year without a smartphone or tablet. it was partly in response to rising concerns over the effect of all those alerts on our brains. over , people clamored for the chance, but author elana a. mugdan’s entry stood out with an amusing video, and in february the company took away her iphone s and handed her a kyocera flip phone. » about words casey bisson on #vitaminwater, #nophoneforayear, #scrollfreeforayear, #smartphones, #ethical technology, #humane technology, nov membership-driven news media from the membership guide’s handbook/manifesto: journalism is facing both a trust crisis and a sustainability crisis. membership answers to both. it is a social contract between a news organization and its members in which members give their time, money, energy, expertise, and connections to support a cause that they believe in. in exchange, the news organization offers transparency and opportunities to meaningfully contribute to both the sustainability and impact of the organization. elsewhere it continues: membership is not subscription by another name, nor a brand campaign that can be toggled on and off. …and: memberful routines are workflows that connect audience members to journalism and the people producing it. routines are the basis for a strong membership strategy. notice that audience members are specified here, which is likely a wider group than your members. casey bisson on #membership, #journalism, #monetization, #publishers, #news organizations, #media, oct political bias in social media algorithms and media monetization models new reports reveal yet more structural political biases in consumption and monetization models. » about wordscasey bisson on #politics, #media, #algorithms, #monetization, #bias, #journalism, #social media, #news organizations, oct media monetization vs. internet advertising media face structural, regulatory, and technical hurdles to effectively monetizing with ads on the internet, but there are some solutions that are working. » about words casey bisson on #advertising, #ads, #media monetization, #monetization models, #media, #journalism, #news organizations, aug the argument against likes: aim for deeper, more genuine interactions sweet pea on the state of social media and dating apps: “we are not creating a healthy society when we’re telling millions of young people that the key to happy relationships is photo worthy of an impulsive right swipe.” » about words casey bisson on #likes, #social media, #dating apps, #social software, #signal, aug paid reactions: virtual awards and tipping reddit and twitch both allow members to pay for the privilege of reacting to other member's content with special awards or stickers. » about words casey bisson on #social media, #reactions, #paid reactions, #virtual awards, #tipping, #revenue, #reddit, #twitch, aug reactions facebook introduced reactions with an emphasis on both the nuance they enabled and the mobile convenience: “[i]f you are sharing something that is sad [...] it might not feel comfortable to like that post.” later: “commenting might afford nuanced responses, but composing those responses on a [mobile] keypad takes too much time.” » about words casey bisson on #reactions, #likes, #social media, #facebook, #instagram, aug “likes” vs. “faves” twitter switched from faves to likes in . “you might like a lot of things, but not everything can be your *favorite*,” they explained. weeks after the change, liking activity for existing users was up % and % for new users. » about words casey bisson on #likes, #faves, #social media, #twitter, #facebook, #microcopy, aug honey cocktails: eau de lavender liquor.com’s recipe for eau de lavender, from a larger collection of cocktails with honey. they all look and sound delightful, but i can vouch for the eau de lavender. ingredients / oz tequila / oz fresh lemon juice / oz honey syrup egg white dash scrappy’s lavender bitters garnish: lavender sprig steps add all ingredients into a shaker and dry-shake (without ice). add ice and shake again to emulsify thoroughly. strain into a chilled coupe glass. garnish with a lavender sprig. honey syrup: add / cup honey and / cup water to a small saucepan over medium heat. (you can experiment and decide how much of a honey flavor you want in your syrup. the more honey you use, the thicker the syrup and stronger in flavor it will be.) stir until blended. strain into a jar and seal tightly with a lid. will keep for month in the refrigerator. ↩︎ casey bisson on #cocktails, #mixology, #honey, may satellite tracking if you’re not reading skyriddles blog, then you’re not tracking the sky above. and you might have missed the re-discovery of a satellite launched in and lost for nearly years. as it turns out, there’s a lot of stuff that’s been forgotten up there, and quite a bit that some are trying to hide. the blog is an entertaining view into the world satellites, including communication, spy, weather, research, and the occasional probe going further afield. casey bisson on #satellite tracking, #space, apr i'm missing restaurants now @nakedlunchsf was notable for having both a strong contender for the best burger in the city, _and_... casey bisson on #photo, #photoblog, #stayhome, #supportlocalbusiness, mar when unzip fails on macos with utf unzip can fail on macos when utf- chars are in the archive. the solution is to use ditto. via a github issue: ditto -v -x -k --sequesterrsrc --rsrc filename.zip destinationdirectory casey bisson on #zip, #unzip, #macos, #utf , feb tiktok vs. instagram zuckerberg describes tiktok as “almost like the explore tab that we have on instagram,” but connie chan suggests he's missing the deeper value of ai, and techcrunch's josh constantine suggests zuck is missing the bigger difference in intent on tiktok. » about words casey bisson on #tiktok, #instagram, #social media, #social software, #social networks, #social signals, #artificial intelligence, #ai, jan swipegram template benjamin lee’s instructions and downloadable template to make panoramic carousel instagrams (aka #swipegram), as illustrated via his animation above. » about words casey bisson on #instagram, #template, #swipegram, dec “it is clear that the books owned the shop... “it is clear that the books owned the shop rather than the other way about. everywhere they... casey bisson on #photo, #photoblog, #lovemaine, #portlandmaine, #mustbevancouver, #penderstreet, #downtownvancouver, dec “life is like riding a bicycle... “life is like riding a bicycle. to keep your balance, you must keep moving.” —wisdom by albert... casey bisson on #photo, #photoblog, #forahappymoment, #voreskbh, #visitcopenhagen, #buyfilmnotmegapixels, #ig_denmark, #fujipro h, #ishootfilm, #travelog, #filmisnotdead, #visitdenmark, #mytinyatlas, #pro h, #fuji, #believeinfilm, #københavn, #analoguepeople, #instapassport, #staybrokeshootfilm, #hasselblad, #igerscopenhagen, #flashesofdelight, #exploringtheglobe, nov notes about spotify creator features spotify often gets bashed by top creators. the service pays just $ . per stream, but with million users listening to an average of hours per month, those streams can add up for creators who can get the listener’s attention. spotify verifies artists who then get additional benefits on the platform. some artists find success the traditional route, some optimize their work for the system, others work the system…and some really work it. relevance to other network/aggregation platforms: tiny payments add up, and given a platform, creators will find a way to get and maximize value from it. the critical component is customers. casey bisson on #spotify, #creators, #social networks, #revenue, #aggregation, nov exiftool examples i use for encoding analog camera details i’m a stickler for detail and love to add exif metadata for my film cameras to my scanned images. these are my notes to self about the data i use most often. i only wish exif had fields to record the film details too. » about wordscasey bisson on #exiftool, #photography, #exif, #metadata, nov random notes on instagram delete your old photos, rebrand your page, and delete it entirely are all common advice. plus some tools and traps to be aware of. » about words casey bisson on #instagram, #social media, #photography, oct every media has its tastemakers and influencers every media, network, or platform has would-be influencers or promoters who can help connect consumers with creators. don’t mistake the value of these tastemakers, and be sure to find a place for them to create new value for your platform. » about wordscasey bisson on #spotify, #instagram, #social media, #social networks, #influencers, #tastemakers, oct storehouse: the most wonderful story sharing flop ever storehouse shuttered in summer , just a couple years after they launched, but the app and website introduced or made beautiful a few features that remain interesting now. » about wordscasey bisson on #storehouse, #photo sharing, #story sharing, #microblogging, #blogging, #social media, #user-generated content, #ugc, oct page of older posts →maisonbisson inkdroid toggle navigation inkdroid about bookmarks photos music software talks - - ~ outgoing - - ~ trump's tweets - - ~ noarchive - - ~ what's the diff? - - ~ for - - ~ diss music - - ~ years of robots.txt - - ~ curation communities - - ~ mystery file! - - ~ kettle - - ~ static-dynamic - - ~ dark reading - - ~ seeing software - - ~ curating corpora - - ~ fuzzy - - ~ penny - - ~ fuzzy file formats - - ~ pandoc - - ~ fuzzy matching - - ~ less is (sometimes) more - - ~ teaching digital curation - - ~ rss - - ~ organizations on twitter - - ~ bibdesk, zotero and jabref - - ~ disinformation metadata - - ~ equipment - - ~ twitter - - ~ music for hard times - - ~ digital curation - - ~ dependency hell - - ~ keyboard - - ~ tech tree - - ~ appraisal talk in web archives - - ~ talk talk - - ~ original voice - - ~ write it down - - ~ sun and moon - - ~ first thought - - ~ studying the covid- web - - ~ in media res - - ~ appraising covid- - - ~ design justice - - ~ folding - - ~ real world of technology (mp s) - - ~ the journalist of algorithms - - ~ anselm - - ~ rose notes - - ~ (kinda) static sites - - ~ metro, washer/dryer - - ~ an avalanche of printed numbers « prev next » unless otherwise noted all the content here is licensed cc-by . . updated: / / * bug fix: installer throws an error when attempting to install per user * bug fix: marceditor -- marcedit will be deprecating legacy page loading. this option is now ignored if set and will be removed entirely in future builds. . . updated: / / * change: allow os to manage supported supported security protocol types. * change: remove com.sun dependency related to dns and httpserver * change: changed appdata path * change: first install automatically imports settings from marcedit . - .x * change: field count - simplify ui (consolidate elements) * change: windows -- update help urls to oclc * change: generate fast headings -- update help urls * change: .net changes thread stats queuing. updating thread processing on forms: * generate fast headings * batch process records * build links * main window * rda helper * delete selected records * marc tools * check url tools * marcvalidator * marcengine * task manager * z . * ils integration processing * character conversions * format handing (delimited text, openrefine, etc.) * change: xml function list -- update process for opening urls * change: z . preferences window - update process for opening urls * change: about windows -- new information, updated how version information is calculated. * change: catalog calculator window -- update process for opening urls * change: generate call numbers -- update process for opening urls * change: generate material formats -- update process for opening urls * change: tab delimiter -- remove context windows * change: tab delimiter -- new options ui * change: tab delimiter -- normalization changes * change: remove old help html page * change: remove old hex editor page * change: updated hex editor to integrate into main program * change: main window -- remove custom scheduler dependency * change: ui update to allow more items * change: main window -- new icon * change: main window -- update process for opening urls * change: main window -- removed context menus * change: main window -- upgrade changes to new executable name * change: main window -- updated the following menu items: * edit linked data tools * removed old help menu item * added new application shortcut * change: oclc bulk downloader -- new ui elements to correspond to new oclc api * change: oclc search page -- new ui elements to correspond to new oclc api * change: preferences -- updates related to various preference changes: * hex editor * integrations * editor * other * change: rda helper -- update process for opening urls * change: rda helper -- opening files for editing * change: removed the script maker * change: templates for perl and vbscripts includes * change: removed find/search xml in the xml editor and consolidated in existing windows * change: delete selected records: exposed the form and controls to the marceditor * change: sparql browser -- update process for opening urls * change: sparql browser -- removed context menus * change: troubleshooting wizard -- added more error codes and kb information to the wizard * change: unimarc utility -- controls change, configurable transform selections * change: marc utilities -- removed the context menu * change: first run wizard -- new options, new agent images * change: xml editor -- delete block addition * change: xml editor -- xquery transform support * change: xml profile wizard -- option to process attributes * change: marceditor -- status bar control doesn't exist in net . . control has changed. * change: marceditor -- improved page loading * change: marceditor -- file tracking updated to handle times when the file opened is a temp record * change: marceditor -- removed ~ k of old code * change: marceditor -- added delete selected records option * change: removed helper code used by installer * change: removed office menu formatting code * change: consolidated extensions into new class (removed files) * change: removed calls marshalled to the windows api -- replaced with managed code * change: openrefine format handler updated to capture changes between openrefine versions * change: marcengine -- namespace update to * change: wizard -- missing unicode font options more obvious * change: wizard install puts font in program directory so that additional users can simply copy (not download) the font on use * change: checkurls: removed support for insecure crypto-types * change: checkurls: additional heuristics to respond dynamically to http status codes * change: all components -- .net . includes a new codepages library that allows for extended codepage support beyond the default framework. added across the project. * change: marcvalidator -- new rules process that attempts to determine if records are too long for processing when validating rules or structure. * change: command-line -- batch process switch has been added to the tasks processing function * change: options -- allow user path to be reset. * bug fix: main window -- corrects process for determining version for update * bug fix: main window -- updated image * bug fix: when doing first run, wizard not showing in some cases. * bug fix: main window -- last tool used sometimes shows duplicates * bug fix: rda helper -- $e processing * bug fix: rda helper -- punctuation in the $e * bug fix: xml profile wizard -- when the top element is selected, it's not viewed for processing (which means not seeing element data or attribute data) * bug fix: marceditor -- page processing correct to handle invalid formatted data better * bug fix: installation wizard -- if a unicode font was installed during the first run process, it wouldn't be recognized. * bug fix: marcvalidator fails when attempting to process a .mrk file from outside the marceditor * bug fix: linked data processing: when processing services with multiple redirects -- process may stop pre-maturely. (example: lc's id.loc.gov xx processing) * bug fix: edit field -- find fields with just spaces are trimmed, causing the field data to process improperly. * bug fix: rda helper will fail if ldr length is incorrect when attempting to determine character encoding topicbox ☹ sorry, your browser does not support the technologies needed to use our web interface. please make sure you have the latest version, and that javascript is enabled. the economic limits of bitcoin and the blockchain | nber skip to main content subscribe media open calls login login close search research explore research findings working papers books & chapters lectures interviews periodicals the digest the reporter the bulletin on retirement and disability the bulletin on health periodicals archive data & business cycles boston research data center business cycle dating public use data archive topics covid- us-china trade energy entrepreneurship growth and productivity programs & projects explore programs & projects programs economics of aging asset pricing children corporate finance development economics development of the american economy economic fluctuations and growth economics of education environment and energy economics health care health economics industrial organization international finance and macroeconomics international trade and investment labor studies law and economics monetary economics political economy productivity, innovation, and entrepreneurship public economics working groups behavioral finance chinese economy cohort studies economics of crime entrepreneurship household finance innovation policy insurance market design organizational economics personnel economics race and stratification in the economy risks of financial institutions urban economics projects & centers center for aging and health research conference on econometrics and mathematical economics conference on research in income and wealth economics of digitization gender in the economy study group illinois workplace wellness improving health outcomes for an aging population macroeconomics annual measuring the clinical and economic outcomes associated with delivery systems oregon health insurance experiment retirement and disability research center the roybal center for behavior change in health satellite national health accounts science of science funding training program in aging and health economics transportation economics in the st century union army data & early indicators value of medical research women working longer conferences affiliated scholars nber news explore nber news research in the news nobel laureates featured working papers archive career resources explore career resources ra positions – not at the nber calls for fellowship applications phd candidates in economics about explore about leadership & governance support & funding history standards of conduct privacy policy staff employment opportunities at nber subscribe media open calls search home research working papers the economic limits of bitcoin and the… the economic limits of bitcoin and the blockchain eric budish share twitter linkedin email working paper doi . /w issue date june the amount of computational power devoted to anonymous, decentralized blockchains such as bitcoin's must simultaneously satisfy two conditions in equilibrium: ( ) a zero-profit condition among miners, who engage in a rent-seeking competition for the prize associated with adding the next block to the chain; and ( ) an incentive compatibility condition on the system's vulnerability to a “majority attack”, namely that the computational costs of such an attack must exceed the benefits. together, these two equations imply that ( ) the recurring, “flow”, payments to miners for running the blockchain must be large relative to the one-off, “stock”, benefits of attacking it. this is very expensive! the constraint is softer (i.e., stock versus stock) if both (i) the mining technology used to run the blockchain is both scarce and non-repurposable, and (ii) any majority attack is a “sabotage” in that it causes a collapse in the economic value of the blockchain; however, reliance on non-repurposable technology for security and vulnerability to sabotage each raise their own concerns, and point to specific collapse scenarios. in particular, the model suggests that bitcoin would be majority attacked if it became sufficiently economically important — e.g., if it became a “store of value” akin to gold — which suggests that there are intrinsic economic limits to how economically important it can become in the first place. acknowledgements and disclosures project start date: feb , . first public draft: may , . for the record, the first large-stakes majority attack of a well-known cryptocurrency, the $ m attack on bitcoin gold, occurred a few weeks later in mid-may (wilmoth, ; wong, ). acknowledgments: thanks are due to susan athey, vitalik buterin, alex frankel, joshua gans, austan goolsbee, zhiguo he, joi ito, steve kaplan, anil kashyap, judd kessler, randall kroszner, robin lee, jacob leshno, neale mahoney, sendhil mullainathan, david parkes, john shim, scott stornetta, aviv zohar, and seminar participants at chicago booth and the mit digital currency initiative. natalia drozdoff and matthew o'keefe have provided excellent research assistance. disclosure: i do not have any financial interests in blockchain companies or cryptocurrencies, either long or short. the views expressed herein are those of the author and do not necessarily reflect the views of the national bureau of economic research. download citation marc ris bibteΧ download citation data related topics other general, teaching microeconomics market structure and distribution general equilibrium macroeconomics money and interest rates financial economics financial markets portfolio selection and asset pricing financial institutions behavioral finance industrial organization industry studies culture programs asset pricing corporate finance industrial organization monetary economics productivity, innovation, and entrepreneurship working groups entrepreneurship risks of financial institutions market design conferences si monetary economics mentioned in the news bitcoin too risky to go mainstream, central bank overseer says june , source: the star (toronto) more from nber in addition to working papers, the nber disseminates affiliates’ latest findings through a range of free periodicals — the nber reporter, the nber digest, the bulletin on retirement and disability, and the bulletin on health — as well as online conference reports, video lectures, and interviews. the economics of digitization article author: shane greenstein the nber economics of digitization project, established in with support from the alfred p. sloan foundation,... the martin feldstein lecture: journey across a century of women lecture claudia goldin, the henry lee professor of economics at harvard university and a past president of the american... summer institute methods lectures: differential privacy for economists lecture the extent to which individual responses to household surveys are protected from discovery by outside parties depends... national bureau of economic research contact us massachusetts ave. cambridge, ma - - info@nber.org follow homepage privacy policy © national bureau of economic research. all rights reserved. outgoing toggle navigation inkdroid about bookmarks photos music software talks outgoing february , politics version control tree rings by tracy o this post is really about exploring historical datasets with version control systems. mark graham posed a question to me earlier today asking what we know about the twitter accounts of the members of congress, specifically whether they have been removed after they left office. the hypothesis was that some members of the house and senate may decide to delete their account on leaving dc. i was immediately reminded of the excellent congress-legislators project which collects all kinds of information about house and senate members including their social media accounts into yaml files that are versioned in a github repository. github is a great place to curate a dataset like this because it allows anyone with a github account to contribute to editing the data, and to share utilities to automate checks and modifications. unfortunately the file that tracks social media accounts is only for current members. once they leave office they are removed from the file. the project does track other historical information for legislators. but the social media data isn’t pulled in when this transition happens, or so it seems. luckily git doesn’t forget. since the project is using a version control system all of the previously known social media links are in the history of the repository! so i wrote a small program that uses gitpython to walk the legislators-social-media.yaml file backwards in time through each commit, parse the yaml at that previous state, and merge that information into a union of all the current and past legislator information. you can see the resulting program and output in us-legislators-social. there’s a little bit of a wrinkle in that not everything in the version history should be carried forward because errors were corrected and bugs were fixed. without digging into the diffs and analyzing them more it’s hard to say whether a commit was a bug fix or if it was simply adding new or deleting old information. if the yaml doesn’t parse at a particular state that’s easy to ignore. it also looks like the maintainers split out account ids from account usernames at one point. derek willis helpfully pointed out to me that twitter don’t care about the capitalization of usernames in urls, so these needed to be normalized when merging the data. the same is true of facebook, instagram and youtube. i guarded against these cases but if you notice other problems let me know. with the resulting merged historical data it’s not too hard to write a program to read in the data, identify the politicians who left office after the th congress, and examine their twitter accounts to see that they are live. it turned out to be a little bit harder than i expected because it’s not as easy as you might think to check if a twitter account is live or not. twitter’s web servers return a http ok message even when responding to requests for urls of non-existent accounts. to complicate things further the error message that displays indicating it is not an account only displays when the page is rendered in a browser. so a simple web scraping job that looks at the html is not sufficient. and finally just because a twitter username no longer seems to work, it’s possible that the user has changed it to a new screen_name. fortunately the unitedstates project also tracks the twitter user id (sometimes). if the user account is still there you can use the twitter api to look up their current screen_name and see if it is different. after putting all this together it’s possible to generate a simple table of legislators who left office at the end of the th congress, and their twitter account information. name url url_ok user_id new_url lamar alexander https://twitter.com/senalexander true michael b. enzi https://twitter.com/senatorenzi true pat roberts https://twitter.com/senpatroberts true tom udall https://twitter.com/senatortomudall true justin amash https://twitter.com/justinamash true rob bishop https://twitter.com/reprobbishop true k. michael conaway https://twitter.com/conawaytx true susan a. davis https://twitter.com/repsusandavis false eliot l. engel https://twitter.com/repeliotengel true bill flores https://twitter.com/repbillflores false cory gardner https://twitter.com/sencorygardner true peter t. king https://twitter.com/reppeteking true steve king https://twitter.com/stevekingia true daniel lipinski https://twitter.com/replipinski true david loebsack https://twitter.com/daveloebsack true nita m. lowey https://twitter.com/nitalowey true kenny marchant https://twitter.com/repkenmarchant true pete olson https://twitter.com/reppeteolson true martha roby https://twitter.com/repmartharoby false https://twitter.com/martharobyal david p. roe https://twitter.com/drphilroe true f. james sensenbrenner, jr. https://twitter.com/jimpressoffice false josé e. serrano https://twitter.com/repjoseserrano true john shimkus https://twitter.com/repshimkus true mac thornberry https://twitter.com/mactxpress true scott r. tipton https://twitter.com/reptipton true peter j. visclosky https://twitter.com/repvisclosky true greg walden https://twitter.com/repgregwalden true rob woodall https://twitter.com/reprobwoodall true ted s. yoho https://twitter.com/reptedyoho true doug collins https://twitter.com/repdougcollins true tulsi gabbard https://twitter.com/tulsipress true susan w. brooks https://twitter.com/susanwbrooks true joseph p. kennedy iii https://twitter.com/repjoekennedy false https://twitter.com/joekennedy george holding https://twitter.com/repholding true denny heck https://twitter.com/repdennyheck false https://twitter.com/ltgovdennyheck bradley byrne https://twitter.com/repbyrne true ralph lee abraham https://twitter.com/repabraham true will hurd https://twitter.com/hurdonthehill true david perdue https://twitter.com/sendavidperdue true mark walker https://twitter.com/repmarkwalker true francis rooney https://twitter.com/reprooney true paul mitchell https://twitter.com/reppaulmitchell true doug jones https://twitter.com/sendougjones true tj cox https://twitter.com/reptjcox true gilbert ray cisneros, jr. https://twitter.com/repgilcisneros true harley rouda https://twitter.com/repharley true ross spano https://twitter.com/reprossspano true debbie mucarsel-powell https://twitter.com/repdmp true donna e. shalala https://twitter.com/repshalala false abby finkenauer https://twitter.com/repfinkenauer true steve watkins https://twitter.com/rep_watkins false xochitl torres small https://twitter.com/reptorressmall true max rose https://twitter.com/repmaxrose true anthony brindisi https://twitter.com/repbrindisi true kendra s. horn https://twitter.com/repkendrahorn false https://twitter.com/kendrashorn joe cunningham https://twitter.com/repcunningham true ben mcadams https://twitter.com/repbenmcadams false https://twitter.com/benmcadamsut denver riggleman https://twitter.com/repriggleman true in most cases where the account has been updated the individual simply changed their twitter username, sometimes remove “rep” from it–like repjoekennedy to joekennedy. as an aside i’m kind of surprised that twitter username wasn’t taken to be honest. maybe that’s a perk of having a verified account or of being a politician? but if you look closely you can see there were a few that seemed to have deleted their account altogether: name url url_ok user_id susan a. davis https://twitter.com/repsusandavis false bill flores https://twitter.com/repbillflores false f. james sensenbrenner, jr. https://twitter.com/jimpressoffice false donna e. shalala https://twitter.com/repshalala false steve watkins https://twitter.com/rep_watkins false there are two notable exceptions to this. the first is vice president kamala harris. my logic for determining if a person was leaving congress was to see if they served in a term ending on - - , and weren’t serving in a term starting then. but harris is different because her term as a senator is listed as ending on - - . her old account @senkamalaharris is no longer available, but her twitter user id is still active and is now attached to the account at @vp. the other of course is joe biden, who stopped being a senator in order to become the president. his twitter account remains the same at @joebiden. it’s worth highlighting here how there seems to be no uniform approach to handling this process. in one case @senkamalaharris is temporarily blessed as the vp, with a unified account history underneath. in the other there is a separation between @joebiden and @potus. it seems like twitter has some work to do on managing identities, or maybe the congress needs to prescribe a set of procedures? or maybe i’m missing part of the picture, and that just as @repjoekennedy somehow changed back to @joekennedy there is some namespace management going on behind the scenes? if you are interested in other social media platforms like facebook, instagram and youtube the unitedstates project tracks information for those platforms too. i merged that information into the legislators.yaml file i discussed here if you want to try to check them. i think that one thing this experiment shows is that if the platform allows for usernames to be changed it is critical to track the user id as well. i didn’t do the work to check that those accounts exist. but that’s a project for another day. i’m not sure this list of five deleted accounts is terribly interesting at the end of all this. possibly? but on the plus side i did learn how to interact with git better from python, which is something i can imagine returning to in the future. it’s not every day that you have to think of the versions of a dataset as an important feature of the data, outside of serving as a backup that can be reverted to if necessary. but of course data changes in time, and if seeing that data over time is useful, then the revision history takes on a new significance. it’s nothing new to see version control systems as critical data provenance technologies, but it felt new to actually use one that way to answer a question. thanks mark! unless otherwise noted all the content here is licensed cc-by none making customizable interactive tutorials with google forms | information wants to be free home about speaking writing contact facebook twitter google+ linkedin skype rss making customizable interactive tutorials with google forms by meredith farkas by meredith farkas on / / with comments free the information!, higher ed, instruction, librarianship, online education, reference, work in september, i gave a talk at oregon state university’s instruction librarian get-together about the interactive tutorials i built at pcc last year that have been integral to our remote instructional strategy. i thought i’d share my slides and notes here in case others are inspired by what i did and to share the amazing assessment data i recently received about the impact of these tutorials that i included in this blog post. you can click on any of the slides to see them larger and you can also view the original slides here (or below). at the end of the post are a few tutorials that you can access or make copies of. i’ve been working at pcc for over six years now, but i’ve been doing online instructional design work for years and i will freely admit that it’s my favorite thing to do. i started working at a very small rural academic library where i had to find creative and usually free solutions to instructional problems. and i love that sort of creative work. it’s what keeps me going. i’ve actually been using survey software as a teaching tool since i worked at portland state university. there, my colleague amy hofer and i used qualtrics to create really polished and beautiful interactive tutorials for students in our university studies program. i also used qualtrics at psu and pcc to create pre-assignments for students to complete prior to an instruction session that both taught students skills and gave me formative assessment data that informed my teaching. so for example, students would watch a video on how to search for sources via ebsco and then would try searching for articles on their own topic. a year and a half ago, the amazing anne-marie dietering led my colleagues in a day of goal-setting retreat for our instruction program. in the end, we ended up selecting this goal, identify new ways information literacy instruction can reach courses other than direct instruction, which was broad enough to encompass a lot of activities people valued. for me, it allowed me to get back to my true love, online instructional design, which was awesome, because i was kind of in a place of burnout going into last fall. at pcc, we already had a lot of online instructional content to support our students. we even built a toolkit for faculty with information literacy learning materials they could incorporate into their classes without working with a librarian. the toolkit contains lots of handouts, videos, in-class or online activities and more. but it was a lot of pieces and they really required faculty to do the work to incorporate them into their classes. what i wanted to build was something that took advantage of our existing content, but tied it up with a bow for faculty. so they really could just take whatever it is, assign students to complete it, and know students are learning and practicing what they learned. i really wanted it to mimic the sort of experience they might get from a library instruction session. and that’s when i came back to the sort interactive tutorials i built at psu. so i started to sketch out what the requirements of the project were. even though we have qualtrics at pcc, i wasn’t % sure qualtrics would be a good fit for this. it definitely did meet those first four criteria given that we already have it, it provides the ability to embed video, for students to get a copy of the work they did, and most features of the software are ada accessible. but i wanted both my colleagues in the library and disciplinary faculty members to be able to easily see the responses of their students and to make copies of the tutorial to personalize for the particular course. and while pcc does have qualtrics, the majority of faculty have never used it on the back-end and many do not have accounts. so that’s when google forms seemed like the obvious choice and i had to give up on my fantasy of having pretty tutorials. i started by creating a proof of concept based on an evaluating sources activity i often use in face-to-face reading and writing classes. you can view a copy of it here and can copy it if you want to use it in your own teaching. in this case, students would watch a video we have on techniques for evaluating sources. then i demonstrate the use of those techniques, which predate caulfield’s four moves, but are not too dissimilar. so they can see how i would go about evaluating this article from the atlantic on the subject of daca. the students then will evaluate two sources on their own and there are specific questions to guide them. during fall term, i showed my proof of concept to my colleagues in the library as well as at faculty department meetings in some of my liaison areas. and there was a good amount of enthusiasm from disciplinary faculty – enough that i felt encouraged to continue. one anthropology instructor who i’ve worked closely with over the years asked if i could create a tutorial on finding sources to support research in her online biological anthropology classes – classes i was going to be embedded in over winter term. and i thought this was a perfect opportunity to really pilot the use of the google form tutorial concept and see how students do. so i made an interactive tutorial where students go through and learn a thing, then practice a thing, learn another thing, then practice that thing. and fortunately, they seemed to complete the tutorial without difficulty and from what i heard from the instructor, they did a really good job of citing quality sources in their research paper in the course. later in the presentation, you’ll see that i received clear data demonstrating the impact of this tutorial from the anthropology department’s annual assessment project. so my vision for having faculty make copies of tutorials to use themselves had one major drawback. let’s imagine they were really successful and we let a thousand flowers bloom. well, the problem with that is that you now have a thousand versions of your tutorials lying around and what do you do when a video is updated or a link changes or some other update is needed? i needed a way to track who is using the tutorials so that i could contact them when updates were made. so here’s how i structured it. i created a qualtrics form that is a gateway to accessing the tutorials. faculty need to put in their name, email, and subject area. they then can view tutorials and check boxes for the ones they are interested in using. once they submit, they are taking to a page where they can actually copy the tutorials they want. so now, i have the contact information for the folks who are using the tutorials. this is not just useful for updates, but possibly for future information literacy assessment we might want to do. the individual tutorials are also findable via our information literacy teaching materials toolkit. so when the pandemic came just when i was ready to expand this, i felt a little like nostradamus or something. the timing was very, very good during a very, very bad situation. so we work with biology every single term in week to teach students about the library and about what peer review means, why it matters, and how to find peer-reviewed articles. as soon as it became clear that spring term was going to start online, i scrambled to create this tutorial that replicates, as well as i could, what we do in the classroom. so they do the same activity we did in-class where they look at a scholarly article and a news article and list the differences they notice. and in place of discussions, i had them watch videos and share insights. i then shared this with the biology faculty on my campus and they assigned it to their students in week . it was great! [you can view the biology tutorial here and make a copy of it here]. and during spring term i made a lot more tutorials. the biggest upside of using google forms is its simplicity and familiarity. nearly everyone has created a google form and they are dead simple to build. i knew that my colleagues in the library could easily copy something i made and tailor it to the courses they’re working with or make something from scratch. and i knew faculty could easily copy an existing tutorial and be able to see student responses. for students, it’s a low-bandwidth and easy-to-complete online worksheet. the barriers are minimal. and on the back-end, just like with libguides, there’s a feature where you can easily copy content from another google form. the downsides of using google forms are not terribly significant. i mean, i’m sad that i can’t create beautiful, modern, sharp-looking forms, but it’s not the end of the world. the formatting features in google forms are really minimal. to create a hyperlink, you actually need to display the whole url. blech. then in terms of accessibility, there’s also no alt tag feature for images, so i just make sure to describe the picture in the text preceding or following it. i haven’t heard any complaints from faculty about having to fill out the qualtrics form in order to get access to the tutorials, but it’s still another hurdle, however small. this spring, we used google form tutorials to replace the teaching we normally do in classes like biology , writing , reading , and many others. we’ve also used them in addition to synchronous instruction, sort of like i did with my pre-assignments. but word about the google form tutorials spread and we ended up working with classes we never had a connection to before. for example, the biology faculty told the anatomy and physiology instructors about the tutorial and they wanted me to make a similar one for a&p. and that’s a key class for nursing and biology majors that we never worked with before on my campus. lots of my colleagues have made copies of my tutorials and tailored them to the classes they’re working with or created their own from scratch. and we’ve gotten a lot of positive feedback from faculty, which really felt good during spring term when i know i was working myself to the bone. since giving this presentation, i learned from my colleagues in anthropology that they actually used my work as the basis of their annual assessment project (which every academic unit has to do). they used a normed rubric to assess student papers in anthropology and compared the papers of students who were in sections in which i was embedded (where they had access to the tutorial) to students in sections where they did not have an embedded librarian or a tutorial. they found that students in the class sections in which i was involved had a mean score of / and students in other classes had a mean score of / . that is significant!!! i am so grateful that my liaison area did this project that so validates my own work. here’s an excerpt from one email i received from an anatomy and physiology instructor: “i just wanted to follow up and say that the library assignment was a huge success! i’ve never had so many students actually complete this correctly with peer-reviewed sources in correct citation format. this is a great tool.” at the end of a term where i felt beyond worked to the bone, that was just the sort of encouragement i needed. i made copies of a few other tutorials i’ve created so others can access them: finding sources for your reading or writing assignment – make a copy finding articles for your social sciences research assignment – make a copy evaluating sources tutorial – make a copy biology library assignment – make a copy ← the crushing expectations on working women and where’s my fucking village? in all the bad… some good things → about the author meredith farkas meredith farkas is a faculty librarian at portland community college in oregon and an adjunct faculty member at san jose state university's ischool. she is the author of the book “social software in libraries: building collaboration, communication and community online” (information today, ) and writes the monthly column “technology in practice” for american libraries. meredith was honored in with the acrl instruction section innovation award, in and with the wise excellence in online education award and in with the lita/library hi tech award for outstanding communication in library and information technology. she has been writing the blog information wants to be free since . comments ruth bird says: / / at : am as a retired librarian with a passion for this sort of thing, may i say how great it is? i was at oxford, in the bodleian, but now i work with the danish institute of human rights to deliver distance learning in myanmar on eresources in law. i have used google forms, but look forward to exploring what you have done and learning from you. thanks so much for sharing! best wishes, ruth meredith farkas says: / / at : am thanks for the kind words, ruth! and best wishes with your work in myanmar; that sounds really interesting and rewarding. dana says: / / at : am thanks for this – loving the evaluating content and have made a copy to play with! meredith farkas, author, information wants to be free subscribe via rss subscribe to this blog from the archives from the archives select month december november august may april february december november september august january june march february december november october september july june march january december november october september august june march january december november october september june april march february january december november september august july june may march february january december november october september august july may april march january december november october september august july june may april february january december october september august july june may march february january december october september august july june may april march february january november october august july june april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november categories about me ala american libraries assessment blogging book career classic blunders comment community college libraries community colleges ebooks election farce free the information! gender general hi higher ed inspiring stuff instruction intellectual freedom job search knowledge management librarianship libraries library school librarydayinthelife management mid-career mpow online education open access open source our digital future random reference research rss and syndication screencasting search social bookmarking social software speaking tech trends tenure track vermont wikis work work-life balance writing recent comments meredith farkas on in all the bad… some good things linda on in all the bad… some good things michelle kidd tackabery on contact dana on making customizable interactive tutorials with google forms meredith farkas on making customizable interactive tutorials with google forms most popular posts the essence of library . ? ( ) skills for the st century librarian ( ) keeping it real ( ) libraries in social networking software ( ) ebooks and libraries: a stream of concerns ( ) disclaimer this blog contains the author’s personal thoughts, which do not necessarily reflect the views of her employer. comments the author reserves the right to delete any comments she deems offensive, irrelevant, fraudulent, or blatant advertisements. go to top hectic pace hectic pace a view on libraries, the library business, and the business of libraries my pre-covid things authors note: these parodies are always about libraries and always based on christmas songs, stories, or poems. being what it is, this year is an exception to both&# ;that&# ;s right, i&# ;m siding with my family and admitting that my favorite things is not a christmas song. (sung to the tune of “my favorite things&# ;) [click the youtube link to listen while you sing along.] eating in restaurants and movies on big screenspeople who don&# ;t doubt the virtue of vaccines.inspiring leaders who don&# ;t act like kings.these were a few of my pre-covid things. live music venues and in-person classes.no masks or ... sitting in the reading room all day (sung to the tune of “walking in a winter wonderland”) [click the youtube link to listen while you sing along.] people shhhhhh, are you listening? in the stacks, laptops glistening the reading light&# ;s bright the library&# ;s right for sitting in the reading room all day. gone away are the book stacks here to stay, the only town&# ;s fax. we share all our books without judgy looks. sitting in the reading room all day. in the lobby we could build a book tree. readers guide is green and they stack well. i&# ;ll say &# ;do we have &# ;em?&# ; you&# ;ll say, &# ;yeah man.&# ; ... it’s the best library time of the year (sung to the tune of “it&# ;s the most wonderful time of the year&# ;) press play to sing along with the instrumental track! it&# ;s the best library time of the year with no more children yelling and no one is telling you &# ;get it in gear!&# ; it&# ;s the best library time of the year it&# ;s the qui-quietest season at school only smile-filled greetings and no more dull meetings where bosses are cruel it&# ;s the qui-quietest season at school there&# ;ll be books for re-stocking vendor end-of-year-hawking and overdue fine cash for beer send the word out to pre-schools drag queen visit ... maybe it’s books we need [i figured this was a song in desperate need of some new lyrics. sung to the tune of baby it&# ;s cold outside. you&# ;re gonna want to grab a singing partner and use the instrumental track for this one!] (listen to the track while you sing!) i really must binge (but maybe it&# ;s books we need) you mustn&# ;t infringe (it&# ;s definitely books we need) this season has been (reading will make you grin) so fun to watch (i&# ;ll hold the remote, you hold my scotch) my netflix queue scrolls forever (mystery, poems, whichever) and stranger things won&# ;t just watch itself (grab ... being a better ally: first, believe warning: i might make you uncomfortable. i’m uncomfortable. but it comes from an earnest place. i was recently lucky enough to participate with my oclc membership &# ; research division colleagues in deetta jones &# ; associates’ cultural competency training. this day-long session has a firm spot in the top of my professional development experiences. (not coincidentally, one of the others in that top was deetta’s management training i took part in when she was with the association of research libraries). a week later, i&# ;m still processing this incredible experience. and i&# ;m very grateful to oclc for sponsoring the workshop! ... fake news forever! librarians were among the first to join the call to arms and combat the onslaught of fake news that has permeated our political discussions for the last several months. frankly, it seems hard for anyone to be on the other side of this issue. but is it? not long after the effort to stop fake news in its tracks, a group of librarians began to consider the long-term implications of eradicating an entire body of content from history. thus began a concerted effort to preserve all the fake news that a vigilant group of librarians could gather up. building on ... how will you be remembered? my grandfather had a sizable library when he passed away, and his son (my father) would wind up with roughly half of it. i remember shelves and shelves of books of quotations. he was a criminal lawyer with a love of quotes. i either inherited this love or caught it through the osmosis of being surrounded by these books throughout my childhood. most of the books were ruined over the years by mold and silverfish and a dose of neglect. but i managed to save a few handfuls of eclectic titles. their smell still transports me to the basement of ... seeking certainty “uncertain times” is a phrase you hear a lot these days. it was actually in the title of the ala town hall that took place in atlanta last month (ala town hall: library advocacy and core values in uncertain times). political turmoil, uncertainty, divisiveness, and vitriol have so many of us feeling a bit unhinged. when i feel rudderless, adrift, even completely lost at sea, i tend to seek a safer port. i’ve exercised this method personally, geographically, and professionally and it has always served me well. for example, the stability and solid foundation provided by my family gives me solace ... no not google search box, just you (to the tune of “all i want for christmas is you”) (if you need a karaoke track, try this one) i don’t need a lot for freedom, peace, or love, democracy, and i don’t care about the congress or their failed bureaucracy i just want a li-brar-y filled with places just for me a librarian or two no not google search box, just you i don’t want a lot of features search results are too grotesque i don’t care about the systems back behind your reference desk i don’t need to download e-books on the de-vice of my choice noisy ... we are ala i’ve been thinking a lot about governance lately. that said, i will avoid the topic of the recent u.s. election as much as possible, even though it is a factor in what makes me think about governance. instead, i will focus on library governance and what makes it work and not work. spoiler alert: active participation. i am an admitted governance junky, an unapologetic lover of robert’s rules of order, and someone who tries to finds beauty in bureaucratic process. i blame my heritage. i come from a long line of federal government employees, all of us born in the ... cdx server api · webrecorder/pywb wiki · github skip to content sign up why github? features → mobile → actions → codespaces → packages → security → code review → project management → integrations → github sponsors → customer stories → security → team enterprise explore explore github → learn & contribute topics → collections → trending → learning lab → open source guides → connect with others the readme project → events → community forum → github education → github stars program → marketplace pricing plans → compare plans → contact sales → nonprofit → education → in this repository all github ↵ jump to ↵ no suggested jump to results in this repository all github ↵ jump to ↵ in this organization all github ↵ jump to ↵ in this repository all github ↵ jump to ↵ sign in sign up {{ message }} webrecorder / pywb sponsor sponsor webrecorder/pywb watch star fork code issues pull requests actions projects wiki security insights more code issues pull requests actions projects wiki security insights cdx server api jump to bottom eoin kilfeather edited this page jun , · revisions in addition to replay capabilities, pywb also provides an extensive api for querying the capture index (cdx). the api can be used to get information about a range of archive captures/mementos, including filtering, sorting, and pagination for bulk query. the actual archive files (warc/arc) files are not loaded during this query, only the generated cdx index. pywb actually uses this same api internally to perform all index lookups in a consistent way. for example, the following query might return the first results from host http://example.com/* where the mime type is text/html: http://localhost: /coll-cdx?url=http://example.com/*&page= &filter=mime:text/html&limit= usage and configuration the cdx-server command line application starts pywb in cdx server only mode (web archive replay functionality is not loaded, only the index). it can be used the same way as the wayback command line application, including the auto-configuration init. the cdx server functionality can also be enabled when running full-replay with wayback by setting: enable_cdx_api: true api endpoint when enable_cdx_api is set to true, a cdx server endpoint is created for each collection and accessible by adding -cdx to the regular collection path, ex: http://localhost: /pywb// - web archive replay http://localhost: /pywb-cdx - cdx server endpoint http://localhost: /other_collection// - web archive replay http://localhost: /other_collection-cdx - cdx server endpoint the suffix can additionally be customized through the enable_cdx_api setting enable_cdx_api: -index this will make the endpoint be http://localhost: /pywb-index instead api reference unless otherwise specified, all api params are available from pywb . . . url the only required parameter to the cdx server api is the url, ex: http://localhost: /coll-cdx?url=example.com will return a list of captures for 'example.com' from / to setting from= or to= will restrict the results to the given date/time range (inclusive). timestamps may be <= digits and will be padded to either lower or upper bound. for example, ...coll-cdx?url=example.com&from= &to= will return results of example.com that have a timestamp between and available from pywb . . matchtype the cdx server supports the following matchtype exact -- default setting, will return captures that match the url exactly prefix -- return captures that begin with a specified path, eg: http://example.com/path/* host -- return captures which for a begin host (the path segment is ignored if specified) domain -- return captures for the current host and all subdomains, eg. *.example.com as a shortcut, instead of specifying a separate matchtype parameter, wildcards may be used in the url: ...coll-cdx?url=http://example.com/path/* is equivalent to ...coll-cdx?url=http://example.com/path/&matchtype=prefix ...coll-cdx?url=*.example.com is equivalent to ...coll-cdx?url=example.com&matchtype=domain note: if you are using legacy cdx index files which are not surt-ordered, the domain option will not be available. if this is the case, you can use the wb-manager convert-cdx option to easily convert any cdx to latest format` limit setting limit= will limit the number of index lines returned. limit must be set to a positive integer. if no limit is provided, all the matching lines are returned, which may be slow. (if using a zipnum compressed cluster, the page size limit is enforced and no captures are read beyond the single page. see pagination api for more info). sort the sort param can be set as follows: reverse -- will sort the matching captures in reverse order. it is only recommended for exact query as reverse a large match may be very slow. (an optimized version is planned) closest -- setting this option also requires setting closest= where is a specific timestamp to sort by. this option will only work correctly for exact query and is useful for sorting captures based on time distance from a certain timestamp. (pywb uses this option internally for replay in order to fallback to 'next closest' capture if one fails) both options may be combined with limit to return the top n closest, or the last n results. output (json output) setting output=json will return each line as a proper json dictionary. (default format is text which will return the native format of the underlying cdx index, and may not be consistent). using output=json is recommended for extensive analysis and it may become the default option in a future release. filter the filter param can be specified multiple times to filter by specific fields in the cdx index. field names correspond to the fields returned in the json output. filters can be specified as follows: ...coll-cdx?url=example.com/*&filter==mime:text/html&filter=!=status: return captures from example.com/* where mime is text/html and http status is not . ...coll-cdx?url=example.com&matchtype=domain&filter=~url:.*\.php$ return captures from the domain example.com which url ends in .php. the ! modifier before =status indicates negation. the = and ~ modifiers are optional and specify exact resp. regular expression matches. the default (no specific modifier) is to filter whether the query string is contained in the field value. negation and exact/regex modifier may be combined, eg. filter=!~mime:text/.* the formal syntax is: filter=[!][=|~]: with the following modifiers: modifier(s) example description (no modifier) filter=mime:html field "mime" contains string "html" = filter==mime:text/html exact match: field "mime" is "text/html" ~ filter=~mime:.*/html$ regex match: expression matches beginning of field "mime" (cf. re.match) ! filter=!mime:html field "mime" does not contain string "html" != filter=!=mime:text/html field "mime" is not "text/html" !~ filter=!~mime:.*/html expression does not match beginning of field "mime" fl the fl param can be used to specify which fields to include in the output. the standard available fields are usually: urlkey, timestamp, url, mime, status, digest, length, offset, filename if a minimal cdx index is used, the mime and status fields may not be available. additional fields may be introduced in the future, especially in the cdx json format. fields can be comma delimited, for example fl=urlkey,timestamp will only include the urlkey, timestamp and filename in the output. pagination api the cdx server supports an optional pagination api, but it is currently only available when using zipnum compressed index instead of a plain text cdx files. (additional pagination support may be added for cdxj files as well). the pagination api supports the following params: page page is the current page number, and defaults to if omitted. if the page exceeds the number of available pages from the page count query, a error will be returned. pagesize pagesize is an optional parameter which can increase or decrease the amount of data returned in each page. the default setting can be configuration dependent. shownumpages=true this is a special query which, if successful, always returns a json result of the form. the query should be very quick regardless of the size of the query. {"blocks": , "pages": , "pagesize": } in this result: pages is the total number of pages available for this query. the page parameter may be between and pages - pagesize is the total number of zipnum compressed blocks that are read for each page. the default value can be set in the pywb config.yaml via the max_blocks: option. blocks is the actual number of compressed blocks that match the query. this can be used to quickly estimate the total number of captures, within a margin of error. in general, blocks / pagesize + = pages (since there is always at least page even if blocks < pagesize) if changing pagesize, the same value should be used for both the shownumpages query and the regular paged query. ex: use ...pagesize= &shownumpages=true and read pages to get total number of pages use ...pagesize= &page=n to read the n-th pages from to pages- showpagedindex=true when this param is set, the returned data is the secondary index instead of the actual cdx. each line represents a compressed cdx block, and the number of lines returned should correspond to the blocks value in shownumpages query. this query is used internally before reading the actual compressed blocks and should be significantly faster. at this time, this option can not be combined with other query params listed in the api, except for output=json. using output=json is recommended with this query as the default text format may change in the future. pages home additional archive loading options auto configuration and wayback collections manager auto configuration and web archive collections manager cdx index format cdx server api converting media formats distributed archive config fuzzy match rules post request replay public projects using pywb pywb architecture and design pywb component apis pywb configuration pywb proxy mode usage pywb record lookup and revisits ui customization video replay and recording show more pages… clone this wiki locally © github, inc. terms privacy security status docs contact github pricing api training blog about you can’t perform that action at this time. you signed in with another tab or window. reload to refresh your session. you signed out in another tab or window. reload to refresh your session. national digital stewardship alliance - digital library federation dlf about overview calendar faq foundational principles leadership strategic plan membership join the ndsa member orientation members groups overview interest groups content infrastructure standards and practices active working groups communications and publications conference program innovation awards levels of preservation ndsa agenda publications overview levels of preservation ndsa agenda osf repository conference digipres conference past digipres conferences news national digital stewardship alliance an international membership organization that supplies advocacy, expertise, and support for the preservation of digital heritage learn more congratulations to the the levels of digital preservation a digital preservation award winner! about the ndsa is a consortium of partnering organizations, including universities, professional associations, businesses, government agencies, and nonprofit organizations, all committed to the long-term preservation of digital information. members work together to preserve access to our digital heritage. ndsa is hosted by the digital library federation. join ndsa join the ndsa and its interest groups! ndsa agenda challenges and opportunities for digital stewardship. leadership providing strategic leadership to the ndsa. awards recognizing excellence in the field. blog a resource for ndsa members and the broader community. annual conference attend digipres and other ndsa events. latest posts twitter tweets by @ndsa ndsa about members groups calendar social twitter itunes youtube news linkedin contact ndsa c/o clir+dlf south clark street, arlington, va e: ndsa@diglib.org ndsa the ndsa is proudly hosted by the digital library federation at clir. all content on this site is available for re-use under a cc by-sa . international license. dlf none information wants to be free information wants to be free a librarian, writer and educator reflecting on the profession and the tools we use to serve our patrons in all the bad&# ; some good things wow, this has been a hard year. no one&# ;s life has been untouched by between the pandemic and unrelenting proof that the social safety net has been dismantled by late-stage capitalism, the state-sanctioned murders of black and brown people and ensuing protests, the horrendous wildfires that felt like horsemen of the coming climate apocalypse, and a stressful election. it&# ;s horrifying. ... making customizable interactive tutorials with google forms in september, i gave a talk at oregon state university&# ;s instruction librarian get-together about the interactive tutorials i built at pcc last year that have been integral to our remote instructional strategy. i thought i&# ;d share my slides and notes here in case others are inspired by what i did and to share the amazing ... the crushing expectations on working women and where&# ;s my fucking village? on friday and saturday, my twitter feed was full of anger and frustration over a blog post on the alsc (association for library services to children) blog. entitled &# ;how motherhood has influenced me as a children’s librarian,&# ; the post was problematic because it suggested (probably unintentionally) that childless children&# ;s librarians could not connect with patrons as much or have ... recognition doesn&# ;t have to be a zero sum game as usual, the week the library journal movers and shakers were announced, i saw plenty of complaints about the award and, in some cases, awardees. i’ve been reading this sort of hurtful negativity since when i was named a mover and shaker (and a friend of mine wrote a blog comment calling us “the ... thoughts on work, well-being, solidarity, and advocacy in our current&# ; situation i have been wanting to blog for weeks. i have several blog posts i started that i just couldn&# ;t get through. my attention span reminds me of my son&# ;s at age when his teacher delicately suggested we should have him assessed for adhd. it rapidly jumps between various tasks at hand, my family, my ... #lismentalhealth: that time my brain and job tried to kill me happy lis mental health week friends! i want to start this post by recognizing someone who has done a great deal to support library workers&# ; mental health in the face of toxic workplaces, kaetrena davis kendrick. kaetrena has done some incredibly valuable research on low morale and toxic workplaces in librarianship and has created an awesome ... my year in books (and podcasts) this was a pretty good year for me. nothing particularly amazing or wonderful or eventful happened to me, though my son has been such a source of pride and light for me that i sometimes can&# ;t believe i&# ;m his mom. i still live in the same messed up world we all do. my migraines have actually ... when libraries and librarians pretend to be neutral, they often cause harm two recent events made me think (again) about the toxic nature of &# ;library neutrality&# ; and the fact that, more often than not, neutrality is whiteness/patriarchy/cis-heteronormativity/ableism/etc. parading around as neutrality and causing harm to folks from historically marginalized groups. the insidious thing about whiteness and these other dominant paradigms is that they are largely invisible to ... thoughts at mid-career part : where to from here? this is the fifth in a series of essays. you can access the rest here, though it’s not necessary to read them all or in order. &# ;to me, the only habit worth ‘designing for’ is the habit of questioning one’s habitual ways of seeing” -jenny odell, how to do nothing &# ;we have to fight for this world, but we ... thoughts at mid-career part &# ; the cult of productivity: you’re never doing enough this is the fourth in a series of essays. you can access the rest here, though it’s not necessary to read them all or in order. “these days, i just want to slow down. i want to pull the shutters closed and block out the world… the more time i have, the more i realize that all that ... none none none none none none none none none none none none none none none none none none none none linux.conf.au | go glam miniconf about about lca location contact news attend dashboard tickets volunteer financial assistance shirts and swag why should employees attend? code of conduct terms and conditions programme overview schedule miniconfs sessions proposals sponsors go glam miniconf organised by hugh rundle and bonnie wildie when saturday january call for sessions submissions for this miniconf are now closed. view the schedule about this miniconf will explore what’s next for open culture and memory. as covid- swept across the planet, advocates for open access to academic and cultural knowledge finally had their moment in the sun, as glam workers have scrambled to provide greater access to digital and digitised collections for students and researchers. open data about covid- is being shared and tracked globally in real time in an unprecedented effort, and cultural institutions have pivoted to online, providing a steady stream of openly available educational blogs, videos, home-schooling packs, and in one memorable case, even an unexpectedly popular synth-pop song. academic, professional and community conferences - like linux.conf.au itself - have rapidly moved from expensive events based in a single physical and temporal space to become global, online and often asynchronous. at the same time, governments are tightening control of the official archives, reducing funding for cultural and memory institutions, expanding confidentiality provisions that obscure decision making, and australian universities are shedding thousands of staff, including dozens of library and technology staff even as digital learning support needs expand. covid- has seen galleries, archives, libraries and museums (glam) around the world close their doors - in some cases permanently. and under the cover of all this chaos, companies and government are destroying some of the world’s most ancient and precious cultural sites - whether it be rio tinto blowing up an ancient sacred rock cave at jukaan gorge, the victorian government felling birthing trees in djab wurrung country to build a highway, or coastal gaslink and the canadian police forces building a pipeline across unceded wet’suwet’en territories. so what’s next? what should glam workers be thinking about, doing, and supporting? what should foss technologists help to build and maintain? how can open knowledge advocates add the most value in our current moment? how can we use open and free technologies to support communities to maintain strong and living cultures, and defend open and transparent government, science and societies? we invite you to share a project you’re working on, a proposal for a new project, a call to action, a history lesson - or any other presentation that can help to answer “so what’s next?” and help us all to get there. sponsors we thank our sponsors for their generous contribution to linux.conf.au . view all sponsors emperor penguin linux.conf.au jan - online, worldwide timezone: aedt - utc+ back to top © linux.conf.au and linux australia linux is a registered trademark of linus torvalds colophon none none none none planet code lib sun, feb : : + sun, feb : : + code lib c l-planet-admins@library.oregonstate.edu none none none none library map library map beta! home about acknowledgements how you can help code loading, please be patient... made by hugh rundle using open data. play with the layers by using the layer control at the top right of the map. email librarymap@hugh.run if you have any tips or suggestions. you need a modern browser with javascript enabled to use library map. this site does not work in internet explorer. this website and the data it uses licensed cc-by- . none none none none none none librarian of things librarian of things weeknote ( ) § last friday i was interviewed for the podcast the grasscast — a game-themed podcast named after the book, the grasshopper: games, life, and utopia. i ramble a little bit in the episode as i tried to be more open and conversational than concise and correct. but i also spoke that way because for some &# ; continue reading "weeknote ( )" weeknote ( ) i don&# ;t have much that i can report in this week&# ;s note. you are just going to have to take my word that this week, a large amount of my time was spent at meetings pertaining to my library department, my union, and anti-black racism work. § last year, around this same time, some colleagues &# ; continue reading "weeknote ( )" weeknote ( ) hey. i missed last week&# ;s weeknote. but we are here now. § this week i gave a class on searching scientific literature to a group of biology masters students. while i was making my slides comparing the advanced search capabilities of web of science and scopus, i discovered this weird behaviour of google scholar: a &# ; continue reading "weeknote ( )" weeknote ( ) this week&# ;s post is not going to capture my ability to be productive while white supremacists appeared to be ushered in and out of the us capitol building by complicit police and covid- continued to ravage my community because our provincial government doesn&# ;t want to spend money on the most vulnerable. instead, i&# ;m just going &# ; continue reading "weeknote ( )" weeknote ( ) § it looks like andromeda yelton is sharing weeknotes (&# ;this week in ai&# ;). i can&# ;t wait to see what she shares with us all in . § earlier this fall, clarivate analytics&# ;announced that it was moving toward a future that calculated the journal impact factor (jif) based on the date of electronic publication and not &# ; continue reading "weeknote ( )" weeknote ( ) § i don&# ;t have much to report in regards to the work i&# ;ve been doing this week. i tried to get our orcid-ojs plugin to work but there is some small strange bug that needs to be squished. luckily, next week i will have the benefit of assistance from the good people of crkn and &# ; continue reading "weeknote ( )" weeknote ( ) § first off is this recommended read from the november th issue of the new yorker, the rise and fall of getting things done by cal &# ;deep work&# ; newport. as newport himself describes his work, it’s not, however, really about david allen’s productivity system, which longtime readers (and listeners) know i really admire. it’s instead &# ; continue reading "weeknote ( )" weeknote ( ) i had a staycation last week. it took me two days just to catch up on email i received while i was gone. and the only reason i was able to do that in two days is because i had booked the days off as meeting-free so i could attend an online conference. said conference &# ; continue reading "weeknote ( )" weeknote ( ) some things i was up to this past week: i registered for the indigenous mapping workshop which will run nov. - ; had meetings pertaining to servers and surveys; attended regular meetings including that of the university library advisory committee, leddy library department heads, my bi-weekly meeting with library admin, and the wufa grievance committee uploaded &# ; continue reading "weeknote ( )" weeknote ( ) this is my third week of weeknotes and i have to say that the format is agreeing with me. i did a quick search online to see if any other librarians have adopted this particular format and i couldn&# ;t find anyone from the librarian profession so i have yet to become an influencer (*snerk*). i &# ; continue reading "weeknote ( )" none none weeknote ( ) – librarian of things skip to content librarian of things weeknote ( ) § last friday i was interviewed for the podcast the grasscast — a game-themed podcast named after the book, the grasshopper: games, life, and utopia. i ramble a little bit in the episode as i tried to be more open and conversational than concise and correct. but i also spoke that way because for some of the questions, no pat answer came immediately to mind. there was one question that stumped me but in my trying to answer, i think i found something i had not considered before. the question was, what is one bad thing about games? and i tried to convey that, unlike video games where you can play with strangers, most tabletop games are generally constrained by the preferences of your social circles. in order to convince others to spend time on a game that might think is too complicated for them or not for them, you need to have be a successful evangelist. also the episode drifts into chatter about libraries, copyright and ebooks. § this week, i reviewed and published another batch of works for our institutional repository from our department of history that was prepared by our library assistants at leddy at this point, we have reviewed and uploaded the works of half the faculty from this department. i’m hoping to finish the rest this month but i think i have some outstanding h p work that might push the end of this project til march. § this morning i assisted with an online workshop called data analysis and visualization in r for ecologists that was being lead by a colleague of mine. r version . . (“bunny-wunnies freak out”) was released on - - . the release of r . . (“lost library book”) is scheduled for monday - - . § on sunday, i published a short response to “windsor works – an economic development strategy” which is going to city council on monday. why am i writing about this document here? i am mention this here because the proposed strategy (l.i.f.t.) lists the following as potential metric for measuring the strategy’s success… take it from me, someone who knows a quite a bit about citations — the city should use another metric — perhaps one pertaining to local unemployment levels instead. § a viral post from resurfaced on my fb feed this week and unlike most of the posts i read there, this one did spark joy: and it struck me how much i loved that the anti-prom was being at the library. so i started doing some research! it appears to me that some anti-proms are technically better described as alternative proms. these proms have been established as an explicitly safe place where lgbtq young people can enjoy prom. other anti-proms are true morps. i now wonder what other anti-traditions should find a home at the public library. share this: twitter facebook author mita williamsposted on february , february , categories weeknotes leave a reply cancel reply your email address will not be published. required fields are marked * comment name * email * website post navigation previous previous post: weeknote ( ) about me librarian of things is a blog by me, mita williams, who used to blog at new jack librarian until blogger.com finally gave up the ghost. if you don’t have an rss reader, you can subscribe for email delivery through mailchimp. you can learn more about my work at aedileworks.com as well as my other blogs and my weekly newsletter. if you are an editor of a scholarly journal and think that a post could be expanded into a more academic form, please let me know. search for: search recent posts weeknote ( ) weeknote ( ) weeknote ( ) weeknote ( ) weeknote ( ) archives february january december november october september june may april october june may april march january july june may april december june may april november august july meta log in entries feed comments feed wordpress.org librarian of things proudly powered by wordpress none none none none none none brown university library digital technologies skip to content find & borrow articles, journals, & databases subject support hours & locations ask a question now off-campus access library a-z brown university library digital technologies menu and widgets home bdr blacklight digital preservation drupal josiah ocra orcid researchers@brown web wordpress search the dt blog search for: authors ben cail ( ) hector correa ( ) jean rainwater ( ) kerri hicks ( ) kevin powell ( ) ted lawless ( ) adam bradley ( ) birkin diana ( ) bundler . . and homeless accounts this week we upgraded a couple of our applications to ruby . and bundler . . and one of the changes that we noticed was that bundler was complaining about not being able to write to the /opt/local directory. turns out this problem shows up because the account that we use to run our application is a system account that does not have a home folder. this is how the problems shows up: $ su - system_account $ pwd /opt/local $ mkdir test_app $ cd test_app $ pwd /opt/local/test_app $ gem install bundler -v . . $ bundler --version `/opt/local` is not writable. bundler will use `/tmp/bundler - - h lz ' as your home directory temporarily. bundler version . . notice that bundler complains about the /opt/local directory not being writable, that’s because we don’t have home for this user, in fact env $home outputs /opt/local rather than the typical /home/username. although bundler is smart enough to use a temporary folder instead and continue, the net result of this is that if we set a configuration value for bundler in one execution and try to use that configuration value in the next execution bundler won’t be able to find the value that we set in the first execution (my guess is because the value was saved in a temporary folder.) below is an example of this. notice how we set the path value to vendor/bundle in the first command, but then when we inspect the configuration in the second command the configuration does not report the value that we just set: # first - set the path value $ bundle config set path 'vendor/bundle' `/opt/local` is not writable. bundler will use `/tmp/bundler - - okmcg ' as your home directory temporarily. # then - inspect the configuration $ bundle config `/opt/local` is not writable. bundler will use `/tmp/bundler - - r oed ' as your home directory temporarily. settings are listed in order of priority. the top value will be used. ideally the call to bundle config will report the vendor/bundle path that we set, but it does not in this case. in fact if we run bundle install next bundler will install the gems in $gem_path rather than using the custom vendor/bundle directory that we indicated. working around the issue one way to work around this issue is to tell bundler that the home directory is the one from where we are running bundler (i.e. /opt/local/test_app) in our case. # first - set the path value # (no warning is reported) $ home=/opt/local/test_app/ bundle config set path 'vendor/bundle' # then - inspect the configuration $ bundle config `/opt/local` is not writable. bundler will use `/tmp/bundler - - dmgcb ' as your home directory temporarily. settings are listed in order of priority. the top value will be used. path set for your local app (/opt/local/test_app/.bundle/config): "vendor/bundle" notice that we didn’t get a warning in the first command (since we indicated a home directory) and then, even though we didn’t pass a home directory to the second command, our value was picked up and shows the correct value for the path setting (vendor/bundle). so it seems to me that when home is set to a non-writable directory (/opt/local in our case) bundler picks up the values from ./bundle/config if it is available even as it complains about /opt/local not being writable. if we were to run bundle install now it will install the gems in our local vendor/bundle directory. this is good for us, bundler is using the value that we configured for the path setting (even though it still complains that it cannot write to /opt/local.) we could avoid the warning in the second command if we pass the home value here too: $ home=/opt/local/test-app/ bundle config settings are listed in order of priority. the top value will be used. path set for your local app (/opt/local/test-app/.bundle/config): "vendor/bundle" but the fact the bundler picks up the correct values from ./bundle/config when home is set to a non-writable directory was important for us because it meant that when the app runs under apache/passenger it will also work. this is more or less how the configuration for our apps in http.conf looks like, notice that we are not setting the home value. passengerbaseuri /test-app passengeruser system_account passengerruby /opt/local/rubies/ruby- . . /bin/ruby passengerapproot /opt/local/test-app setenv gem_path /opt/local/.gem/ruby/ . . / some final thoughts perhaps a better solution would be to set a home directory for our system_account, but we have not tried that, we didn’t want to make such a wide reaching change to our environment just to please bundler. plus this might be problematic in our development servers where we share the same system_account for multiple applications (this is not a problem in our production servers) we have no idea when this change took effect in bundler. we went from bundler . . (released in october/ ) to bundler . . (released in january/ ) and there were many releases in between. perhaps this was documented somewhere and we missed it. in our particular situation we noticed this issue because one of our gems needed very specific parameters to be built during bundle install. we set those values via a call to bundle config build.mysql --with-mysql-dir=xxx mysql-lib=yyy and those values were lost by the time we ran bundle install and the installation kept failing. luckily we found a work around and were able to install the gem with the specific parameters. posted on july , february , author hcorreacategories programming upgrading from solr to solr a few weeks ago we upgraded the version of solr that we use in our discovery layer, we went from solr . to solr . . although we have been using solr .x in other areas of the library this was a significant upgrade for us because searching is the raison d’être of our discovery layer and we wanted to make sure that the search results did not change in unexpected ways with the new field and server configurations in solr. all in all the process went smooth for our users. this blog post elaborates on some of the things that we had to do in order to upgrade. managed schema this is the first solr that we setup to use the managed-schema feature in solr. this allows us to define field types and fields via the schema api rather than by editing xml files. all in all this was a good decision and it allows us to recreate our solr instances by running a shell script rather than by copying xml files. this feature was very handy during testing when we needed to recreate our solr core for testing purposes multiple times. you can see the script that we use to recreate our solr core in github. we are still tweaking how we manage updates to our schema. for now we are using a low-tech approach in which we create small scripts to add fields to the schema that is conceptually similar to what rails does with database migrations, but our approach is still very manual. default field definitions the default field definitions in solr are different from the default field definitions in solr , this is not surprising given that we skipped two major versions of solr, but it was one one the hardest things to reconcile. our solr was setup and configured many years ago and the upgrade forced us to look very close into exactly what kind of transformations we were doing to our data and decide what should be modified in solr to support the solr behavior versus what should be updated to use new solr features. our first approach was to manually inspect the “schema.xml” in solr and compare it with the “managed-schema” file in solr which is also an xml file. we soon found that this was too cumbersome and error prone. but we found the output of the lukerequesthandler to be much more concise and easier to compare between the versions of solr, and lucky us, the output of the lukerequesthandler is identical in both versions of solr! using the lukerequesthandler we dumped our solr schema to xml files and compare those files with a traditional file compare tool, we used the built-in file compare option in vs code but any file compare tool would do. these are the commands that we used to dump the schema to xml files: curl http://solr- -url/admin/luke?numterms= > luke .xml curl http://solr- -url/admin/luke?numterms= > luke .xml the output of the lukerequesthandler includes both the type of field (e.g. string) and the schema definition (single value vs multi-value, indexed, tokenized, et cetera.) string --sd------------l another benefit of using the lukerequesthandler instead of going by the fields defined in schema.xml is that the lukerequesthandler only outputs fields that are indeed used in the solr core, whereas schema.xml lists fields that were used at one point even if we don’t use them anymore. icufoldingfilter in solr a few of the default field types used the icufoldingfilter which handles diacritics so that a word like “méxico” is equivalent to “mexico”. this filter used to be available by default in a solr installation but that is not the case anymore. in solr icufoldingfilter is not enabled by default and you must edit your solrconfig.xml as indicated in the documentation to enable it (see previous link). and then you can use it in a field type by adding it as a filter: curl -x post -h 'content-type:application/json' --data-binary '{ "add-field-type" : { "name":"text_search", "class":"solr.textfield", "analyzer" : { "tokenizer":{"class":"solr.standardtokenizerfactory"}, "filters":[ {"class":"solr.icufoldingfilterfactory"}, ... ] } } }' $solr_core_url/schema handle select handleselect is a parameter that is defined in the solrconfig.xml and in previous versions of solr it used to default to true but starting in solr it defaults to false. the version of blacklight that we are using ( . ) expects this value to be true. this parameter is what allows blacklight to use a request handler like “search” (without a leading slash) instead of “/search”. enabling handleselect is easy, just edit the requestdispatcher setting in the solrconfig.xml localparams and dereferencing our current version of blacklight uses localparams and dereferencing heavily and support for these two features changed drastically in solr . . this is a good enhancement in solr but it caught us by surprise. the gist of the problem is that if the solrconfig.xml sets the query parser to dismax or edismax then solr will not recognize a query like this: {!qf=$title_qf} we tried several workarounds and settled on setting the default parser (deftype) in solrconfig.xml to lucene and requesting edismax explicitly from the client application: {!type=dismax qf=$title_qff}coffee&df=id it’s worth nothing that passing deftype as a normal query string parameter to change the parser did not work for us for queries using localparams and dereferencing. stop words one of the settings that we changed in our new field definitions was the use of stop words. we are now not using stop words when indexing title fields. this was one of the benefits of us doing a full review of each one of our field types and tweak them during the upgrade. the result is that now searches for titles that are only stop words (like “there there”) return the expected results. validating results to validate that our new field definitions and server side configuration in solr were compatible with that we had in solr we did several kinds of tests, some of them manual and others automated. we have small suite of unit tests that jeanette norris and ted lawless created years ago and that we still use to validate some well known scenarios that we want to support. you can see those “relevancy” tests in our github repository. we also captured thousands of live searches from our discovery layer using solr and replayed them with solr to make sure that the results of both systems were compatible. to determine that results were compatible we counted how many of the top results, top , and top were included in the results of both solr instances. the following picture shows an example of how the results looks like. the code that we used to run the searches on both solr and generate the table is on our github repo. cjk searches the main reason for us to upgrade from solr to solr was to add support for chinese, japanese, and korean (cjk) searches. the way our solr index was created we did not support searches in these languages. in our solr core we are using the built-in cjk fields definitions and our results are much better. this will be the subject of future blog post. stay tuned. posted on january , february , author hcorreacategories solrtags blacklight, josiah, solr pypi packages recently, we published two python packages to pypi: bdrxml and bdrcmodels. no one else is using those packages, as far as i know, and it takes some effort to put them up there, but there are benefits from publishing them. putting a package on pypi makes it easier for other code we package up to depend on bdrxml. for our indexing package, we can switch from this: ‘bdrxml @ https://github.com/brown-university-library/bdrxml/archive/v . a .zip#sha = ed ee a cbb fe c f ad c’, to this: ‘bdrxml>= . ’, in setup.py, which is simpler. this also lets us using python’s package version checking to not pin bdrxml to just one version, which is helpful when we embed the indexing package in another project that may use a different version of bdrxml. publishing these first two packages also gave us experience, which will help if we publish more packages to pypi. posted on june , author ben cailcategories uncategorized new riamco website a few days ago we released a new version of the rhode island archival and manuscript collections online (riamco) website. the new version is a brand new codebase. this post describes a few of the new features that we implemented as part of the rewrite and how we designed the system to support them. the riamco website hosts information about archival and manuscript collections in rhode island. these collections (also known as finding aids) are stored as xml files using the encoded archival description (ead) standard and indexed into solr to allow for full text searching and filtering. look and feel the overall look and feel of the riamco site is heavily influenced by the work that the folks at the nyu libraries did on their site. like nyu’s site and brown’s discovery tool the riamco site uses the typical facets on the left, content on the right style that is common in many library and archive websites. below a screenshot on how the main search page looks like: architecture our previous site was put together over many years and it involved many separate applications written in different languages: the frontend was written in php, the indexer in java, and the admin tool in (python/django). during this rewrite we bundled the code for the frontend and the indexer into a single application written in ruby on rails. [as of september th, the rails application also provides the admin interface.] you can view a diagram of this architecture and few more notes about it on this document. indexing like the previous version of the site, we are using solr to power the search feature of the site. however, in the previous version each collection was indexed as a single solr document whereas in the new version we are splitting each collection into many solr documents: one document to store the main collection information (scope, biographical info, call number, et cetera), plus one document for each item in the inventory of the collection. this new indexing strategy significantly increased the number of solr documents that we store. we went from from + solr documents (one for each collection) to , + solr documents (one for each item in the inventory of those collections). the advantage of this approach is that now we can search and find items at a much granular level than we did before. for example, we can tell a user that we found a match on “box he- folder ” of the harris ephemera collection for their search on blue moon rather than just telling them that there is a match somewhere in the boxes ( , folders) in the “harris ephemera” collection. in order to keep the relationship between all the solr documents for a given collection we are using an extra ead_id_s field to store the id of the collection that each document belongs to. if we have a collection “a” with three items in the inventory they will have the following information in solr: {id: "a", ead_id_s: "a"} // the main collection record {id: "a- ", ead_id_s: "a"} // item in the inventory {id: "a- ", ead_id_s: "a"} // item in the inventory {id: "a- ", ead_id_s: "a"} // item in the inventory this structure allows us to use the result grouping feature in solr to group results from a search into the appropriate collection. with this structure in place we can then show the results grouped by collection as you can see in the previous screenshot. the code to index our ead files into solr is on the ead class. we had do add some extra logic to handle cases when a match is found only on a solr document for an inventory item (but not on the main collection) so that we can also display the main collection information along the inventory information in the search results. the code for this is on the search_grouped() function of the search class. hit highlighting another feature that we implemented on the new site is hit highlighting. although this is a feature that solr supports out of the box there is some extra coding that we had to do to structure the information in a way that makes sense to our users. in particular things get tricky when the hit was found in a multi value field or when solr only returns a snippet of the original value in the highlights results. the logic that we wrote to handle this is on the searchitem class. advanced search we also did an overhaul to the advanced search feature. the layout of the page is very typical (it follows the style used in most blacklight applications) but the code behind it allows us to implement several new features. for example, we allow the user to select any value from the facets (not only one of the first values for that facet) and to select more than one value from those facets. we also added a “check” button to show the user what kind of boolean expression would be generated for the query that they have entered. below is a screenshot of the results of the check syntax for a sample query. there are several tweaks and optimizations that we would like to do on this page, for example, opening the facet by format is quite slow and it could be optimized. also, the code to parse the expression could be written to use a more standard tokenizer/parser structure. we’ll get to that later on… hopefully : ) individual finding aids like on the previous version of the site, the rendering of individual finding aids is done by applying xslt transformations to the xml with the finding aid data. we made a few tweaks to the xslt to integrate them on the new site but the vast majority of the transformations came as-is from the previous site. you can see the xslt files in our github repo. it’s interesting that github reports that half of the code for the new site is xslt: % xslt, % html, and % ruby. keep in mind that these numbers do not take into account the ruby on rails code (which is massive.) source code the source code for the new application is available in github. acknowledgements although i wrote the code for the new site, there were plenty of people that helped me along the way in this implementation, in particular karen eberhart and joe mancino. karen provided the specs for the new site, answered my many questions about the structure of ead files, and suggested many improvements and tweaks to make the site better. joe helped me find the code for the original site and indexer, and setup the environment for the new one. posted on june , february , author hcorreacategories programming, riamco, solrtags solr deploying with shiv i recently watched a talk called “containerless django – deploying without docker”, by peter baumgartner. peter lists some benefits of docker: that it gives you a pipeline for getting code tested and deployed, the container adds some security to the app, state can be isolated in the container, and it lets you run the exact same code in development and production. peter also lists some drawbacks to docker: it’s a lot of code that could slow things down or have bugs, docker artifacts can be relatively large, and it adds extra abstractions to the system (eg. filesystem, network). he argues that an ideal deployment would include downloading a binary, creating a configuration file, and running it (like one can do with compiled c or go programs). peter describes a process of deploying django apps by creating a zipapp using shiv and goodconf, and deploying it with systemd constraints that add to the security. he argues that this process achieves most of the benefits of docker, but more simply, and that there’s a sweet spot for application size where this type of deploy is a good solution. i decided to try using shiv with our image server loris. i ran the shiv command “shiv -o loris.pyz .”, and i got the following error: user “loris” and or group “loris” do(es) not exist. please create this user, e.g.: `useradd -d /var/www/loris -s /sbin/false loris` the issue is that in the loris setup.py file, the install process not only checks for the loris user as shown in the error, but it also sets up directories on the filesystem (including setting the owner and permission, which requires root permissions). i submitted a pr to remove the filesystem setup from the python package installation (and put it in a script the user can run), and hopefully in the future it will be easier to package up loris and deploy it different ways. posted on may , author ben cailcategories bdr, programming checksums in the bdr, we calculate checksums automatically on ingest (fedora provides that functionality for us), so all new content binaries going into the bdr get a checksum, which we can go back and check later as needed. we can also pass checksums into the bdr api, and then we verify that fedora calculates the same checksum for the ingested file, which shows that the content wasn’t modified since the first checksum was calculated. we have only been able to use md checksums, but we want to be able to use more checksum types. this isn’t a problem for fedora, which can calculate multiple checksum types, such as md , sha , sha , and sha . however, there is a complicating factor – if fedora gets a checksum mismatch, by default it returns a response code with no message, so we can’t tell whether it was a checksum mismatch or some other server error. thanks to ben armintor, though, we found that we can update our fedora configuration so it returns the checksum mismatch information. another issue in this process is that we use eulfedora (which doesn’t seem to be maintained anymore). if a checksum mismatch happens, it raises a digitalobjectsavefailure, but we want to know that there was a checksum mismatch. we forked eulfedora and exposed the checksum mismatch information. now we can remove some extra code that we had in our apis, since more functionality is handled in fedora/eulfedora, and we can use multiple checksum types. posted on march , author ben cailcategories bdr exporting django data we recently had a couple cases where we wanted to dump the data out of a django database. in the first case (“tracker”), we were shutting down a legacy application, but needed to preserve the data in a different form for users. in the second case (“deposits”), we were backing up some obsolete data before removing it from the database. we handled the processes in two different ways. tracker for the tracker, we used an export script to extract the data. here’s a modified version of the script: def export_data(): now = datetime.datetime.now() dir_name = 'data_%s_%s_%s' % (now.year, now.month, now.day) d = os.mkdir(dir_name) file_name = os.path.join(dir_name, 'tracker_items.dat') with open(file_name, 'wb') as f: f.write(u'\u f'.join([ 'project name', 'container identifier', 'container name', 'identifier', 'name', 'dimensions', 'note', 'create digital surrogate', 'qc digital surrogate', 'create metadata record', 'qc metadata record', 'create submission package']).encode('utf ')) f.write('\u e'.encode('utf ')) for project in models.project.objects.all(): for container in project.container_set.all(): print(container) for item in container.item_set.all(): data = u'\u f'.join([ project.name.strip(), container.identifier.strip(), container.name.strip(), item.identifier.strip(), item.name.strip(), item.dimensions.strip(), item.note.strip() ]) item_actions = u'\u f'.join([str(item_action) for item_action in item.itemaction_set.all().order_by('id')]) line_data = u'%s\u f%s\u e' % (data, item_actions) f.write(line_data.encode('utf ')) as you can see, we looped through different django models and pulled out fields, writing everything to a file. we used the unicode record and unit separators as delimiters. one advantage of using those is that your data can have commas, tabs, newlines, … and it won’t matter. you still don’t have to quote or escape anything. then we converted the data to a spreadsheet that users can view and search: import openpyxl workbook = openpyxl.workbook() worksheet = workbook.active with open('tracker_items.dat', 'rb') as f: data = f.read() lines = data.decode('utf ').split('\u e') print(len(lines)) print(lines[ ]) print(lines[- ]) for line in lines: fields = line.split('\u f') worksheet.append(fields) workbook.save('tracker_items.xlsx') deposits for the deposits project, we just used the built-in django dumpdata command: python manage.py dumpdata -o data_ .dat that output file could be used to load data back into a database if needed. posted on march , author ben cailcategories uncategorized searching for hierarchical data in solr recently i had to index a dataset into solr in which the original items had a hierarchical relationship among them. in processing this data i took some time to look into the ancestor_path and descendent_path features that solr provides out of the box and see if and how they could help to issue searches based on the hierarchy of the data. this post elaborates on what i learned in the process. let’s start with some sample hierarchical data to illustrate the kind of relationship that i am describing in this post. below is a short list of databases and programming languages organized by type. databases ├─ relational │ ├─ mysql │ └─ postgresql └─ document ├─ solr └─ mongodb programming languages └─ object oriented ├─ ruby └─ python for the purposes of this post i am going to index each individual item shown in the hierarchy, not just the children items. in other words i am going to create solr documents: one for “databases”, another for “relational”, another for “mysql”, and so on. each document is saved with an id, a title, and a path. for example, the document for “databases” is saved as: { "id": " ", "title_s": "databases", "x_ancestor_path": "db", "x_descendent_path": "db" } and the one for “mysql” is saved as: { "id": " ", "title_s": "mysql", "x_ancestor_path": "db/rel/mysql", "x_descendent_path": "db/rel/mysql" } the x_ancestor_path and x_descendent_path fields in the json data represent the path for each of these documents in the hierarcy. for example, the top level “databases” document uses the path “db” where the lowest level document “mysql” uses “db/rel/mysql”. i am storing the exact same value on both fields so that later on we can see how each of them provides different features and addresses different use cases. ancestor_path and descendent_path the ancestor_path and descendent_path field types come predefined in solr. below is the definition of the descendent_path in a standard solr core: $ curl http://localhost: /solr/your-core/schema/fieldtypes/descendent_path { ... "indexanalyzer":{ "tokenizer":{ "class":"solr.pathhierarchytokenizerfactory", "delimiter":"/"}}, "queryanalyzer":{ "tokenizer":{ "class":"solr.keywordtokenizerfactory"}}}} notice how it uses the pathhierarchytokenizerfactory tokenizer when indexing values of this type and that it sets the delimiter property to /. this means that when values are indexed they will be split into individual tokens by this delimiter. for example the value “db/rel/mysql” will be split into “db”, “db/rel”, and “db/rel/mysql”. you can validate this in the analysis screen in the solr admin tool. the ancestor_path field is the exact opposite, it uses the pathhierarchytokenizerfactory at query time and the keywordtokenizerfactory at index time. there are also two dynamic field definitions *_descendent_path and *_ancestor_path that automatically create fields with these types. hence the wonky x_descendent_path and x_ancestor_path field names that i am using in this demo. finding descendants the descendent_path field definition in solr can be used to find all the descendant documents in the hierarchy for a given path. for example, if i query for all documents where the descendant path is “db” (q=x_descendent_path:db) i should get all document in the “databases” hierarchy, but not the ones under “programming languages”. for example: $ curl "http://localhost: /solr/your-core/select?q=x_descendent_path:db&fl=id,title_s,x_descendent_path" { ... "response":{"numfound": ,"start": ,"docs":[ { "id":" ", "title_s":"databases", "x_descendent_path":"db"}, { "id":" ", "title_s":"relational", "x_descendent_path":"db/rel"}, { "id":" ", "title_s":"mysql", "x_descendent_path":"db/rel/mysql"}, { "id":" ", "title_s":"postgresql", "x_descendent_path":"db/rel/pg"}, { "id":" ", "title_s":"document", "x_descendent_path":"db/doc"}, { "id":" ", "title_s":"mongodb", "x_descendent_path":"db/doc/mongo"}, { "id":" ", "title_s":"solr", "x_descendent_path":"db/doc/solr"}] }} finding ancestors the ancestor_path not surprisingly can be used to achieve the reverse. given the path of a given document we can query solr to find all its ancestors in the hierarchy. for example if i query solr for the documents where x_ancestor_path is “db/doc/solr” (q=x_ancestor_path:db/doc/solr) i should get “databases”, “document”, and “solr” as shown below: $ curl "http://localhost: /solr/your-core/select?q=x_ancestor_path:db/doc/solr&fl=id,title_s,x_ancestor_path" { ... "response":{"numfound": ,"start": ,"docs":[ { "id":" ", "title_s":"databases", "x_ancestor_path":"db"}, { "id":" ", "title_s":"document", "x_ancestor_path":"db/doc"}, { "id":" ", "title_s":"solr", "x_ancestor_path":"db/doc/solr"}] }} if you are curious how this works internally, you could issue a query with debugquery=true and look at how the query value “db/doc/solr” was parsed. notice how solr splits the query value by the / delimiter and uses something called synonymquery() to handle the individual values as synonyms: $ curl "http://localhost: /solr/your-core/select?q=x_ancestor_path:db/doc/solr&debugquery=true" { ... "debug":{ "rawquerystring":"x_ancestor_path:db/doc/solr", "parsedquery":"synonymquery(synonym(x_ancestor_path:db x_ancestor_path:db/doc x_ancestor_path:db/doc/solr))", ... } one little gotcha given that solr is splitting the path values by the / delimiter and that we can see those values in the analysis screen (or when passing debugquery=true) we might expect to be able to fetch those values from the document somehow. but that is not the case. the individual tokens are not stored in a way that you can fetch them, i.e. there is no way for us to fetch the individual “db”, “db/doc”, and “db/doc/solr” values when fetching document id “ ”. in hindsight this is standard solr behavior but something that threw me off initially. posted on january , february , author hcorreacategories programming, solr monitoring passenger’s requests in queue over time as i mentioned in a previous post we use phusion passenger as the application server to host our ruby applications. a while ago upon the recommendation of my coworker ben cail i created a cron job that calls passenger-status every minutes to log the status of passenger in our servers. below is a sample of the passenger-status output: version : . . date : mon jul : : - instance: x dq ux (apache/ . . (unix) dav/ phusion_passenger/ . . ) ----------- general information ----------- max pool size : app groups : processes : requests in top-level queue : ----------- application groups ----------- /path/to/our/app: app root: /path/to/our/app requests in queue: * pid: sessions: processed: uptime: d h m s cpu: % memory : m last used: s ag * pid: sessions: processed: uptime: h m s cpu: % memory : m last used: s ago * pid: sessions: processed: uptime: h m s cpu: % memory : m last used: s ago * pid: sessions: processed: uptime: h m s cpu: % memory : m last used: s ago * pid: sessions: processed: uptime: m s cpu: % memory : m last used: s ago * pid: sessions: processed: uptime: m s cpu: % memory : m last used: s ago our cron job to log this information over time is something like this: /path/to/.gem/gems/passenger- . . /bin/passenger-status >> ./logs/passenger_status.log last week we had some issues in which our production server was experiencing short outages. upon review we noticed that we were having a unusual amount of traffic coming to our server (most of it from crawlers submitting bad requests.) one of the tools that we used to validate the status of our server was the passenger_status.log file created via the aforementioned cron job. the key piece of information that we use is the “requests in queue” value highlighted above. we parsed this value of out the passenger_status.log file to see how it changed in the last days. the result showed that although we have had a couple of outages recently the number of “requests in queue” dramatically increased about two weeks ago and it had stayed high ever since. the graph below shows what we found. notice how after august th the value of “requests in queue” has been constantly high, whereas before august th it was almost always zero or below . we looked closely to our apache and rails logs and determined the traffic that was causing the problem. we took a few steps to handle it and now our servers are behaving as normal again. notice how we are back to zero requests in queue on august st in the graph above. the ruby code that we use to parse our passenger_status.log file is pretty simple, it just grabs the line with the date and the line with the number of requests in queue, parses their values, and outputs the result to a tab delimited file that then we can use to create a graph in excel or rawgraphs. below is the ruby code: require "date" log_file = "passenger_status.log" excel_date = true def date_from_line(line, excel_date) index = line.index(":") return nil if index == nil date_as_text = line[index+ ..- ].strip # thu aug : : - datetime = datetime.parse(date_as_text).to_s # - - t : : - : if excel_date return datetime[ .. ] + " " + datetime[ .. ] # - - : end datetime end def count_from_line(line) return line.gsub("requests in queue:", "").to_i end puts "timestamp\trequest_in_queue" date = "n/a" file.readlines(log_file).each do |line| if line.start_with?("date ") date = date_from_line(line, excel_date) elsif line.include?("requests in queue:") request_count = count_from_line(line) puts "\"#{date}\"\t#{request_count}" end end in this particular case the number of requests in queue was caused by bad/unwanted traffic. if the increase in traffic had been legitimate we would have taken a different route, like adding more processes to our passenger instance to handle the traffic. posted on september , february , author hcorreacategories blacklight, josiah, programming, web looking at the oxford common filesystem layout (ocfl) currently, the bdr contains about tb of content. the storage layer is fedora , and the data is stored internally by fedora (instead of being stored externally). however, fedora is end-of-life. this means that we either maintain it ourselves, or migrate to something else. however, we don’t want to migrate tb, and then have to migrate it again if we change software again. we’d like to be able to change our software, without migrating all our data. this is where the oxford common filesystem layout (ocfl) work is interesting. ocfl is an effort to define how repository objects should be laid out on the filesystem. ocfl is still very much a work-in-progress, but the “need” section of the specification speaks directly to what i described above. if we set up our data using ocfl, hopefully we can upgrade and change our software as necessary without having to move all the data around. another benefit of the ocfl effort is that it’s work being done by people from multiple institutions, building on other work and experience in this area, to define a good, well-thought-out layout for repository objects. finally, using a common specification for the filesystem layout of our repository means that there’s a better chance that other software will understand how to interact with our files on disk. the more people using the same filesystem layout, the more potential collaborators and applications for implementing the ocfl specification – safely creating, updating, and serving out content for the repository. posted on july , author ben cailcategories bdr posts navigation page page … page next page proudly powered by wordpress weeknote ( ) – librarian of things skip to content librarian of things weeknote ( ) i don’t have much that i can report in this week’s note. you are just going to have to take my word that this week, a large amount of my time was spent at meetings pertaining to my library department, my union, and anti-black racism work. § last year, around this same time, some colleagues from the university and i organized an speaking event called safer communities in a ‘smart tech’ world: we need to talk about amazon ring in windsor. windsor’s mayor proposes we be the first city in canada to buy into the ring network. as residents of windsor, we have concerns with this potential project. seeing no venue for residents of windsor to share their fears of surveillance and loss of privacy through this private-partnership, we hosted an evening of talks on january nd, at the performance hall at the university of windsor’s school of creative arts windsor armories building. our keynote speaker was chris gilliard, heard recently on cbc’s spark. since that evening, we have been in the media raising our concerns, asking questions, and encouraging others to do the same. the city of windsor has yet to have entered an agreement with amazon ring. this is good news. this week, the city of windsor announced that it has entered a one-year deal partnership with ford mobility canada to share data and insights via ford’s safety insights platform. i don’t think this is good news for reasons outlined in this post called safety insights, data privacy, and spatial justice. § this week i learned a neat tweetdeck hack. if set up a search as column, you can limit the results for that term using the number of ‘engagements’: § § i haven’t read this but i have it bookmarked for potential future reference: the weaponization of web archives: data craft and covid- publics: an unprecedented volume of harmful health misinformation linked to the coronavirus pandemic has led to the appearance of misinformation tactics that leverage web archives in order to evade content moderation on social media platforms. here we present newly identified manipulation techniques designed to maximize the value, longevity, and spread of harmful and non-factual content across social media using provenance information from web archives and social media analytics. after identifying conspiracy content that has been archived by human actors with the wayback machine, we report on user patterns of “screensampling,” where images of archived misinformation are spread via social platforms. we argue that archived web resources from the internet archive’s wayback machine and subsequent screenshots contribute to the covid- “misinfodemic” in platforms. understanding these manipulation tactics that use sources from web archives reveals something vexing about information practices during pandemics—the desire to access reliable information even after it has been moderated and fact-checked, for some individuals, will give health misinformation and conspiracy theories more traction because it has been labeled as specious content by platforms. § i’m going to leave this tweet here because i might pick up this thread in the future: this reminds me of a talk given in by data & society founder and president, danah boyd called you think you want media literacy… do you? this essay still haunts me, largely because we still don’t have good answers for the questions that dr. boyd asks of us and the stakes have only gotten higher. share this: twitter facebook author mita williamsposted on january , january , categories weeknotes leave a reply cancel reply your email address will not be published. required fields are marked * comment name * email * website post navigation previous previous post: weeknote ( ) next next post: weeknote ( ) about me librarian of things is a blog by me, mita williams, who used to blog at new jack librarian until blogger.com finally gave up the ghost. if you don’t have an rss reader, you can subscribe for email delivery through mailchimp. you can learn more about my work at aedileworks.com as well as my other blogs and my weekly newsletter. if you are an editor of a scholarly journal and think that a post could be expanded into a more academic form, please let me know. search for: search recent posts weeknote ( ) weeknote ( ) weeknote ( ) weeknote ( ) weeknote ( ) archives february january december november october september june may april october june may april march january july june may april december june may april november august july meta log in entries feed comments feed wordpress.org librarian of things proudly powered by wordpress software development at royal danish library | a peekhole into the life of the software development department at the royal danish library software development at royal danish library a peekhole into the life of the software development department at the royal danish library skip to content home about net archive search ← older posts which type bug? posted on june , by toke eskildsen a light tale of bug hunting an out of memory problem with solrcloud. the setup and the problem at the royal danish library we provide full text search for the danish netarchive. the heavy lifting is done in a single collection solrcloud made up of shards (for a total of tb / billion documents). all queries are issued to a solr instance with an empty shard, with the sole responsibility of aggregating responses from the real shards. one of the frontends is solrwayback, which is a javascript application backed by a middle layer acting as an advanced proxy; issuing searches, rewriting html, doing streaming exports and so. the problem this time was that the aggregating solr node occasionally crashed with an out of memory error, where occasionally means that it sometimes took months to crash, sometimes days. clues and analysis access to the netarchive search is strictly controlled, so there were no chance of denial of service or fimilar foul play. log analysis showed modest activity (a maximum of concurrent searches) around the time of the latest crash. the queries themselves were run-of-the-mill, but the crashing queries themselves were not logged, as solr only logs the query when is has been completed, not when it starts. the garbage collection logs showed that everything was a-ok, right up til the time when everything exploded in progressively longer collections, culminating in a second stop-the-world and no heap space left. heap graph with stop-the-world gc as red triangles, courtesy of gceasy.io should be simple to pinpoint, right? and (plot twist) for once it was! of course we chased the usual red herrings, but ultimately “dissect the logs around the problematic time slots” won the day. pop quiz: what is wrong with the log entries below? (meticulously unearthed from too many nearly-but-not-fully-similar entries and with timestamps adjusted to match graph-timezone). ) - - : : . info (qtp - ) [c:ns s:shard r:core_node x:ns _shard _replica_n ] o.a.s.c.s.request [ns _shard _replica_n ] webapp=/solr path=/select params={q=facebook.com&facet.field=domain&facet.field=content_type_norm&facet.field=type&facet.field=crawl_year&facet.field=status_code&facet.field=public_suffix&hl=on&indent=true&fl=id,score,title,hash,source_file_path,source_file_offset,url,url_norm,wayback_date,domain,content_type,crawl_date,content_type_norm,type&start= &q.op=and&fq=record_type:response+or+record_type:arc&fq=domain:"facebook.com"+and+crawl_year:" "&rows= &wt=json&facet=true&f.crawl_year.facet.limit= } hits= status= qtime= ) - - : : . info (qtp - ) [c:ns s:shard r:core_node x:ns _shard _replica_n ] o.a.s.c.s.request [ns _shard _replica_n ] webapp=/solr path=/select params={q=facebook.com&facet.field=domain&facet.field=content_type_norm&facet.field=type&facet.field=crawl_year&facet.field=status_code&facet.field=public_suffix&hl=on&indent=true&fl=id,score,title,hash,source_file_path,source_file_offset,url,url_norm,wayback_date,domain,content_type,crawl_date,content_type_norm,type&start= &q.op=and&fq=record_type:response+or+record_type:arc&fq=domain:"facebook.com"+and+crawl_year:" "&rows= &wt=json&facet=true&f.crawl_year.facet.limit= } hits= status= qtime= ) - - : : . info (qtp - ) [c:ns s:shard r:core_node x:ns _shard _replica_n ] o.a.s.c.s.request [ns _shard _replica_n ] webapp=/solr path=/select params={q=facebook.com&facet.field=domain&facet.field=content_type_norm&facet.field=type&facet.field=crawl_year&facet.field=status_code&facet.field=public_suffix&hl=on&indent=true&fl=id,score,title,hash,source_file_path,source_file_offset,url,url_norm,wayback_date,domain,content_type,crawl_date,content_type_norm,type&start= &q.op=and&fq=record_type:response+or+record_type:arc&fq=domain:"facebook.com"+and+crawl_year:" "&rows= &wt=json&facet=true&f.crawl_year.facet.limit= } hits= status= qtime= ) - - : : . info (qtp - ) [c:ns s:shard r:core_node x:ns _shard _replica_n ] o.a.s.c.s.request [ns _shard _replica_n ] webapp=/solr path=/select params={q=facebook.com&facet.field=domain&facet.field=content_type_norm&facet.field=type&facet.field=crawl_year&facet.field=status_code&facet.field=public_suffix&hl=on&indent=true&fl=id,score,title,hash,source_file_path,source_file_offset,url,url_norm,wayback_date,domain,content_type,crawl_date,content_type_norm,type&start= &q.op=and&fq=record_type:response+or+record_type:arc&fq=domain:"facebook.com"+and+crawl_year:" "&rows= &wt=json&facet=true&f.crawl_year.facet.limit= } hits= status= qtime= ) - - : : . info (qtp - ) [c:ns s:shard r:core_node x:ns _shard _replica_n ] o.a.s.c.s.request [ns _shard _replica_n ] webapp=/solr path=/select params={q=facebook.com&facet.field=domain&facet.field=content_type_norm&facet.field=type&facet.field=crawl_year&facet.field=status_code&facet.field=public_suffix&hl=on&indent=true&fl=id,score,title,hash,source_file_path,source_file_offset,url,url_norm,wayback_date,domain,content_type,crawl_date,content_type_norm,type&start= &q.op=and&fq=record_type:response+or+record_type:arc&fq=domain:"facebook.com"+and+crawl_year:" "&rows= &wt=json&facet=true&f.crawl_year.facet.limit= } hits= status= qtime= if your answer was “hey, what’s up with start!?” then you are now officially a big search analyst. your badge will arrive shortly. for those not catching it (that included me for a long time): a search is issued with a query for facebook material from with the parameters start= &rows= (corresponding to page in a ui which shows results/page). response time is seconds. the same query is repeated, this time with start= &rows= . if the intent was to go to page in the ui, we would expect start= &rows= . response time is seconds. the query is changed to facebook material from , still with start= &rows= . seems like someone’s url hacking. response time is ½ seconds. same query as in # , but now with start= &rows= . response time jumps to seconds. the query is changed to facebook material from , with the previous start= &rows= intact. response time jumps to seconds. locating the error time to inspect the code responsible for the paging: if (this.start + < this.totalhits) { this.start = this.start + ; } seems innocent enough and when we tested by pressing “next” a few times in solrwayback, it did indeed behave exemplary: start= , start= , start= and so on. looking further down we encounter this nugget: /* method used on creation, reload and route change to get query parameters */ getqueryparams:function(){ this.myquery = this.$route.query.query; this.start= this.$route.query.start; this.filters = this.$route.query.filter; ... a quick appliance of console.log(typeof this.start) in the right place tells us that when the ui page is reloaded, which happens when the url is changed by hand, the type of this.start becomes a string! loosely typed languages is a taste not acquired by your humble author. back to the code for skipping to the next page: this.start = this.start + ; if this.start is to begin with and if it is a string, we suddenly have " " + , which javascript handles by casting the number to the string : " " + " " = " ". that translates to page instead of page , which of course is not what the user wants, but how does it become a memory problem? solrcloud internals and the smoking gun the solrcloud for netarchive search is a distributed one (remember the shards?), so when documents starting at position are needed, the master must request start= &rows= document representations from each shard, sort them and deliver documents - . for our setup that means holding up to * = million document representations in memory. the master node has one job and this it it, so it handles the load. yes, it bumps heap requirements temporarily with a gigabyte or two, but that’s okay. it still delivers the result in seconds. so what happens when the user presses next again? yes, " " + = " ". that’s a factor right there, as we move decimal places. and master has -xmx= g… fortunately the logged request only matched million documents, so the master solr got by with a gb bump to the heap (the first spike in the graph) at that time. knowing what to look for (start=xxxx, where xxxx is at least digits), it is simple to find the last relevant log entry before the crash: grep "start=[ - ][ - ][ - ][ - ]" solr.log. - - : : . info (qtp - ) [c:ns s:shard r:core_node x:ns _shard _replica_n ] o.a.s.c.s.request [ns _shard _replica_n ] webapp=/solr path=/select params={q=twitter.com&facet.field=domain&facet.field=content_type_norm&facet.field=type&facet.field=crawl_year&facet.field=status_code&facet.field=public_suffix&hl=on&indent=true&fl=id,score,title,hash,source_file_path,source_file_offset,url,url_norm,wayback_date,domain,content_type,crawl_date,content_type_norm,type&start= &q.op=and&fq=record_type:response+or+record_type:arc&fq=domain:"twitter.com"+and+content_type_norm:"html"+and+crawl_year:" "&rows= &wt=json&facet=true&f.crawl_year.facet.limit= } hits= status= qtime= here we have start= and million hits. the aggregating solr died minutes later. $ says that the request that crashed the master solr was for the same query, but with start= . as document representations * shards equals million document representations, the master jvm might have survived with -xmx= g. if not for the huge amount of tiny objects overloading the garbage collector. fixes and take aways easy fix of course: cast this.start in the javascript layer to integer and enforce an upper limit for start & rows in the middle layer for good measure. for next time we’ve learned to closely examine slow queries (captain obvious says hello) keep gc-logs a few restarts back (we only had the current and the previous one to start with) plot the gc pauses to see if there are spikes that did not crash the jvm (trivial to do with gceasy.io), then inspect the request logs around the same time as the spikes posted in eskildsen, solr | tagged bughunting, memory | leave a comment touching encouraged (an ongoing story) posted on october , by toke eskildsen a recurring theme at kb labs is to show a lot of pixels. by chance we got our hands on a k touch-sensitive display, capable of showing a non-trivial amount of said pixels on a non-trivial surface area. our cunning plan is to adapt some of the labs products to work on the display put the display somewhere in the public area of the library watch people swoon when they delve into beautiful cultural heritage data this post is intended to be a diary of sorts, journaling what we learn. coincidental activation ( - - ) we have talked about experimenting with interactive large displays for years. with emphasis on talked. it took someone with youthful initiative to actually do something: max odsbjerg pedersen discovered a usable & unused display and promptly sent us a video showing him using a labs product on the display. days later he brokered a loan agreement and days later we verified that no one questions two people removing a large display, as long as it is transported in a cardboard box. adding heavy box moving to résumé fair warning ( - - ) the software development department has a – not entirely undeserved – reputation of being loose cannons that tend to muck about in ways that unexpectedly affects other departments. to atone for blunders past and primarily because it really is the most constructive practice, representatives of the cultural heritage and the communications departments were duly informed about the initiated process and invited to participate in discussion hereof. in other words: we met them at lunch as usual and said “hey, we’ve got this nifty idea …”, to which they answered “sounds good!”. what have we got? ( - - ) the display is a ″ samsung flᴉp. its internal software seems focused on providing a digital flip-over with some connectivity possibilities? it does not have a build-in web browser, but connecting it to a windows machine is exceedingly easy. we will just have to duct tape a laptop to its back or something to that effect. it was a pleasant surprise to discover that the excellent tool openseadragon works perfectly out of the box with multi touch on desktop browsers: tap, double tap, drag, pinch & spread. well, as long as you are not a lazy developer that still use a pre- version of openseadragon where pinching is wonky *cough*. adapt, by which we mean “remove stuff” ( - - ) three kb labs products, which would benefit from a large display, were selected: at the core they are all web pages using openseadragon and as such, adaptation mostly meant removing features and interface elements. a simple navigational area was added to switch between the products and the poc mark i alpha was born: best viewed on a ″ tablet or larger. secure, by which we mean “fail” ( - - ) since the whole thing is intended for public display & interaction, we want to make sure the users stay on the designated pages. a developer navigating one of the designated pages pressing f switches to full screen with no navigational bar in chrome and the end users does not have access to the keyboard, so problem solved? our boss bjarne andersen passed by, stopped and played with the presentations. it took him minutes to somehow activate right-click and presto, the box was cracked. thanks boss! well, chrome has a designated kiosk mode: simply add -kiosk as an argument and all is taken care off. at least until co-worker kim christensen discovers that there is a handy swipe-from-a-vertical-edge gesture that opens the windows menu and other elements. cracked again. thanks kim! disabling swipe gestures did not seem possible without admin rights, which we do not have on the current computer. there seems to be a windows kiosk mode that also requires admin rights. oh well, maybe monday? weekend calls. broken windows and tweaks ( - - ) colleague thomas egense brought a private windows laptop to work (no worries, we only connect those to the eduroam network). it would not connect to the large display. reboot. it did connect to the display in k, but not to wifi. reboot. it did connect to wifi, but would no longer connect to the display. reboot. same. reboot. same. give up. actively avoid defenestrating the laptop. drink coffee. at least it went a little better when colleague gry vindelev elstrøm stopped by. she suggested adding some sort of map overlay, so that the users would not get lost in the big collages? and of course openseadragon delivers again: seconds and a reload later the wish were granted: openseadragon with navigator overlay gry’s other wish: to have visual-similarity spatial grouping of the maps collection is both valid and entirely possible to fulfill. buuut it is not a second job and the touch screen project is a side project, so that idea was filed under when we find the time. and then they were two ( - - ) heroic display digger max odsbjerg pedersen phoned in and said he had found a twin display lying around. he’ll put it up somewhere at au library, nobel park, mirroring the display we’re working with at the reoyal danish library, aarhus. thank you, max. you do realize we’re at the early proof on concept stage, right? go ahead is a given ( - - ) gitte bomholt christensen deals with the public space at the library. she visited to take a look at the project. her first question was if we should put the display on a movable stand or if a wall mount were better. we’ll take that as a “yes, we’ll go forward with this experiment”. soon they will be five ( - - ) early in the day miniboss katrine hofmann gasser asked for requirements for extra touch displays. later in the day, miniboss katrine hofmann gasser had ordered extra touch displays. damn, people! what happened to the concept of testing a minimum viable product followed by iterative improvement? the hunt for k ( - - ) the afternoon was not free (they never are), but at least it was not bogged down with other stuff. so what about upping the resolution from hd to k? how hard can that be? yeah, trips to operations and different computers produced the new knowledge that passive displayport → hdmi cables have trouble delivering the goods. native hdmi . handles k though: admittedly at hz only, but that works well enough when the interface reflects finger movements directly. the only situation where the hz is noticeable is when the user pans by flinging. gridified tsne ( - - ) running image collections through a trained network and positioning them according to their visual similarity is one of those “the future is now”-things. one favourite is pix-plot which produces an interactive d-visualization. but the touch screen is meant for large images and pix-plot is not made to display those. plotting directly to d does not solve the problem: a bit hard to enjoy all the images when they cover each other a marriage between pix-plot and the existing zoomable grid-based layout was proposed. some hacking later with the tools ml a & rasterfairy and… yeah, kinda-sorta? as can be seen on the screenshot below, there are definitely areas of similar images, but there are also multiple areas that looks like they should be collapsed into one. something’s off, but that will have to be solved later. there’s definitely some grouping there. and groups that looks suspiciously similar there are no image duplicates – we checked! frontpage material ( - - ) thomas egense wanted something else on the large touch screen, so he extracted all frontpages from the danish newspaper berlingske tidende, available from mediestream (of course he cheated and took an internal shortcut). it is quite an old newspaper, so “all” means , pages. rendering k fairly-high-res images is no technical problem, but as our scans are greyscale the result was somewhat bland and extremely cumbersome to navigate with intention. a sea of grey thankfully newspaper frontpages do possess one singular reliable piece of metadata: the date of the paper. adding an ugly date picker was easy enough and presto! intuitive navigation of the primary navigational axis for the material. proper tsne ( - - ) a breakthrough discovery was made today: if you clumsily swap the x and y axis for the coordinates, but keep the calculated width and height, when you plot gridified data, the result is … less good. corollary: if you un-swap said axes the result looks much better! as demonstrated by these before and after images: sorry about doing this on a not-fully-public-yet dataset (the awesome “anskuelsesbilleder” at au library, emdrup): we can only show the scaled-down versions of the properly gridified tsne layout, but they should convey the gist. maybe machines can label our stuff? ( - - ) since machine learning was great at positioning images according to visual similarity (or rather a mix of visual similarity and content similarity), maybe use it to automatically label our material? well, not so much with the collection of anskuelsesbilleder: the network (imagenet) is great for labelling photographies but poor for drawings: “binder”, “web site”, “envelope” and “jigsaw puzzle” were recurring and absolutely wrong guesses. again, sorry for not being allowed to show the images. hopefully later, when the rights has been cleared, ok? ideas aplenty ( - - ) karen williams was the nearest innocent bystander to show the latest experiment with the large touch screen and she upped the ante, asking for drive-by crowd-sourcing capabilities and visualization of sound. so much untapped potential in this! organisations gotta organise ( - - ) one does not simply walk down a put a touch screen on the wall. it is hard to have patience with a new toy in hand, but it is understandable that a mix of people from different departments must participate on something that involves display of cultural heritage data in the public areas of the library. unfortunately it will be nearly weeks before said mix of people are able to meet and determine how to go about the project. deep breath. inner peace. tempus fugit. pong detour ( - - ) annual christmas party time! and jesper lauridsen did not miss the oportunity that a big touch screen presented. he whipped up a multi-ball pong game where the balls were the faces of the people at the party. will you be hitting your colleague with a bat or let said colleague fall into oblivion? great success and nobody spilled beer on the touch table! and no, sorry, not allowed to show it due to the face thing. privacy and all. last details finished in the real hardware department by leet hacker jesper proper public tsne ( - - ) the image classification → tsne → rasterfairy → juxta chain is our new golden hammer, and the next collection to hits were our maps & atlases collection. given that the network was never trained explicitly for the minute differences in maps, it went surprisingly well. and this time we’re allowed to show the result: don’t just sit there! try it yourself secure, by which we mean “nearly succeed” ( - - ) there was a meeting with the right people and it took all of ½ minute to decide that yes, the large tablet should definitely be displayed in the public areas. then it took minutes to hammer out the details: the plan is to mount in on wheels and move it around to see where it works best. progress! the afternoon was spend trying to make the big screen less easy to hack. it is driven by an ubuntu . box using google chrome as the browser. as discovered earlier, chrome has a “kiosk” mode, which disables right click, the address bar and more. easy! the real problem was ubuntu itself: it has tablet support, meaning clever swipe gestures that activates program selection, unmaximizes windows, shows an on screen keyboard and other goodies. goodies meaning baddies, when trying to build a display kiosk! most of the solution was to use the disable gestures extension (and reboot to get the full disablement), but the on screen keyboard (activated by swiping in from the bottom of the screen) is apparently hard baked into the system (block caribou did not help us). we might have to uninstall it completely. to be continued posted in eskildsen, visualization | leave a comment docvalues jump tables in lucene/solr posted on march , by toke eskildsen lucene/solr is about to be released. among a lot of other things is brings lucene- , written by your truly with a heap of help from adrien grand. lucene- introduces jump-tables for docvalues, is all about performance and brings speed-ups ranging from worse than baseline to x, extremely dependent on index and access pattern. this is a follow-up post to faster docvalues in lucene/solr +. the previous post contains an in-depth technical explanation of the docvalues mechanisms, while this post focuses on the final implementation. docvalues? whenever the content of a field is to be used for grouping, faceting, sorting, stats or streaming in solr (or elasticsearch or lucene, where applicable), it is advisable to store it using docvalues. it is also used for document retrieval, depending on setup. docvalues in lucene : linked lists lucene shifted the api for docvalues from random access to sequential. this meant smaller storage footprint and cleaner code, but also caused the worst case single value lookup to scale linear with document count: getting the value for a docvalued field from the last document in a segment required a visit to all other value blocks. the linear access time was not a problem for small indexes or requests for values for a lot of documents, where most blocks needs to be visited anyway. thus the downside of the change was largely unnoticeable or at least unnoticed. for some setups with larger indexes, it was very noticeable and for some of them it was also noticed. for our netarchive search setup, where each segments has m documents, there was a severe performance regression: - x for common interactive use. text book optimization: jump-tables the lucene docvalues structure behaves as a linked list of data-nodes, with the specializations that it is build sequentially and that it is never updated after the build has finished. this makes it possible to collect the node offsets in the underlying index data during build and to store an array of these offsets along with the index data. with the node offsets quickly accessible, worst-case access time for a docvalue entry becomes independent of document count. of course, there is a lot more to this: see the previously mentioned faster docvalues in lucene/solr + for details. one interesting detail for jump-tables is that they can be build both as a cache on first access (see lucene- ) and baked into the index-data (see lucene- ). i much preferred having both options available in lucene, to get instant speed up with existing indexes and technically superior implementation for future indexes. alas, only lucene- was deemed acceptable. best case test case our netarchive search contains solr collections, each holding m documents in gb of index data. each collection is shard, merged down to segment and never updated. most fields are docvalues and they are heavily used for faceting, grouping, statistics, streaming exports and document retrieval. the impact of lucene- should be significant. in netarchive search, all collections are searched together using an alias. for the tests below only a single collection was used for practical reasons. there are three contenders: unmodified solr collection, using solr . . rc . codename solr . in this setup, jump-tables are not active as solr . . rc , which includes lucene- , only supports index-time jump-tables. this is the same as solr behaviour. solr collection upgraded to solr , using solr . . rc . codename solr r . in this setup, jump-tables are active and baked into the index data. this is the expected future behaviour when solr is released. solr collection, using lucene/solr at git commit point d f a b ab eda e c b a fc. codename solr l . during lucene- (search time jump tables) development, the implementation was committed to master. this was later reverted, but the checkout allow us to see what the performance would have been if this path had been chosen. test hardware is a puny -core i desktop with gb of ram, a tb rpm drive and a tb ssd. about gb of ram free for disk cache. due to time constraints only the streaming export test has been done on the spinning drive, the rest is ssd only. streaming exports premise: solr’s export function is used by us to extract selected fields from the collection, typically to deliver a csv-file with urls, mime types, file sizes etc for a corpus defined by a given filter. it requires docvalues to work. dv-problem: the current implementation of streaming export in solr does not retrieve the field values in document order, making the access pattern extremely random. this is absolute worst case for sequential docvalues. note that solr- will hopefully remedy this at some point. the test performs a streaming export of fields for , documents in the m index. the same export is done times, to measure the impact of caching. curl '/export?q=text:hestevogn&sort=id+desc& fl=content_type_ext,content_type_served,crawl_date,content_length' run seconds run seconds run seconds run seconds solr spin solr r spin solr l spin solr ssd solr r ssd solr l ssd observation: both solr r and solr l vastly outperforms solr . on a spinning drive there is a multi-minute penalty for run after which the cache has been warmed. this is a well known phenomenon. faceting premise: faceting is used everywhere and it is a hard recommendation to use docvalues for the requested fields. dv-problem: filling the counters used when faceting is done in document order, which works well with sequential access as long as the jumps aren’t too long: small result sets are relatively heavier penalized than large result sets. simple term-based searches with top- faceting on fields of varying type and cardinality: domain, crawl_year, public_suffix, content_type_norm, status_code and host. reading the graphs: all charts in this blog post follows the same recipe: x-axis is hit count (aka result set size), y-axis is response time (lower is better) hit counts are bucketed by order of magnitude and for each magnitude, boxes are shown for the three contenders: blue boxes are solr , pink are solr r and green are solr l the bottom of a box is the percentile, the top is the percentile. the black line in the middle is the median. minimum response time for the bucket is the bottom spike, while the top spike is percentile maximum response times are not shown as they tend to jitter severely due to garbage collection observation: modest gains from jump-tables with both solr rc and solr l . surprisingly the gains scale with hit count, which should be investigated further. grouping premise: grouping is used in netarchive search to collapse multiple harvests of the same url. as with faceting, using docvalues for grouping fields are highly recommended. dv-problem: as with faceting, group values are retrieved in document order and follows the same performance/scale logic. simple term-based searches with grouping on the high-cardinality (~ m unique values) field url_norm. observations: modest gains from jump-tables, similar to faceting. sorting premise: sorting is a basic functionality. dv-problem: as with faceting and grouping, the values used for sorting are retrieved in document order and follows the same performance/scale logic. this tests performs simple term-based searches with sorting on the high-cardinality field content_length. observations: modest gains from jump-tables. contrary to faceting and grouping, performance for high hit counts are the same for all setups, which fits with the theoretical model. positively surprising is that the theoretical overhead of the jump-tables does not show for higher hit counts. document retrieval premise: content intended for later retrieval can either be stored explicitly or as docvalues. doing both means extra storage, but also means that everything is retrieved from the same stored (and compressed) blocks, minimizing random access to the data. for the netarchive search at the royal danish library we don’t double-store field data and nearly all of the retrievable fields are docvalues. dv-problem: getting a search result is a multi-step process. early on, the top-x documents matching the query are calculated and their ids are collected. after that the ids are used for retrieving document representations. if this is done from docvalues, it means random access linear to the number of documents requested. simple term-based relevance ranked searches for the top- matching documents with core fields: id, source_file_s, url_norm, host, domain, content_type_served, content_length, crawl_date and content_language. observations: solid performance improvement with jump-tables. production request premise: the different functionalities are usually requested in combination. at netarchive search a typical request uses grouping, faceting, cardinality counting and top- document retrieval. dv-problem: combining functionality often means that separate parts of the index data are accessed. this can cause cache thrashing if there is not enough free memory for disk cache. with sequential docvalues, all intermediate blocks needs to be visited, increasing the need for disk cache. jump-tables lowers the number of storage requests and are thus less reliant on cache size. simple term-based relevance ranked searches for the top- matching documents, doing grouping, faceting, cardinality and document retrieval as described in the tests above. observations: solid performance improvement with jump tables. as with the previous analysis of search-time jump tables, utilizing multiple docvalues-using functionality has a cocktail effect where the combined impact is larger than the sum of the parts. this might be due to disk cache thrashing. overall observations & conclusions the effect of jump tables, both with solr . . rc and lucene- , is fairly limited; except for export and document retrieval, where the gains are solid. the two different implementations of jump tables performs very similar. do remember that these tests does not involve index updates at all: as lucene- is search-time, it does have a startup penalty when indexes are updated. for a the large segment index tested above, the positive impact of jump tables is clear. furthermore there is no significant slow down for higher hit counts with faceting/grouping/statistics, where the jump tables has no positive impact. before running these tests, it was my suspicion that the search-time jump tables in lucene- would perform better than the baked-in version. this showed not to be the case. as such, my idea of combining the approaches by creating in-memory copies of some of the on-storage jump tables has been shelved. missing performance testing is never complete, it just stops. some interesting thing to explore could be spinning drives concurrent requests raw search speed with rows= smaller corpus variations of rows, facet.limit and group.limit kibana and similar data-visualization tools posted in eskildsen, hacking, low-level, lucene, performance, solr, uncategorized | comments faster docvalues in lucene/solr + posted on october , by toke eskildsen this is a fairly technical post explaining lucene- and its implications on lucene, solr and (qualified guess) elasticsearch search and retrieval speed. it is primarily relevant for people with indexes of m+ documents. teaser we have a solr setup for netarchive search at the royal danish library. below are response times grouped by the magnitude of the hitcount with and without the lucene patch. grouping on url_norm, cardinality stats on url_norm, faceting on fields and retrieval of all stored & docvalued fields for the top- documents in our search result. as can be seen, the median response time with the patch is about half that of vanilla solr. the % percentile shows that the outliers has also been markedly reduced. long explanation follows as to what the patch does and why indexes with less than m documents are not likely to see the same performance boost. lucene/solr (birds eye) lucene is a framework for building search engines. solr is a search engine build using lucene. lucene, and thereby solr, is known as an inverted index, referring to the terms⟶documents structure that ensures fast searches in large amounts of material. as with most things, the truth is a little more complicated. fast searches are not enough: quite obviously it also helps to deliver a rich document representation as part of the search. more advanced features are grouping, faceting, statistics, mass exports etc. all of these have in common that they at some point needs to map documents⟶terms. lucene indexes are logically made up of segments containing documents made up of fields containing terms (or numbers/booleans/raws…). fields can be indexed for searching, which means terms⟶documents lookup stored for document retrieval docvalues for documents⟶terms lookup stored and docvalues representations can both be used for building a document representation as part of common search. stored cannot be used for grouping, faceting and similar purposes. the two strengths of stored are compression, which is most effective for “large” content. locality, meaning that all the terms for stored fields for a given document are stored together, making is low-cost to retrieve the content for multiple fields. whenever grouping, faceting etc. needs the documents⟶terms mapping, it can either be resolved from docvalues, which are build for this exact purpose, or by un-inverting the indexed terms. un-inversion costs time & memory, so the strong recommendation is to enable docvalues for grouping, faceting etc. docvalues in lucene/solr + (technical) so the mission is to provide a documents⟶terms (and numbers/booleans/etc) lookup mechanism. in lucene/solr , & this mechanism had a random access api, meaning that terms could be requested for documents in no particular order. the implementation presented some challenges and from lucene/solr this was changed to an iterator api (see lucene- ), meaning that terms must be resolved in increasing document id order. if the terms are needed for a document with a lower id that previously requested, a new iterator must be created and the iteration starts from the beginning. most of the code for this is available in lucene docvaluesproducer and indexeddisi. digging into it, the gains from the iterative approach becomes apparent: besides a very clean implementation with lower risk of errors, the representation is very compact and requires very little heap to access. indeed, the heap requirement for the search nodes in netarchive search at the royal danish library was nearly halved when upgrading from solr to solr . the compact representation is primarily the work of adrian grand in lucene- and lucene- . when reading the wall of text below, it helps to mentally view the structures as linked lists: to get to a certain point in the list, all the entries between the current entry and the destination entry needs to be visited. docvalues sparseness and packing it is often the case that not all documents contains terms for a given field. when this is case, the field is called sparse. a trivial representation for mapping documents⟶terms for a field with or long values per document would be an array of long[#documents_in_segment], but this takes up bytes/document, whether the document has a value defined or not. lucene- optimizes sparse values by using indirection: first step is to determine whether a document has a value or not. if it has a value, an index into a value-structure is derived. the second step is to retrieve the value from the value-structure. indeddisi takes care of the first step: for each docvalues field, documents are grouped in blocks of documents. each block starts with meta-data stating the block-id and the number of documents in the block that has a value for the field. there are types of blocks: empty: documents in the block has a term. sparse: - documents in the block has a term. dense: - documents in the block has a term. all: documents in the block has a term. step . : block skipping to determine if a document has a value and what the index of the value is, the following pseudo-code is used: while (blockindex < docid/ ) { valueindex += block.documents_with_values_count block = seektoblock(block.nextblockoffset) blockindex++} if (!block.hasvalue(docid% )) { // no value for docid return } valueindex += block.valueindex(docid% ) unfortunately it does not scale with index size: at the netarchive at the royal danish library, we use segments with m values (not a common use case), which means that , blocks must be iterated in the worst case. introducing an indexvalue cache solves this and the code becomes valueindex = valuecache[docid/ ] block = seektoblock(offsetcache[docid/ ]) if (!block.hasvalue(docid% ) { // no value for docid return } valueindex += block.valueindex(docid% ) the while-loop has been removed and getting to the needed block is constant-time. step . : block internals determining the value index inside of the block is trivial for empty and all blocks. sparse is a list of the documentids with values that is simply iterated (this could be a binary search). this leaves dense, which is the interesting one. dense blocks contains a bit for each of its documents, represented as a bitmap = long[ ]. getting the value index is a matter of counting the set bits up to the wanted document id: inblockid = docid% while (inblockindex < inblockid/ ) { valueindex += total_set_bits(bitmap[inblockindex++]) } valueindex += set_bits_up_to(bitmap[inblockindex], inblockid% ) this is not as bad as it seems as counting bits in a long is a single processor instruction on modern cpus. still, doing of anything to get a value is a bit much and this worst-case is valid for even small indexes. this is solved by introducing another cache: rank = char[ ] (a char is bytes): inblockid = docid% valueindex = rank[inblockid/ ] inblockindex = inblockid/ * while (inblockindex < inblockid/ ) { valueindex += total_set_bits(bitmap[inblockindex++]) } valueindex += set_bits_up_to(bitmap[inblockindex], inblockid% ) worst-case it reduced to a rank-cache lookup and summing of the bits from longs. now that step : value existence and value index has been taken care of, the value itself needs to be resolved. step : whole numbers representation there are different types of values lucene/solr: strings, whole numbers, floating point numbers, booleans and binaries. on top of that a field can be single- or multi-valued. most of these values are represented in a way that provides direct lookup in lucene/solr , but whole numbers are special. in java whole numbers are represented in a fixed amount of bytes, depending on type: byte for byte, bytes for short or char, bytes for integer and bytes for long. this is often wasteful: the sequence [ , , , ] could be represented using only bits/value. the sequence [ , , , ] could also be represented using only bits/value if it is known that the greatest common divisor is ⁷. the list of tricks goes on. for whole numbers, lucene/solr uses both the smallest amount of bits required by packedints for a given sequence as well as greatest common divisor and constant offset. these compression techniques works poorly both for very short sequences and for very long ones; lucene- splits whole numbers into sequences of numbers. getting the value for a given index is a matter of locating the right block and extracting the value from that block: while (longblockindex < valueindex/ ) { longblock = seektolongblock(longblock.nextblockoffset) longblockindex++ } value = longblock.getvalue(valueindex% ) this uses the same principle as for value existence and the penalty for iteration is also there: in our m documents/segment index, we have numeric fields where most values are present. they have , blocks each, which must be all be visited in the worst case. the optimization is the same as for value existence: introduce a jump table. longblock = seektolongblock(longjumps[valueindex/ )) value = longblock.getvalue(valueindex% ) value retrieval becomes constant time. theoretical observations with a pure iterative approach, performance goes down when segment size goes up and the amount of data to retrieve goes up slower than index size. the performance slowdown only happens after a certain point! as long as the gap between the docids is small enough to be within the current or the subsequent data chunk, pure iteration is fine. consequently, the requests that involves lots of monotonically increasing docid lookups (faceting, sorting & grouping for large result sets) fits the iterative api well as they needs data from most data blocks. requests that involves fewer monotonically increasing docid lookups (export & document retrieval for all requests, faceting, sorting & grouping for small result sets) fits poorly as they result in iteration over data blocks that do not provide any other information than a link to the next data block. as all the structures are storage-backed, iterating all data blocks – even when it is just to get a pointer to the next block – means a read request. this is problematic, unless there is plenty of ram for caching: besides the direct read-time impact, the docvalues structures will hog the disk cache. with this in mind, it makes sense to check the patch itself for performance regressions with requests for a lot of values as well as test with the disk cache fully warmed and containing the structures that are used. alas, this has to go on the to-do for now. tests hardware & index testing was done against our production netarchive search. it consists of collections, accessed as a single collection using solr’s alias mechanism. each collection is roughly m documents / gb of index data optimized to segment, each segment on a separate ssd. each machine has gb of ram with about gb free for disk cache. there are machines, each serving collections (except the last one that only serves at the moment). this means that ~ % of total index size is disk cached. methodology queries were constructed by extracting terms of varying use from the index and permutating them for simple - term queries all tests were against the full production index, issued at times when it was not heavily used queries were issued single-threaded, with no repeat queries all test setups were executed times, with a new set of queries each time the order of patch vs. sans-patch tests was always patch first, to ensure that any difference in patch favour was not due to better disk caching how to read the charts all charts are plotted with number of hits on the x-axis and time on the y-axis. the x-axis is logarithmic with the number of hits bucketed by magnitude: first bucket holds all measurements with - hits, second bucket holds those with - hits, the third holds those with - hits and so forth. the response times are displayed as box plots where upper whisker is the % percentile top of the box is % percentile black bar is % percentile (the median) bottom of the box is % percentile lower whisker is minimum measured time each bucket holds boxes test run , patch enabled test run , vanilla solr test run , patch enabled test run , vanilla solr test run is discarded to avoid jitter from cache warming. ideally the boxes from run should be the same as for run . however, as the queries are always new and unique, an amount of variation is to be expected. important note : the y-axis max-value changes between some of the charts. document retrieval there seems to be some disagreement as to whether the docvalues mechanism should ever be used to populate documents, as opposed to using stored. this blog post will only note that docvalues are indeed used for this purpose at the royal danish library and let it be up to the reader to seek more information on the matter. there are about fields in total in netarchive search, with the vast majority being docvalued string fields. there are numeric docvalued fields. retrieval of top- documents with all field values observation: response times for patched (blue & green) are markedly lower than vanilla (ping & orange). the difference is fairly independent of hit count, which matches well with the premise that the result set size is constant at documents. grouping grouping on the string field url_norm field is used in netarchive search to avoid seeing too many duplicates. to remove the pronounced difference caused by document retrieval, only the single field url_norm is requested for only group with document. grouping on url_norm observation: the medians for patched vs. vanilla are about the same, with a slight edge to patched. the outliers (the top t of the boxes) are higher for vanilla. faceting faceting is done for fields of varying cardinality. as with grouping, the effect of document retrieval is sought minimized. faceting on fields domain, crawl_year, public_suffix, content_type_norm, status_code, host observation: patched is an improvement over vanilla up to m+ hits. sorting in this test, sorting is done descending on content_length, to locate the largest documents in the index. as with grouping, the effect of document retrieval is sought minimized. sorting on content_length observation: patched is a slight improvement over vanilla. cardinality in order to provide an approximate hitcount with grouping, the cardinality of the url_norm field is requested. as with grouping, the effect of document retrieval is sought minimized. hyperloglog cardinality on url_norm observation: too much jitter to say if patch helps here. numeric statistics statistics (min, max, average…) on content_length is a common use case in netarchive search. as with grouping, the effect of document retrieval is sought minimized. numeric statistics on content_length observation: patched is a slight improvement over vanilla. cocktail effect, sans document combining faceting, grouping, stats and sorting while still minimizing the effect of document retrieval. faceting on fields, grouping on url_norm, stats on content_length and sorting on content_length observation: patched is a clear improvement over vanilla. production request combination the solrwayback front end for netarchive search commonly use document retrieval for top- results, grouping, cardinality and faceting. this is the same chart as the teaser at the top, with the addition of test run . grouping on url_norm, cardinality stats on url_norm, faceting on fields and retrieval of all stored & docvalued fields for the top- documents in our search result. observation: patched is a pronounced improvement over vanilla. the combination of multiple docvalues using request parameters is interesting as the effect of the patch on the whole seems greater than the sum of the individual parts. this could be explained by cache/io saturation when using vanilla solr. whether the cause, this shows that it is important to try and simulate real-world workflows as close as possible. overall observations for most of the performance tests, the effect of the lucene- patch vs. vanilla is pronounced, but limited in magnitude besides lowing the median, there seems to be a tendency for the patch to reduce outliers, notably for grouping for document retrieval, the patch improved performance significantly. separate experiments shows that export gets a similar speed boost for all the single-feature tests, the active parts of the index data are so small that they are probably cached. coupled with the limited improvement that the patch gives for these tests, it indicates that the patch will in general have little effect on systems where the full index is is disk cache the performance gains with the “production request combination” aka the standard requests from our researchers, are very high future testing potential regression for large hit counts max response times (not just percentile ) concurrent requests io-load during tests smaller corpus export/streaming disk cache influence want to try? there is a patch for solr trunk at lucene- and it needs third party validation from people with large indexes. i’ll port it to any solr .x-version requested and if building solr is a problem, i can also compile it and put it somewhere. hopefully it will be part of solr at some point. update : patch overhead and status currently the patch is search-time only. technically is could also be index-time by modifying the codec. for a single index in the netarchive search setup, the patch adds seconds to first search-time and mb of heap out of gb allocated for the whole solr. the seconds is in the same ballpark (this is hard to measure) as a single unwarmed search with top- document retrieval. the patch is ported to solr . . and used in production at the royal danish library. it is a debug-patch, meaning that the individual optimizations can be enabled selectively for easy performance comparison. see the lucene- jira-issue for details. posted in eskildsen, hacking, low-level, lucene, performance, solr | comment prebuild big data word vec dictionaries posted on july , by thomasegense prebuild and trained word vec dictionaries ready for use two different prebuild big data word vec dictionaries has been added to loar (library open access repository) for download. these dictionaries are build from the text of , e-books from project gutenberg and . . danish newspaper pages. . of the gutenberg e-books are english, but over different languages are present in the dictionaries. even though they are different languages the word vec algorithm did a good job of separating the different languages so it is almost like different word vec dictionaries. the text from the danish newspapers is not public available so you would not be able to build this dictionary yourself. a total of gb of raw text went into building the dictionary, so it is probably the largest word vec dictionary build on a danish corpus. since the danish newspapers suffer from low quality ocr, many of words in the dictionary are misspellings. using this dictionary it was possible to fix many of the ocr errors due the nature of the word vec algorithm, since a given word appears in similar contexts despite its misspellings and is identified by its context. (see https://sbdevel.wordpress.com/ / / /automated-improvement-of-search-in-low-quality-ocr-using-word vec/) download and more information about the word vec dictionaries: download online demo of the two corpora: word vec demo posted in uncategorized | leave a comment solrwayback software bundle has been released posted on may , by thomasegense the solrwayback software bundle can be used to search and playback archived webpages in warc format. it is an out of the box solution with index workflow, solr and tomcat webserver and a free text search interface with playback functionality. just add your warc to a folder and start the index job. the search interface has additional features besides freetext search. this includes: image search similar to google images search by uploading a file. (image/pdf etc.) see if the resource has been harvested and from where. link graph showing links (ingoing/outgoing) for domains using the d javascript framework. raw download of any harvested resource from the binary arc/warc file. export a search resultset to a warc-file. streaming download, no limit of size of resultset. an optional built in socks proxy can be used to view historical webpages without browser leaking resources from the live web. see the github page for screenshots of solrwayback and scroll down to the install guide try it out. link: solrwayback posted in uncategorized | leave a comment visualising netarchive harvests posted on march , by nielskristianhansenskovmand an overview of website harvest data is important for both research and development operations in the netarchive team at det kgl. bibliotek. in this post we present a recent frontend visualisation widget we have made. from the solrwayback machine we can extract an array of dates of all harvests of a given url. these dates are processed in the browser into a data object containing the years, months, weeks and days to enable us to visualise the data. futhermore the data is given an activity level from - . the high-level overview seen below is the year-month graph. each cell is coloured based on the activity level relative to the number of harvests in the most active month. for now we use a linear calculation so gray means no activity, activity level is - % of the most active month, and level is - % of the most active month. as github fans, we have borrowed the activity level colouring from the user commit heat map. we can visualise a more detailed view of the data as either a week-day view of the whole year, or as a view of all days since the first harvest. clicking one of these days reveals all the harvests for the given day, with links back to solrwayback to see a particular harvest. in the graph above we see the days of all weeks of as vertical rows. the same visualisation can be made for all harvest data for the url, as seen below (cut off before , for this blog post). there are both advantages and disadvantages to using the browser-based data calculation. one of the main advantages is a very portable frontend application. it can be used with any backend application that outputs an array of dates. the initial idea was to make the application usable for several different in-house projects. drawbacks to this approach is, of course, the scalability. currently the application processes . dates in about - seconds on the computer used to develop the application (a quad core intel i ). the application uses the frontend library vuejs and only one other dependency, the date-fns library. it is completely self-contained and it is included in a single script tag, including styles. ideas for further development. we would like to expand this to also include both: multiple urls, which would be nice for resources that have changed domain, subdomain or protocol over time, e.g. the url http://pol.dk, http://www.pol.dk and https://politiken.dk could be used for the danish newspaper politiken. domain visualisation for all urls on a domain. a challenge here will of course be the resources needed to process the data in the browser. perhaps a better calculation method must be used – or a kind of lazy loading. posted in blogging, solr, web | tagged frontend, solrwayback | leave a comment solrwayback machine posted on february , by thomasegense another ‘google innovation week’ at work has produced the solrwayback machine. it works similar to the internet archive: wayback machine (https://archive.org/web/) and can be used to show harvested web content (warc files). the danish internet archive has over billion harvested web objects and takes petabyte of storage. the solrwayback engine require you have indexed the warc files using the warc-indexer tool from british library. (https://github.com/ukwa/webarchive-discovery/tree/master/warc-indexer). it is quite fast and comes with some additional features as well: image search similar to google images link graphs showing links (ingoing/outgoing) for domains using the d javascript framework. raw download of any harvested resource from the binary arc/warc file. unfortunately the collection is not available for the public so i can not show you the demo. but here is a few pictures from the solrwayback machine. solrwayback at github: https://github.com/netarchivesuite/solrwayback/ posted in uncategorized | comment juxta – image collage with metadata posted on february , by toke eskildsen creating large collages of images to give a bird’s eye view of a collection seems to be gaining traction. two recent initiatives: the new york public library has a very visually pleasing presentation of public domain digitizations, but with a somewhat coarse switch between overview and details. nick ruest has created very large collages ( million+ images) with smooth zoom from full overview to single image, but without metadata for the individual images. combining those two ideas seemed like a logical next step and juxta was born: a fairly small bash-script for creating million-scale collages of images, with no special server side. there’s a small (just images) demo at sblabs. presentation principle the goal is to provide a seamless transition from the full collection to individual items, making it possible to compare nearby items with each other and locate interesting ones. contextual metadata should be provided for general information and provenance. concretely, the user is presented with all images at once and can zoom in to individual images in full size. beyond a given threshold, metadata are show for the image currently under the cursor, or finger if a mobile device is used. an image description is displayed just below the focused image, to avoid disturbing the view. a link to the source of the image is provided on top. overview of historical maps meta-data for a specific map technical notes, mostly on scaling on the display side, openseadragon takes care of the nice zooming. when the user moves the focus, a tiny bit of javascript spatial math resolves image identity and visual boundaries. openseadragon uses pyramid tiles for display and supports the deep zoom protocol can be implemented using only static files. the image to display is made up of tiles of (typically) × pixels. when the view is fully zoomed, only the tiles within the viewport are requested. when the user zooms out, the tiles from the level above are used. the level above is half the width and half the height and is thus represented by ¼ the amount of tiles. and so forth. generating tiles is heavy a direct way of creating the tiles is create one large image of the full collage (imagemagick’s montage is good for this) generate tiles for the image scale the image down to %× % if the image is larger than × pixel then goto unfortunately this does not scale particularly well. depending on size and tools, it can take up terabytes of temporary disk space to create the full collage image. by introducing a size constraint, juxta removes this step: all individual source images are scaled & padded to have the exact same size. the width and height of the images are exact multiples of . then the tiles can be created by for each individual source image, scale, pad and split the image directly into tiles create the tiles at the level above individually by joining the corresponding tiles below and scale to %× % size if there are more than tile or that tile is larger than × pixel then goto as the tiles are generated directly from either source images or other tiles, there is no temporary storage overhead. as each source image and each tile are processed individually, it is simple to do parallel processing. metadata takes up space too displaying image-specific metadata is simple when there are just a few thousand images: use an in-memory array of strings to hold the metadata and fetch it directly from there. but when the number of images goes into the millions, this quickly becomes unwieldy. juxta groups the images spatially in buckets of × images. the metadata for all the images in a bucket are stored in the same file. when the user moved the focus to a new image, the relevant bucket is fetched from the server and the metadata are extracted. a bucket cache is used to minimize repeat calls. most file systems don’t like to hold a lot of files in the same folder while the limits differ, common file systems such as ext, hfs & ntfs all experience performance degradation with high numbers of files in the same folder. the deep zoom protocol in conjunction with file-based tiles means that the amount of files at the deepest zoom level is linear to the number of source images. if there are million source images, with full-zoom size × pixels ( × tiles), the number of files in a single folder will be * * m = million. far beyond the comfort-zone fo the mentioned file systems (see the juxta readme for tests of performance degradation). juxta mitigates this by bucketing tiles in sub-folders. this ensures linear scaling of build time at least up to - million images. million+ images would likely deteriorate build performance markedly, but at that point we are also entering “is there enough free inodes on the file system?” territory. unfortunately the bucketing of the tile files is not in the deep zoom standard. with openseadragon, it is very easy to change the mapping, but it might be more difficult for other deep zoom-expecting tools. some numbers using a fairly modern i desktop and threads, generating a collage of mpixel images, scaled down to × pixels ( × tiles) took seconds or about images/second. repeating the experiment with a down-scale to × pixels (smallest possible size) raised the speed to about ½ image/second. juxta comes with a scale-testing script that generates sample images that are close (but not equal) to the wanted size and repeats them for the collage. with this near-ideal match, processing speed was ½ images/second for × tiles and images/second for × tiles. the scale-test script has been used up to million images, with processing time practically linear to the number of images. at images/second that is hours. posted in uncategorized | leave a comment automated improvement of search in low quality ocr using word vec posted on february , by thomasegense this abstract has been accepted for digital humanities in the nordic countries nd conference, http://dhn .eu/ in the danish newspaper archive[ ] you can search and view million newspaper pages. the search engine[ ] uses ocr (optical character recognition) from scanned pages but often the software converting the scanned images to text makes reading errors. as a result the search engine will miss matching words due to ocr error. since many of our newspapers are old and the scans/microfilms is also low quality, the resulting ocr constitutes a substantial problem. in addition, the ocr converter performs poorly with old font types such as fraktur. one way to find ocr errors is by using the unsupervised word vec[ ] learning algorithm. this algorithm identifies words that appear in similar contexts. for a corpus with perfect spelling the algorithm will detect similar words synonyms, conjugations, declensions etc. in the case of a corpus with ocr errors the word vec algorithm will find the misspellings of a given word either from bad ocr or in some cases journalists. a given word appears in similar contexts despite its misspellings and is identified by its context. for this to work the word vec algorithm requires a huge corpus and for the newspapers we had gb of raw text. given the words returned by word vec we use a danish dictionary to remove the same word in different grammatical forms. the remaining words are filtered by a similarity measure using an extended version of levenshtein distance taking the length of the word and an idempotent normalization taking frequent one and two character ocr errors into account. example: let’s say you use the word vec to find words for banana and it returns: hanana, bananas, apple, orange. remove bananas using the (english) dictionary since this is not an ocr error. for the three remaining words only hanana is close to banana and it is thus the only misspelling of banana found in this example. the word vec algorithm does not know how a words is spelled/misspelled, it only uses the semantic and syntactic context. this method is not an automatic ocr error corrector and cannot output the corrected ocr. but when searching it will appear as if you are searching in an ocr corrected text corpus. single word searches on the full corpus gives an increase from % to % in the number of results returned. preliminary tests on the full corpus shows only relative few false positives among the additional results returned, thus increasing recall substantially without a decline in precision. the advantage of this approach is a quick win with minimum impact on a search engine [ ] based on low quality ocr. the algorithm generates a text file with synonyms that can be used by the search engine. not only single words but also phrase search with highlighting works out of the box. an ocr correction demo[ ] using word vec on the danish newspaper corpus is available on the labs[ ] pages of the state and university library, denmark. [ ] mediestream, the danish digitized newspaper archive. http://www .statsbiblioteket.dk/mediestream/avis [ ] solr or elasticsearch etc. [ ] mikolov et al., efficient estimation of word representations in vector space https://arxiv.org/abs/ . [ ] ocr error detection demo (change word parameter in url) http://labs.statsbiblioteket.dk/dsc/ocr_fixer.jsp?word=statsminister [ ] labs for state and university library, denmark http://www.statsbiblioteket.dk/sblabs/ posted in uncategorized | comments ← older posts search for: archives june october march october july may march february november july march january november october september july june may april march february january december november october september august june april march february january december november october september june april march february january september may march february january november october september june april march february january december october september august july june may april march february january december november october september meta register log in software development at royal danish library create a free website or blog at wordpress.com. software development at royal danish library blog at wordpress.com. add your thoughts here... (optional) post to cancel privacy & cookies: this site uses cookies. by continuing to use this website, you agree to their use. to find out more, including how to control cookies, see here: cookie policy happy new year to everyone, including the haters and the fake news media! happy new year to everyone, including the haters and the fake news media! , images tweeted at @realdonaldtrump | full dataset is available here. | created with juxta. generated - - : this work is licensed under a creative commons attribution . international license. web archives for historical research none downloads – marcedit development marcedit development downloads for everyone beta development marcedit . .x/macos . .x (updated / / (windows [changelog]) windows (consolidated builds) | macos | linux current development marcedit . .x/macos . .x (updated / / (windows [changelog]); / / (macos [changelog]); / / (linux [changelog])) windows -bit download | windows -bit download | macos | linux | previous builds managing marcedit installations [help] best practices for organizations managing marcedit installations. msi cleaner (updated nov. ) zipped download the msi cleaner is a simple tool that can be used to “clean” a marcedit installation on those rare occasions when an installation problem occurs and marcedit can no longer be uninstalled (and re-installed). this tool cleans the registry and installation cache so that marcedit can be uninstalled and re-installed. xslt downloads: various xslt files created over the years can be found on github at: https://github.com/reeset/marcedit_xslt_files simple instructions for loading new xslt translations into marcedit can be found at: https://marcedit.reeset.net/software/xslt/load.txt plug-in downloads: makecheckdigit plugin oclchelper plugin ris marc plugin (github) internet archive to hathitrust plugin (github) past versions (maintenance fixes only) marcedit . . (updated / / ) .net . . (supports vista+) archive builds marcedit . . (updated / ) archive builds marcedit . .x (updated oct. , (windows); oct. , (linux); sept. , (osx)) windows -bit download | windows -bit download | mac app download (installation instructions)| zipped download (linux/other) (installation instructions) | previous builds license by downloading and installing any of the applications/components, you are agreeing to abide by the licensing agreement as specified by the author. for a copy of the licensing agreement, please see: marcedit end user licensing agreement. home downloads features getting help help knowledge base sample data for education marcedit webinar series tutorials marcedit faqs technical details about about marcedit about the author marcedit end user license agreement marcedit user map privacy policy contact me search © marcedit development marcedit macos download macos version the marcedit macos is a native macos application designed to work on all macos systems . and greater. the existence of a native mac version of marcedit owes a great deal of gratitude to whitni watkins, who helped coordinate its development and community around this version of the product. without her push and organization, it’s very likely that a native version of the software wouldn’t have been developed and continue to be a development priority. system requirements: to run marcedit macos , you must must be running macos . or greater. as of today, if you want z . support, you need to download the yaz library through a service like homebrew. (optional) z . support — marcedit includes a z . client that utilizes the yaz library. this must be installed on your system, and preferably using homebrew. use the following instructions: install homebrew: http://brew.sh/ if homebrew is already installed, run the update command: >>brew update (marcedit . . +): as of marcedit . . +, marcedit is only being compiled as a -bit application. when installing yaz via homebrew, you can simply install using the default settings (which will default to -bit). use the following command: >>brew install yaz youtube video: download the macos version close -bit download marcedit -bit downloads marcedit has two -bit download options — a version that does not require administrator permissions and one that does require administrator permissions. functionally, these versions offer the exact same functionality, with the primary difference being that the non-administrative installation installs into a single user space, and the version that requires administrator permissions are installed into the program files space to enable multiple users to run marcedit. please review the table below to determine which version of marcedit you should download. non-administrator you should download this version if: you cannot install marcedit as an administrator you are the only user on this computer or you are the only user that will use marcedit on this computer you want to run marcedit in a sandbox this is the recommended version of marcedit for most users. the administrator download is really only recommended for system administrators managing software using tools like microsoft’s software center. download marcedit (all users) administrator you should download this version if: multiple users work on and sign-in on this computer and may use marcedit the administrator download is really only recommended for system administrators managing software using tools like microsoft’s software center. download marcedit (administrator users) close -bit download marcedit -bit downloads marcedit has two -bit download options — a version that does not require administrator permissions and one that does require administrator permissions. functionally, these versions offer the exact same functionality, with the primary difference being that the non-administrative installation installs into a single user space, and the version that requires administrator permissions are installed into the program files space to enable multiple users to run marcedit. please review the table below to determine which version of marcedit you should download. non-administrator you should download this version if: you cannot install marcedit as an administrator you are the only user on this computer or you are the only user that will use marcedit on this computer you want to run marcedit in a sandbox this is the recommended version of marcedit for most users. the administrator download is really only recommended for system administrators managing software using tools like microsoft’s software center. download marcedit (all users) administrator you should download this version if: multiple users work on and sign-in on this computer and may use marcedit the administrator download is really only recommended for system administrators managing software using tools like microsoft’s software center. download marcedit (administrator users) close linux download download linux version the marcedit linux downloader is a self-contained installer utilizing the makeself command. when installing the linux version of marcedit , the installer will extract the program into the same directory that you run the installer. following extraction, the tool will run a script that will create a shortcut on your linux desktop that can then be used to run marcedit. if you want to specify a particular target directory, use the -target option when running marcedit .run. for example: >marcedit .run –target ~/documents/marcedit system requirements: to run marcedit , you must have a current version of mono installed. please see: mono download for instructions on how to download the mono runtime. please note, you must install the mono-complete package. typically, no other requirements are necessary. for more information, please see the following youtube video download linux version close none planet code lib planet code lib planet code lib - http://planet.code lib.org david rosenthal: talk at berkeley's information access seminar once again cliff lynch invited me to give a talk to the information access seminar at uc berkeley's ischool. preparation time was limited because these days i'm a full-time grandparent so the talk, entitled securing the digital supply chain summarizes and updates two long posts from two years ago: certificate transparency securing the software supply chainthe abstract was:the internet is suffering an epidemic of supply chain attacks, in which a trusted supplier of content is compromised and delivers malware to some or all of their clients. the recent solarwinds compromise is just one glaring example. this talk reviews efforts to defend digital supply chains. below the fold, the text of the talk with links to the sources. solarwinds, and many other recent system and network compromises have been supply chain attacks. these are extremely efficient, because unlike one-at-a-time attacks such as phishing, they provide a built-in mass deployment mechanism. a single compromise of solarwinds infected at least , networks. clearly, the vendors' security practices, and their vendors' security practices, and so on ad infinitum are important, but the sad truth is that current digital supply chain technologies are incapable of mitigating the inevitable security lapses along the chain.this talk reviews the efforts to defend supply chains that deliver digital content, such as software. but lets start with a simpler case, web pages.web page supply chainhow do i know that i'm talking to the right web site? because there's a closed padlock icon in the url bar, right?[slide ]mozilla says:a green padlock (with or without an organization name) indicates that:you are definitely connected to the website whose address is shown in the address bar; the connection has not been intercepted.the connection between firefox and the website is encrypted to prevent eavesdropping.nb - this is misleading!the padlock icon appears when the browser has validated that the connection to the url in the url bar supplied a certificate for the site in question carrying a signature chain ending in one of the root certificates the browser trusts. browsers come with a default list of root certificates from certificate authorities (cas). my firefox trusts certificates from different organizations including, for example, amazon and google, but also chunghwa telecom co., ltd. and the dutch government. why is this list a problem?the browser trusts all of them equally.the browser trusts cas that the cas on the list delegate trust to. back in , the eff found more than organizations that internet explorer and firefox trusted.commercial cas on the list, and cas they delegate to, have regularly been found to be issuing false or insecure certificates. [slide ]one of these trusted organizations is the internet security research group, a not-for-profit organization hosted by the linux foundation and sponsored by many organizations including mozilla and the eff, which has greatly improved the information hygiene of the web through a program called let's encrypt. this has provided over million web sites with free certificates carrying a signature chain rooted in a certificate that almost all browsers trust. my blog's certificate is one of them, as you can see by clicking on the padlock icon.[slide ]barysevich identified four such sellers of counterfeit certificates since . two of them remain in business today. the sellers offered a variety of options. in , one provider calling himself c@t advertised certificates that used a microsoft technology known as authenticode for signing executable files and programming scripts that can install software. c@t offered code-signing certificates for macos apps as well. ... "in his advertisement, c@t explained that the certificates are registered under legitimate corporations and issued by comodo, thawte, and symantec—the largest and most respected issuers," dan goodin one-stop counterfeit certificate shops for all your malware-signing needsabuse of the trust users place in cas is routine:in one case, a prominent dutch ca (diginotar) was compromised and the hackers were able to use the ca’s system to issue fake ssl certificates. the certificates were used to impersonate numerous sites in iran, such as gmail and facebook, which enabled the operators of the fake sites to spy on unsuspecting site users. ... more recently, a large u.s.-based ca (trustwave) admitted that it issued subordinate root certificates to one of its customers so the customer could monitor traffic on their internal network. subordinate root certificates can be used to create ssl certificates for nearly any domain on the internet. although trustwave has revoked the certificate and stated that it will no longer issue subordinate root certificates to customers, it illustrates just how easy it is for cas to make missteps and just how severe the consequences of those missteps might be. in sennheiser provided another example:the issue with the two headsetup apps came to light earlier this year when german cyber-security firm secorvo found that versions . , . , and . installed two root certification authority (ca) certificates into the windows trusted root certificate store of users' computers but also included the private keys for all in the senncomcckey.pem file. certificates depend on public-key cryptography, which splits keys into public/private key pairs. private keys can decrypt text encrypted by the public key, and vice versa. the security of the system depends upon private keys being kept secret. this poses two problems:as the sennheiser example shows, it is easy for the private keys to leak. another common way for them to leak is for a server to be compromised. for the server to be able to verify its identity, and thus unlock the padlock, the private key needs to be stored on the server in cleartext. so an intruder can steal it to impersonate the server.there is no alarm bell or notification to the owner or affected users when a private key leaks. so, as in the sennheiser case, the attacker may be able to use it unimpeded for a long time, until some security researcher notices some anomaly.catalin cimpanu continues:in a report published today, secorvo researchers published proof-of-concept code showing how trivial would be for an attacker to analyze the installers for both apps and extract the private keys.making matters worse, the certificates are also installed for mac users, via headsetup macos app versions, and they aren't removed from the operating system's trusted root certificate store during current headsetup updates or uninstall operations....sennheiser's snafu ... is not the first of its kind. in , lenovo shipped laptops with a certificate that exposed its private key in a scandal that became known as superfish. dell did the exact same thing in in a similarly bad security incident that became known as edellroot. cimpanu also reports on a more recent case:under the guise of a "cybersecurity exercise," the kazakhstan government is forcing citizens in its capital of nur-sultan (formerly astana) to install a digital certificate on their devices if they want to access foreign internet services.once installed, the certificate would allow the government to intercept all https traffic made from users' devices via a technique called mitm (man-in-the-middle). this type of “mistake” allows attackers to impersonate any web site to affected devices.cas are supposed to issue three grades of certificate based on increasingly rigorous validation:domain validated (dv) certificates verify control over the dns entries, email and web content of the specified domain. they can be issued via automated processes, as with let's encrypt.organization validated (ov) certificates are supposed to verify the legal entity behind the dv-level control of the domain, but in practice are treated the same as dv certificates.extended validation (ev) certificates require "verification of the requesting entity's identity by a certificate authority (ca)". verification is supposed to be an intrusive, human process.[slide ] sourcebut, as can be seen from the advert, the extended verification process is far from fool-proof. this lack of trustworthiness of cas should not be a surprise. six years ago security collapse in the https market, a fascinating analysis of the (lack of) security on the web from an economic rather than a technical perspective by axel arnbak et al from amsterdam and delft universities showed that cas lack incentives to be trustworthy. they write:information asymmetry prevents buyers from knowing what cas are really doing. buyers are paying for the perception of security, a liability shield, and trust signals to third parties. none of these correlates verifiably with actual security. given that ca security is largely unobservable, buyers’ demands for security do not necessarily translate into strong security incentives for cas.negative externalities of the weakest-link security of the system exacerbate these incentive problems. the failure of a single ca impacts the whole ecosystem, not just that ca’s customers. all other things being equal, these interdependencies undermine the incentives of cas to invest, as the security of their customers depends on the efforts of all other cas.the reason for the weakest-link is:a crucial technical property of the https authentication model is that any ca can sign certificates for any domain name. in other words, literally anyone can request a certificate for a google domain at any ca anywhere in the world, even when google itself has contracted one particular ca to sign its certificate. this "technical property" is actually important, it is what enables a competitive market of cas. symantec in particular has exploited it wholesale:google's investigation revealed that over a span of years, symantec cas have improperly issued more than , certificates. ... they are a major violation of the so-called baseline requirements that major browser makers impose of cas as a condition of being trusted by major browsers. but symantec has suffered no effective sanctions because they are too big to fail:symantec's repeated violations underscore one of the problems google and others have in enforcing terms of the baseline requirements. when violations are carried out by issuers with a big enough market share they're considered too big to fail. if google were to nullify all of the symantec-issued certificates overnight, it might cause widespread outages. my firefox still trusts symantec root certificates. because google, mozilla and others prioritize keeping the web working over keeping it secure, deleting misbehaving big cas from trust lists won't happen. when mozilla writes:you are definitely connected to the website whose address is shown in the address bar; the connection has not been intercepted. they are assuming a world of honest cas that isn't this world. if you have the locked padlock icon in your url bar, you are probably talking to the right web site, but there is a chance you aren't.[slide ]recent data from anti-phishing company phishlabs shows that percent of all phishing sites in the third quarter of bore the padlock security icon next to the phishing site domain name as displayed in a browser address bar. that’s up from percent just one year ago, and from percent in the second quarter of . brian krebs half of all phishing sites now have the padlockbuilding on earlier work by wendlandt et al, moxie marlinspike, the eff and others, in google started work on an approach specified in rfc , and called certificate transparency (ct). the big difference from earlier efforts, which didn't require cooperation from website owners and cas, was that google's did require cooperation and they had enough leverage to obtain it:[slide ]google's certificate transparency project fixes several structural flaws in the ssl certificate system, which is the main cryptographic system that underlies all https connections. these flaws weaken the reliability and effectiveness of encrypted internet connections and can compromise critical tls/ssl mechanisms, including domain validation, end-to-end encryption, and the chains of trust set up by certificate authorities. if left unchecked, these flaws can facilitate a wide range of security attacks, such as website spoofing, server impersonation, and man-in-the-middle attacks.certificate transparency helps eliminate these flaws by providing an open framework for monitoring and auditing ssl certificates in nearly real time. specifically, certificate transparency makes it possible to detect ssl certificates that have been mistakenly issued by a certificate authority or maliciously acquired from an otherwise unimpeachable certificate authority. it also makes it possible to identify certificate authorities that have gone rogue and are maliciously issuing certificates. certificate transparencythe basic idea is to accompany the certificate with a hash of the certificate signed by a trusted third party, attesting that the certificate holder told the third party that the certificate with that hash was current. thus in order to spoof a service, an attacker would have to both obtain a fraudulent certificate from a ca, and somehow persuade the third party to sign a statement that the service had told them the fraudulent certificate was current. clearly this is:more secure than the current situation, which requires only compromising a ca, and:more effective than client-only approaches, which can detect that a certificate has changed but not whether the change was authorized.ct also requires participation from browser manufacturers:in order to improve the security of extended validation (ev) certificates, google chrome requires certificate transparency (ct) compliance for all ev certificates issued after jan . clients now need two lists of trusted third parties, the cas and the sources of ct attestations. the need for these trusted third parties is where the blockchain enthusiasts would jump in and claim (falsely) that using a blockchain would eliminate the need for trust. but ct has a much more sophisticated approach, ronald reagan's "trust, but verify". in the real world it isn't feasible to solve the problem of untrustworthy cas by eliminating the need for trust. ct's approach instead is to provide a mechanism by which breaches of trust, both by the cas and by the attestors, can be rapidly and unambiguously detected.[slide ]sourcehere is a brief overview of how ct works to detect breaches of trust. the system has the following components:logs, to which cas report their current certificates, and from which they obtain attestations, called signed certificate timestamps (scts), that owners can attach to their certificates. clients can verify the signature on the sct, then verify that the hash it contains matches the certificate. if it does, the certificate was the one that the ca reported to the log, and the owner validated. it is envisaged that there will be tens but not thousands of logs; chrome currently trusts logs operated by organizations. each log maintains a merkle tree data structure of the certificates for which it has issued scts.monitors, which periodically download all newly added entries from the logs that they monitor, verify that they have in fact been added to the log, and perform a series of validity checks on them. they also thus act as backups for the logs they monitor.auditors, which use the merkle tree of the logs they audit to verify that certificates have been correctly appended to the log, and that no retroactive insertions, deletions or modifications of the certificates in the log have taken place. clients can use auditors to determine whether a certificate appears in a log. if it doesn't, they can use the sct to prove that the log misbehaved.in this way, auditors, monitors and clients cooperate to verify the correct operation of logs, which in turn provides clients with confidence in the [certificate,attestation] pairs they use to secure their communications. although the process works if certificate owners each obtain their scts from only one log, they should get them from multiple logs and send a random selection of their scts to each client to improve robustness. note the key architectural features of ct:[slide ] certificate transparency architecture:the certificate data is held by multiple independent services.they get the data directly from the source, not via replication from other services.clients access the data from a random selection of the services.there is an audit process continually monitoring the services looking for inconsistencies.these are all also features of the protocol underlying the lockss digital preservation system, published in . in both cases, the random choice among a population of independent services makes life hard for attackers. if they are to avoid detection, they must compromise the majority of the services, and provide correct information to auditors while providing false information to victims.looking at the list of logs chrome currently trusts, it is clear that almost all are operated by cas themselves. assuming that each monitor at each ca is monitoring some of the other logs as well as the one it operates, this does not represent a threat, because misbehavior by that ca would be detected by other cas. a ca's monitor that was tempted to cover up misbehavior by a different ca's log it was monitoring would risk being "named and shamed" by some other ca monitoring the same log, just as the misbehaving ca would be "named and shamed".it is important to observe that, despite the fact that cas operate the majority of the ct infrastructure, its effectiveness in disciplining cas is not impaired. all three major cas have suffered reputational damage from recent security failures, although because they are "too big to fail" this hasn't impacted their business much. however, as whales in a large school of minnows it is in their interest to impose costs (for implementing ct) and penalties (for security lapses) on the minnows. note that google was sufficiently annoyed with symantec's persistent lack of security that it set up its own ca. the threat that their business could be taken away by the tech oligopoly is real, and cooperating with google may have been the least bad choice. because these major corporations have an incentive to pay for the ct infrastructure, it is sustainable in a way that a market of separate businesses, or a permissionless blockchain supported by speculation in a cryptocurrency would not be.fundamentally, if applications such as ct attempt to provide absolute security they are doomed to fail, and their failures will be abrupt and complete. it is more important to provide the highest level of security compatible with resilience, so that the inevitable failures are contained and manageable. this is one of the reasons why permissionless blockchains, subject to % attacks, and permissioned blockchains, with a single, central point of failure, are not suitable.software supply chain[slide ] when the mass compromise came to light last month, microsoft said the hackers also stole signing certificates that allowed them to impersonate any of a target’s existing users and accounts through the security assertion markup language. typically abbreviated as saml, the xml-based language provides a way for identity providers to exchange authentication and authorization data with service providers. the full impact of the recent compromise of solarwind's orion network management software will likely never be known, it affected at least , networks, including microsoft's and:the treasury department, the state department, the commerce department, the energy department and parts of the pentagon it was not detected by any of the us government's network monitoring systems, but by fireeye, a computer security company that was also a victim. but for a mistake by the attackers at fireeye it would still be undetected. it was an extremely sophisticated attack, which has rightfully gained a lot of attention.to understand how defenses against attacks like this might work, it is first necessary to understand how the supply chain that installs and updates the software on your computer works. i'll use apt, the system used by debian linux and its derivatives, as the example.a system running debian or another apt-based linux distribution runs software it received in "packages" that contain the software files, and metadata that includes dependencies. their hashes can be verified against those in a release file, signed by the distribution publisher. packages come in two forms, source and compiled. the source of a package is signed by the official package maintainer and submitted to the distribution publisher. the publisher verifies the signature and builds the source to form the compiled package, whose hashes are then included in the release file.the signature on the source package verifies that the package maintainer approves this combination of files for the distributor to build. the signature on the release file verifies that the distributor built the corresponding set of packages from approved sources and that the combination is approved for users to install.[slide ] there are thus two possible points of entry for an attacker:they could compromise the developer, so that the signed source code files received by the distributor contained malware (type a),or they could compromise the distributor, so that the package whose hash was in the signed release file did not reflect the signed source code, but contained malware (type b).an example of a type a attack occurred in november . dan goodin reported that:the malicious code was inserted in two stages into event-stream, a code library with million downloads that's used by fortune companies and small startups alike. in stage one, version . . , published on september , included a benign module known as flatmap-stream. stage two was implemented on october when flatmap-stream was updated to include malicious code that attempted to steal bitcoin wallets and transfer their balances to a server located in kuala lumpur. how were the attackers able to do this? goodin explains:according to the github discussion that exposed the backdoor, the longtime event-stream developer no longer had time to provide updates. so several months ago, he accepted the help of an unknown developer. the new developer took care to keep the backdoor from being discovered. besides being gradually implemented in stages, it also narrowly targeted only the copay wallet app. the malicious code was also hard to spot because the flatmap-stream module was encrypted. all that was needed to implement this type a attack was e-mail and github accounts, and some social engineering.dan goodin describes a simple type b attack in new supply chain attack uses poisoned updates to infect gamers’ computers:in a nutshell, the attack works this way: on launch, nox.exe sends a request to a programming interface to query update information. the bignox api server responds with update information that includes a url where the legitimate update is supposed to be available. eset speculates that the legitimate update may have been replaced with malware or, alternatively, a new filename or url was introduced.malware is then installed on the target’s machine. the malicious files aren’t digitally signed the way legitimate updates are. that suggests the bignox software build system isn’t compromised; only the systems for delivering updates are. the malware performs limited reconnaissance on the targeted computer. the attackers further tailor the malicious updates to specific targets of interest. [slide ] sourcethe solarwinds attackers tried but failed to penetrate the network of crowdstrike, another computer security company. sunspot: an implant in the build process, crowdstrike's analysis of the attack, reveals the much greater sophistication of this type b attack. once implanted in solarwinds' build system:sunspot runs once a second scanning for instances of msbuild.exe, the tool used to build the target software.if sunspot finds an msbuild.exe, it next locates the directory in which the build is running.then sunspot checks whether what is being built is the target software.if it is, sunspot checks whether the target source file has changed.if it hasn't, sunspot carefully substitutes the modified source file for the target source file.sunspot waits until the build completes, then carefully restores the target source file and erases the traces of its work.solarwinds forensic timeline shows that the attackers penetrated their network in september , and a month later tested sunspot by injecting test code into the next release of orion. an improved sunspot was deployed from february to june, when it was removed having successfully compromised the orion release with the production malware. no-one noticed until december, when fireeye spotted suspicious activity on their internal network and traced it to orion.microsoft's analysis reveals a lot more sophistication of the attacker's operations once they had penetrated the network:[slide ]each cobalt strike dll implant was prepared to be unique per machine and avoided at any cost overlap and reuse of folder name, file name, export function names, c domain/ip, http requests, timestamp, file metadata, config, and child process launched. this extreme level of variance was also applied to non-executable entities, such as wmi persistence filter name, wmi filter query, passwords used for -zip archives, and names of output log files. how could software supply chains be enhanced to resist these attacks? in an important paper entitled software distribution transparency and auditability, benjamin hof and georg carle from tu munich:describe how apt works to maintain up-to-date software on clients by distributing signed packages.review previous efforts to improve the security of this process.propose to enhance apt's security by layering a system similar to certificate transparency (ct) on top.detail the operation of their systems' logs, auditors and monitors, which are similar to ct's in principle but different in detail.describe and measure the performance of an implementation of their layer on top of apt using the trillian software underlying some ct implementations.their system's ct-like logs contain the hashes of both the source and the binaries of each version of each package, and ensure that attackers would be detected if they, for example, create a short-lived version containing malware for a specific victim. it certainly defeats a significant class of attacks but, alas, does not address either the solarwinds or the event-stream attacks.as regards the solarwinds attack, there are two important "missing pieces" in their system, and all the predecessors. each is the subject of a separate effort:[slide ]reproducible builds.bootstrappable compilers.suppose solarwinds had been working in hof and carle's system. they would have signed their source code, built it, and signed the resulting binaries. the attackers would have arranged that the source that was built was not the source that solarwinds signed, but solarwinds would not have known that. so the signatures on both the unmodified source and the modified binaries would appear valid in the logs, but the binaries would be malign.the problem is that the connection between the source and the binaries rests on an assumption that the distributor's build environment has not been compromised - i.e. no type b attack. as with the multiple logs of ct, what is needed is multiple independent builds of the signed source. unless all of the independent build environments are compromised, a compromised build will differ from the others because it contains malware.this is a great idea, but in practice it is very hard to achieve for both technical and organizational reasons:the first technical reason is that in general, building the same source twice results in different binaries. compiler and linker output typically contains timestamps, temporary file names, and other sources of randomness. the build system needs to be reproducible.the second technical reason is that, in order to be reproducible, the multiple independent builds have to use the same build environment. so each of the independent build environments will have the same vulnerabilities, allowing for the possibility that the attacker could compromise them all.the organizational reason is that truly independent builds can only be done in an open source environment in which anyone, and in particular each of the independent builders, can access the source code.to enable binaries to be securely connected to their source, a reproducible builds effort has been under way for more than years. debian project lead chris lamb's -minute talk think you're not a target? a tale of developers ... provides an overview of the problem and the work to solve it using three example compromises:alice, a package developer who is blackmailed to distribute binaries that don't match the public source (a type a attack).bob, a build farm sysadmin whose personal computer has been compromised, leading to a compromised build toolchain in the build farm that inserts backdoors into the binaries (a type b attack).carol, a free software enthusiast who distributes binaries to friends. an evil maid attack has compromised her laptop.as lamb describes, eliminating all sources of irreproducibility from a package is a painstaking process because there are so many possibilities. they include non-deterministic behaviors such as iterating over hashmaps, parallel builds, timestamps, build paths, file system directory name order, and so on. the work started in with % of debian packages building reproducibly. currently, around % of the debian packages for the amd and arm architectures are now reproducible. that is good, but % coverage is really necessary to provide security.[slide ] way back in , paul karger and roger schell discovered a devastating attack against computer systems. ken thompson described it in his classic speech, "reflections on trusting trust." basically, an attacker changes a compiler binary to produce malicious versions of some programs, including itself. once this is done, the attack perpetuates, essentially undetectably. thompson demonstrated the attack in a devastating way: he subverted a compiler of an experimental victim, allowing thompson to log in as root without using a password. the victim never noticed the attack, even when they disassembled the binaries -- the compiler rigged the disassembler, too. in , bruce schneier summarized the message of perhaps the most famous of acm's annual turing award lectures. in this attack, the compromised build environment inserts malware even though it is building the unmodified source code. unlike the solarwinds attack, the signatures testifying that the binaries are the output of building the signed source code are correct.[slide ] sourcethis is the motivation for the bootstrappable builds project, whose goal is to create a process for building a complete toolchain starting from a "seed" binary that is simple enough to be certified "by inspection". recently, they achieved a major milestone. starting from a tiny "seed" binary, they were able to create a working tinycc compiler for the arm architecture. starting from tinycc, it is possible to build the entire gnucc toolchain and thus, in principle, a working linux. there is clearly a long way still to go to a bootstrapped full toolchain proof against type b attacks.the event-stream attack can be thought of as the organization-level analog of a sybil attack on a peer-to-peer system. creating an e-mail identity is almost free. the defense against sybil attacks is to make maintaining and using an identity in the system expensive. as with proof-of-work in bitcoin, the idea is that the white hats will spend more (compute more useless hashes) than the black hats. even this has limits. eric budish's analysis shows that, if the potential gain from an attack on a blockchain is to be outweighed by its cost, the value of transactions in a block must be less than the block reward.would a similar defense against "sybil" type a attacks on the software supply chain be possible? there are a number of issues:the potential gains from such attacks are large, both because they can compromise very large numbers of systems quickly (event-stream had m downloads), and because the banking credentials, cryptocurrency wallets, and other data these systems contain can quickly be converted into large amounts of cash.thus the penalty for mounting an attack would have to be an even larger amount of cash. package maintainers would need to be bonded or insured for large sums, which implies that distributions and package libraries would need organizational structures capable of enforcing these requirements.bonding and insurance would be expensive for package maintainers, who are mostly unpaid volunteers. there would have to be a way of paying them for their efforts, at least enough to cover the costs of bonding and insurance.thus users of the packages would need to pay for their use, which means the packages could neither be free, nor open source.which would make implementing the reproducible builds and bootstrapped compilers needed to defend against type b attacks extremely difficult.the foss (free open source software) movement will need to find other ways to combat sybil attacks, which will be hard if the reward for a successful attack greatly exceeds the cost of mounting it. adequately rewarding maintainers for their essential but under-appreciated efforts is a fundamental problem for foss.it turns out that this talk is timely. two days ago, eric brewer, rob pike et al from google posted know, prevent, fix: a framework for shifting the discussion around vulnerabilities in open source, an important and detailed look at the problem of vulnerabilities in open source and what can be done to reduce them. their summary is:it is common for a program to depend, directly or indirectly, on thousands of packages and libraries. for example, kubernetes now depends on about , packages. open source likely makes more use of dependencies than closed source, and from a wider range of suppliers; the number of distinct entities that need to be trusted can be very high. this makes it extremely difficult to understand how open source is used in products and what vulnerabilities might be relevant. there is also no assurance that what is built matches the source code. taking a step back, although supply-chain attacks are a risk, the vast majority of vulnerabilities are mundane and unintentional—honest errors made by well-intentioned developers. furthermore, bad actors are more likely to exploit known vulnerabilities than to find their own: it’s just easier. as such, we must focus on making fundamental changes to address the majority of vulnerabilities, as doing so will move the entire industry far along in addressing the complex cases as well, including supply-chain attacks. the bulk of their post addresses improvements to the quality of the development process, with three goals:know about the vulnerabilities in your software prevent the addition of new vulnerabilities, and fix or remove vulnerabilities. then, in a section entitled prevention for critical software they specifially address the security of the development process and thus the two types of supply chain attacks we have been discussing. they write:this is a big task, and currently unrealistic for the majority of open source. part of the beauty of open source is its lack of constraints on the process, which encourages a wide range of contributors. however, that flexibility can hinder security considerations. we want contributors, but we cannot expect everyone to be equally focused on security. instead, we must identify critical packages and protect them. such critical packages must be held to a range of higher development standards, even though that might add developer friction. [slide ]define criteria for “critical” open source projects that merit higher standards no unilateral changes to critical software require code review for critical software changes to critical software require approval by two independent parties authentication for participants in critical software for critical software, owners and maintainers cannot be anonymous strong authentication for contributors of critical software a federated model for identities notification for changes in risk transparency for artifacts trust the build process their goals for the "higher development standards" include identifying the important packages that require higher standards, implementing review and signoff of changes by at least two independent developers, "transparency for artifacts", by which they mean reproducible builds, and "trust the build process" which implies a bootstrappable toolchain.they acknowledge that these are very aggressive goals, because in many ways they cut against the free-wheeling development culture of open source that has sparked its remarkable productivity. if google were to persuade other major corporations to put significant additional resources of money and manpower into implementing them they would likely succeed. absent this, the additional load on developers will likely cause resistance. andromeda yelton: archival face recognition for fun and nonprofit in , dominique luster gave a super good code lib talk about applying ai to metadata for the charles “teenie” harris collection at the carnegie museum of art — more than , photographs of black life in pittsburgh. they experimented with solutions to various metadata problems, but the one that’s stuck in my head since is the face recognition one. it sure would be cool if you could throw ai at your digitized archival photos to find all the instances of the same person, right? or automatically label them, given that any of them are labeled correctly? sadly, because we cannot have nice things, the data sets used for pretrained face recognition embeddings are things like lots of modern photos of celebrities, a corpus which wildly underrepresents ) archival photos and ) black people. so the results of the face recognition process are not all that great. i have some extremely technical ideas for how to improve this — ideas which, weirdly, some computer science phds i’ve spoken with haven’t seen in the field. so i would like to experiment with them. but i must first invent the universe set up a data processing pipeline. three steps here: fetch archival photographs;do face detection (draw bounding boxes around faces and crop them out for use in the next step);do face recognition. for step , i’m using dpla, which has a super straightforward and well-documented api and an easy-to-use python wrapper (which, despite not having been updated in a while, works just fine with python . , the latest version compatible with some of my dependencies). for step , i’m using mtcnn, because i’ve been following this tutorial. for step , face recognition, i’m using the steps in the same tutorial, but purely for proof-of-concept — the results are garbage because archival photos from mid-century don’t actually look anything like modern-day celebrities. (neural net: “i have % confidence this is stevie wonder!” how nice for you.) clearly i’m going to need to build my own corpus of people, which i have a plan for (i.e. i spent some quality time thinking about numpy) but haven’t yet implemented. so far the gotchas have been: gotcha : if you fetch a page from the api and assume you can treat its contents as an image, you will be sad. you have to treat them as a raw data stream and interpret that as an image, thusly: from pil import image import requests response = requests.get(url, stream=true) response.raw.decode_content = true data = requests.get(url).content image.open(io.bytesio(data)) this code is, of course, hilariously lacking in error handling, despite fetching content from a cesspool of untrustworthiness, aka the internet. it’s a first draft. gotcha : you see code snippets to convert images to pixel arrays (suitable for ai ingestion) that look kinda like this: np.array(image).astype('uint '). except they say astype('float ') instead of astype('uint '). i got a creepy photonegative effect when i used floats. gotcha : although pil was happy to manipulate the .pngs fetched from the api, it was not happy to write them to disk; i needed to convert formats first (image.convert('rgb')). gotcha : the suggested keras_vggface library doesn’t have a pipfile or requirements.txt, so i had to manually install keras and tensorflow. luckily the setup.py documented the correct versions. sadly the tensorflow version is only compatible with python up to . (hence the comment about dpyla compatibility above). i don’t love this, but it got me up and running, and it seems like an easy enough part of the pipeline to rip out and replace if it’s bugging me too much. the plan from here, not entirely in order, subject to change as i don’t entirely know what i’m doing until after i’ve done it: build my own corpus of identified peoplethis means the numpy thoughts, aboveit also means spending more quality time with the api to see if i can automatically apply names from photo metadata rather than having to spend too much of my own time manually labeling the corpusdecide how much metadata i need to pull down in my data pipeline and how to store itfigure out some kind of benchmark and measure ittry out my idea for improving recognition accuracybenchmark againhopefully celebrate awesomeness mita williams: weeknote ( ) § last friday i was interviewed for the podcast the grasscast — a game-themed podcast named after the book, the grasshopper: games, life, and utopia. i ramble a little bit in the episode as i tried to be more open and conversational than concise and correct. but i also spoke that way because for some of the questions, no pat answer came immediately to mind. there was one question that stumped me but in my trying to answer, i think i found something i had not considered before. the question was, what is one bad thing about games? and i tried to convey that, unlike video games where you can play with strangers, most tabletop games are generally constrained by the preferences of your social circles. in order to convince others to spend time on a game that might think is too complicated for them or not for them, you need to have be a successful evangelist. also the episode drifts into chatter about libraries, copyright and ebooks. § this week, i reviewed and published another batch of works for our institutional repository from our department of history that was prepared by our library assistants at leddy at this point, we have reviewed and uploaded the works of half the faculty from this department. i’m hoping to finish the rest this month but i think i have some outstanding h p work that might push the end of this project til march. § this morning i assisted with an online workshop called data analysis and visualization in r for ecologists that was being lead by a colleague of mine. r version . . (“bunny-wunnies freak out”) was released on - - . the release of r . . (“lost library book”) is scheduled for monday - - . § on sunday, i published a short response to “windsor works – an economic development strategy” which is going to city council on monday. why am i writing about this document here? i am mention this here because the proposed strategy (l.i.f.t.) lists the following as potential metric for measuring the strategy’s success… take it from me, someone who knows a quite a bit about citations — the city should use another metric — perhaps one pertaining to local unemployment levels instead. § a viral post from resurfaced on my fb feed this week and unlike most of the posts i read there, this one did spark joy: and it struck me how much i loved that the anti-prom was being at the library. so i started doing some research! it appears to me that some anti-proms are technically better described as alternative proms. these proms have been established as an explicitly safe place where lgbtq young people can enjoy prom. other anti-proms are true morps. i now wonder what other anti-traditions should find a home at the public library. david rosenthal: chromebook linux update my three acer c chromebooks running linux are still giving yeoman service, although for obvious reasons i'm not travelling these days. but it is time for an update to 's travels with a chromebook. below the fold, an account of some adventures in sysadmin.battery replacementthe battery in c # , which is over six years old, would no longer hold a charge. i purchased a dentsing ap j k replacement battery from amazon. i opened the c , removed the old battery, inserted the new one, closed up the case, and all was well. it was an impressively easy fix. sleeping & wakingsometime around last october linux mint switched from kernels in the . series to kernels in the . series. mint uses . series kernels. the . kernels on the c went to sleep properly when the lid closed, and woke properly when the lid opened. the . kernels appeared to go to sleep correctly, but when the lid opened did a cold boot. because this problem happens immediately on wake, and because sleep appears to work correctly, there is no useful information in the logs; this appears to be a very hard problem to diagnose.here is my work-around to use the . . - kernel (the last i installed via updates) on a vanilla linux mint installation: install linux mint . mate edition. add repositories using administration/software sources: deb http://archive.ubuntu.com/ubuntu bionic main restricted universe multiverse deb http://archive.ubuntu.com/ubuntu bionic-updates main restricted universe multiverse deb http://security.ubuntu.com/ubuntu/ bionic-security main restricted universe multiverse install kernel . . - -generic: sudo apt-get install linux-headers- . . - linux-headers- . . - -generic linux-image- . . - -generic linux-modules- . . - -generic linux-modules-extra- . . - -generic edit /etc/default/grub to show the menu of kernels: grub_timeout_style=menu grub_timeout= edit /etc/default/grub so that your most recent choice of kernel becomes the default: grub_savedefault=true grub_default=saved run update-grub after you choose the . . - kernel the first time, it should boot by default, and sleep and wake should work properly. the problem with ehci-pci on wakeup has gone away, there is no need to install the userland files from galliumos.disk & home directory encryptionnote that you should ideally install linux mint . with full-disk encryption. the release notes explain: the move to systemd caused a regression in ecrypts which is responsible for mounting/unmounting encrypted home directories when you login and logout. because of this issue, please be aware that in mint and newer releases, your encrypted home directory is no longer unmounted on logout: https://bugs.launchpad.net/ubuntu/+source/gnome-session/+bug/ . mint with a full-disk encryption had this problem but i haven't been able to reproduce it with mint and the . . - kernel. home directory encryption works, but will leave its contents decrypted after you log out, rather spoiling the point. touchpadas i described here, the touchpad isn't one of the c 's best features, and it is necessary to disable it while typing, or while using a mouse such as the excellent tracpoint. i use ataraeo's touchpad-indicator, but this doesn't seem to work on mint . out of the box, using x.org's libinput driver. the release notes discuss using the synaptics driver instead. i installed it and, after creating the directory ~/.config/autostart the touchpad-indicator starts on login and works fine. open knowledge foundation: what is a public impact algorithm? meg foulkes discusses public impact algorithms and why they matter. “when i look at the picture of the guy, i just see a big black guy. i don’t see a resemblance. i don’t think he looks like me at all.” this is what robert williams said to police when he was presented with the evidence upon which he was arrested for stealing watches in june . williams had been identified by an algorithm, when detroit police ran grainy security footage from the theft through a facial recognition system. before questioning williams, or checking for any alibi, he was arrested. it was not until the matter came to trial that detroit police admitted that he had been falsely, and solely, charged on the output of an algorithm. it’s correct to say that in many cases, when ai and algorithms go wrong, the impact is pretty innocuous – like when a music streaming service recommends music you don’t like. but often, ai and algorithms go wrong in ways that cause serious harm, as in the case of robert williams. although he had done absolutely nothing wrong, he was deprived of a fundamental right on the basis of a computer output: his liberty. it’s not just on an individual scale that these harms are felt. algorithms are written by humans, so they can reflect human biases. what algorithms can do is amplify, through automatedly entrenching the bias, this prejudice over a massive scale. the bias isn’t exclusively racialised; last year, an algorithm used to determine exam grades disproportionately downgraded disadvantaged students. throughout the pandemic, universities have been turning to remote proctoring software that falsely identifies students with disabilities as cheats. for example, those who practice self-stimulatory behaviour or ‘stimming’ may get algorithmically flagged again and again for suspicious behaviour, or have to disclose sensitive medical information to avoid this. we identify these types of algorithms as ‘public impact algorithms’ to clearly name the intended target of our concern. there is a big difference between the harm caused by inaccurate music suggestions and algorithms that have the potential to deprive us of our fundamental rights. to call out these harms, we have to precisely define the problem. only then can we hold the deployers of public impact algorithms to account, and ultimately to achieve our mission of ensuring public impact algorithms do no harm. sign up to our mailing list or follow the open knowledge justice programme on twitter to receive updates. ed summers: outgoing tree rings by tracy o this post is really about exploring historical datasets with version control systems. mark graham posed a question to me earlier today asking what we know about the twitter accounts of the members of congress, specifically whether they have been removed after they left office. the hypothesis was that some members of the house and senate may decide to delete their account on leaving dc. i was immediately reminded of the excellent congress-legislators project which collects all kinds of information about house and senate members including their social media accounts into yaml files that are versioned in a github repository. github is a great place to curate a dataset like this because it allows anyone with a github account to contribute to editing the data, and to share utilities to automate checks and modifications. unfortunately the file that tracks social media accounts is only for current members. once they leave office they are removed from the file. the project does track other historical information for legislators. but the social media data isn’t pulled in when this transition happens, or so it seems. luckily git doesn’t forget. since the project is using a version control system all of the previously known social media links are in the history of the repository! so i wrote a small program that uses gitpython to walk the legislators-social-media.yaml file backwards in time through each commit, parse the yaml at that previous state, and merge that information into a union of all the current and past legislator information. you can see the resulting program and output in us-legislators-social. there’s a little bit of a wrinkle in that not everything in the version history should be carried forward because errors were corrected and bugs were fixed. without digging into the diffs and analyzing them more it’s hard to say whether a commit was a bug fix or if it was simply adding new or deleting old information. if the yaml doesn’t parse at a particular state that’s easy to ignore. it also looks like the maintainers split out account ids from account usernames at one point. derek willis helpfully pointed out to me that twitter don’t care about the capitalization of usernames in urls, so these needed to be normalized when merging the data. the same is true of facebook, instagram and youtube. i guarded against these cases but if you notice other problems let me know. with the resulting merged historical data it’s not too hard to write a program to read in the data, identify the politicians who left office after the th congress, and examine their twitter accounts to see that they are live. it turned out to be a little bit harder than i expected because it’s not as easy as you might think to check if a twitter account is live or not. twitter’s web servers return a http ok message even when responding to requests for urls of non-existent accounts. to complicate things further the error message that displays indicating it is not an account only displays when the page is rendered in a browser. so a simple web scraping job that looks at the html is not sufficient. and finally just because a twitter username no longer seems to work, it’s possible that the user has changed it to a new screen_name. fortunately the unitedstates project also tracks the twitter user id (sometimes). if the user account is still there you can use the twitter api to look up their current screen_name and see if it is different. after putting all this together it’s possible to generate a simple table of legislators who left office at the end of the th congress, and their twitter account information. name url url_ok user_id new_url lamar alexander https://twitter.com/senalexander true michael b. enzi https://twitter.com/senatorenzi true pat roberts https://twitter.com/senpatroberts true tom udall https://twitter.com/senatortomudall true justin amash https://twitter.com/justinamash true rob bishop https://twitter.com/reprobbishop true k. michael conaway https://twitter.com/conawaytx true susan a. davis https://twitter.com/repsusandavis false eliot l. engel https://twitter.com/repeliotengel true bill flores https://twitter.com/repbillflores false cory gardner https://twitter.com/sencorygardner true peter t. king https://twitter.com/reppeteking true steve king https://twitter.com/stevekingia true daniel lipinski https://twitter.com/replipinski true david loebsack https://twitter.com/daveloebsack true nita m. lowey https://twitter.com/nitalowey true kenny marchant https://twitter.com/repkenmarchant true pete olson https://twitter.com/reppeteolson true martha roby https://twitter.com/repmartharoby false https://twitter.com/martharobyal david p. roe https://twitter.com/drphilroe true f. james sensenbrenner, jr. https://twitter.com/jimpressoffice false josé e. serrano https://twitter.com/repjoseserrano true john shimkus https://twitter.com/repshimkus true mac thornberry https://twitter.com/mactxpress true scott r. tipton https://twitter.com/reptipton true peter j. visclosky https://twitter.com/repvisclosky true greg walden https://twitter.com/repgregwalden true rob woodall https://twitter.com/reprobwoodall true ted s. yoho https://twitter.com/reptedyoho true doug collins https://twitter.com/repdougcollins true tulsi gabbard https://twitter.com/tulsipress true susan w. brooks https://twitter.com/susanwbrooks true joseph p. kennedy iii https://twitter.com/repjoekennedy false https://twitter.com/joekennedy george holding https://twitter.com/repholding true denny heck https://twitter.com/repdennyheck false https://twitter.com/ltgovdennyheck bradley byrne https://twitter.com/repbyrne true ralph lee abraham https://twitter.com/repabraham true will hurd https://twitter.com/hurdonthehill true david perdue https://twitter.com/sendavidperdue true mark walker https://twitter.com/repmarkwalker true francis rooney https://twitter.com/reprooney true paul mitchell https://twitter.com/reppaulmitchell true doug jones https://twitter.com/sendougjones true tj cox https://twitter.com/reptjcox true gilbert ray cisneros, jr. https://twitter.com/repgilcisneros true harley rouda https://twitter.com/repharley true ross spano https://twitter.com/reprossspano true debbie mucarsel-powell https://twitter.com/repdmp true donna e. shalala https://twitter.com/repshalala false abby finkenauer https://twitter.com/repfinkenauer true steve watkins https://twitter.com/rep_watkins false xochitl torres small https://twitter.com/reptorressmall true max rose https://twitter.com/repmaxrose true anthony brindisi https://twitter.com/repbrindisi true kendra s. horn https://twitter.com/repkendrahorn false https://twitter.com/kendrashorn joe cunningham https://twitter.com/repcunningham true ben mcadams https://twitter.com/repbenmcadams false https://twitter.com/benmcadamsut denver riggleman https://twitter.com/repriggleman true in most cases where the account has been updated the individual simply changed their twitter username, sometimes remove “rep” from it–like repjoekennedy to joekennedy. as an aside i’m kind of surprised that twitter username wasn’t taken to be honest. maybe that’s a perk of having a verified account or of being a politician? but if you look closely you can see there were a few that seemed to have deleted their account altogether: name url url_ok user_id susan a. davis https://twitter.com/repsusandavis false bill flores https://twitter.com/repbillflores false f. james sensenbrenner, jr. https://twitter.com/jimpressoffice false donna e. shalala https://twitter.com/repshalala false steve watkins https://twitter.com/rep_watkins false there are two notable exceptions to this. the first is vice president kamala harris. my logic for determining if a person was leaving congress was to see if they served in a term ending on - - , and weren’t serving in a term starting then. but harris is different because her term as a senator is listed as ending on - - . her old account (???) is no longer available, but her twitter user id is still active and is now attached to the account at (???). the other of course is joe biden, who stopped being a senator in order to become the president. his twitter account remains the same at (???). it’s worth highlighting here how there seems to be no uniform approach to handling this process. in one case (???) is temporarily blessed as the vp, with a unified account history underneath. in the other there is a separation between (???) and (???). it seems like twitter has some work to do on managing identities, or maybe the congress needs to prescribe a set of procedures? or maybe i’m missing part of the picture, and that just as (???) somehow changed back to (???) there is some namespace management going on behind the scenes? if you are interested in other social media platforms like facebook, instagram and youtube the unitedstates project tracks information for those platforms too. i merged that information into the legislators.yaml file i discussed here if you want to try to check them. i think that one thing this experiment shows is that if the platform allows for usernames to be changed it is critical to track the user id as well. i didn’t do the work to check that those accounts exist. but that’s a project for another day. i’m not sure this list of five deleted accounts is terribly interesting at the end of all this. possibly? but on the plus side i did learn how to interact with git better from python, which is something i can imagine returning to in the future. it’s not every day that you have to think of the versions of a dataset as an important feature of the data, outside of serving as a backup that can be reverted to if necessary. but of course data changes in time, and if seeing that data over time is useful, then the revision history takes on a new significance. it’s nothing new to see version control systems as critical data provenance technologies, but it felt new to actually use one that way to answer a question. thanks mark! david rosenthal: stablecoins i have long been skeptical of bitcoin's "price" and, despite its recent massive surge, i'm still skeptical. but it turns out i was wrong two years ago when i wrote in blockchain: what's not to like?:permissionless blockchains require an inflow of speculative funds at an average rate greater than the current rate of mining rewards if the "price" is not to collapse. to maintain bitcoin's price at $ k requires an inflow of $ k/hour. i found it hard to believe that this much actual money would flow in, but since then bitcoin's "price" hasn't dropped below $ k, so i was wrong. caution — i am only an amateur economist, and what follows below the fold is my attempt to make sense of what is going on.first, why did i write that? the economic argument is that, because there is a low barrier to entry for new competitors, margins for cryptocurrency miners are low. so the bulk of their income in terms of mining rewards has to flow out of the system in "fiat" currency to pay for their expenses such as power and hardware. these cannot be paid in cryptocurrencies. at the time, the bitcoin block reward was . btc/block, or btc/hour. at $ k/btc this was $ k/hour, so on average k usd/hour had to flow in from speculators if the system was not to run out of usd.sourcewhat has happened since then? miners' income comes in two parts, transaction fees (currently averaging around btc/day) and mining rewards ( btc/day) for a total around k btc/day. at $ k/btc, that is $ k/hour. the combination of halving of the block reward, increasing transaction fees, and quintupling the "price" has roughly tripled the required inflow.second, lets set the context for what has happened in cryptocurrencies in the last year.sourcein the last year bitcoin's "market cap" went from around $ b to around $ b ( . x) and its "price" went from about $ k to about $ k.sourcein the last year ethereum's "market cap" went from around $ b to around $ b ( . x) and its "price went from around $ to around $ .sourcethe key observation that explains why i write "price" in quotes is shown in this graph. very little of the trading in btc is in terms of usd, most of it is in terms of tether (usdt). the "price" is set by how many usdt people are prepared to pay for btc, not by how many usd. the usd "price" follows because people believe that usdt ≅ usd. sourcein the past year, tether's "market cap" has gone from about b usdt to about b usdt ( x).tether (usdt) is a "stablecoin", intended to maintain a stable price of usd = usdt. initially, tether claimed that it maintained a stable "price" because every usdt was backed by an actual usd in a bank account. does that mean that investors transferred around sixteen billion us dollars into tether's bank account in the past year? no-one believes that. there has never been an audit of tether to confirm what is backing usdt. tether themselves admitted to the new york attorney general in october that:the $ . billion worth of tethers are only % backed:tether has cash and cash equivalents (short term securities) on hand totaling approximately $ . billion, representing approximately percent of the current outstanding tethers. if usdt isn't backed by usd, what is backing it, and is usdt really worth usd?sourcejust in october, tether minted around b usdt. the graph tracks the "price" of bitcoin against the "market cap" of usdt. does it look like they're correlated? amy castor thinks so.tether transfers newly created usdt to an exchange, where one of two things can happen to it: it can be used to buy usd or an equivalent "fiat" currency. but only a few exchanges allow this. for example, coinbase, the leading regulated exchange, will not provide this "fiat off-ramp": please note that coinbase does not support usdt — do not send it to your bitcoin account on coinbase. because of usdt's history and reputation, exchanges that do offer a "fiat off-ramp" are taking a significant risk, so they will impose a spread; the holder will get less than $ . why would you send $ to tether to get less than $ back? it can be used to buy another cryptocurrency, such as bitcoin (btc) or ethereum (eth), increasing demand for that cryptocurrency and thus increasing its price.since newly created usdt won't be immediately sold for "fiat", they will pump the "price" of cryptocurrencies.for simplicity of explanation, lets imagine a world in which there are only usd, usdt and btc. in this world some proportion of the backing for usdt is usd and some is btc.someone sends usd to tether. why would they do that? they don't want usdt as a store of value, because they already have usd, which is obviously a better store of value than usdt. they want usdt in order to buy btc. tether adds the usd to the backing for usdt, and issues the corresponding number of usdt, which are used to buy btc. this pushes the "price" of btc up, which increases the "value" of the part of the backing for usdt that is btc. so tether issues the corresponding amount of usdt, which is used to buy btc. this pushes the "price" of btc up, which increases the "value" of the part of the backing for usdt that is btc. ...tether has a magic "money" pump, creating usdt out of thin air. but there is a risk. suppose for some reason the "price" of btc goes down, which reduces the "value" of the backing for usdt. now there are more usdt in circulation than are backed. so tether must buy some usdt back. they don't want to spend usd for this, because they know that usd are a better store of value than usdt created out of thin air. so they need to sell btc to get usdt. this pushes the "price" of btc down, which reduces the "value" of the part of the backing for usdt that is btc. so tether needs to buy more usdt for btc, which pushes the "price" of btc down. ...the magic "money" pump has gone into reverse, destroying the usdt that were created out of thin air. tether obviously wants to prevent this happening, so in our imaginary world what we would expect to see is that whenever the "price" of btc goes down, tether supplies the market with usdt, which are used to buy btc, pushing the price back up. over time, the btc "price" would generally go up, keeping everybody happy. but there is a second-order effect. over time, the proportion of the backing for usdt that is btc would go up too, because each usd that enters the backing creates r> usd worth of "value" of the btc part of the backing. and, over time, this effect grows because the greater the proportion of btc in the backing, the greater r becomes.sourcein our imaginary world we would expect to see:the "price" of btc correlated with the number of usdt in circulation. the graph shows this in the real world.both the "price" of btc and the number of usdt in circulation growing exponentially. the graph shows this in the real world.spikes in the number of usdt in circulation following falls in the "price" of btc. is bitcoin really untethered? by john griffin and amit shams shows that:rather than demand from cash investors, these patterns are most consistent with the supply‐based hypothesis of unbacked digital money inflating cryptocurrency prices. their paper was originally published in and updated in and .tether being extremely reluctant to be audited because that would reveal how little money and how much "money" was supporting the btc "price".our imaginary world replicates key features of the real world. of course, since tether has never been audited, we don't know the size or composition of usdt's backing. so we don't know whether tether has implemented a magic "money" pump. but the temptation to get rich quick by doing so clearly exists, and tether's history isn't reassuring about their willingness to skirt the law. because of the feedback loops i described, if they ever dipped a toe in the flow from a magic "money" pump, they would have to keep doubling down.apart from the work of griffin and shams, there is a whole literature pointing out the implausibility of tether's story. here are a few highlights:jp konig's things about tether stablecoinssocial capital's series explaining tether and the "stablecoin" scam:pumps, spoofs and boiler roomstether, part one: the stablecoin dreamtether, part two: pokedextether, part three: crypto islandprice manipulation in the bitcoin ecosystem by neil gandal et alcryptocurrency pump-and-dump schemes by tao li et alpatrick mckenzie's tether: the story so far:a friend of mine, who works in finance, asked me to explain what tether was.short version: tether is the internal accounting system for the largest fraud since madoff. bernie madoff's $ . b ponzi scheme was terminated in but credible suspicions had been raised nine years earlier, not least by the indefatigable harry markopolos. credible suspicions were raised against wirecard shortly after it was incorporated in , but even after the financial times published a richly documented series based on whistleblower accounts it took almost a year before wirecard declared bankruptcy owing € . b.massive frauds suffer from a "wile e. coyote" effect. because they are "too big to fail" there is a long time between the revelation that they are frauds, and the final collapse. it is hard for people to believe that, despite numbers in the billions, there is no there there. both investors and regulators get caught up in the excitement and become invested in keeping the bubble inflated by either attacking or ignoring negative news. for example, we saw this in the wirecard scandal:bafin conducted multiple investigations against journalists and short sellers because of alleged market manipulation, in response to negative media reporting of wirecard. ... critics cite the german regulator, press and investor community's tendency to rally around wirecard against what they perceive as unfair attack. ... after initially defending bafin's actions, its president felix hufeld later admitted the wirecard scandal is a "complete disaster". similarly, the cryptocurrency world has a long history of both attacking and ignoring realistic critiques. an example of ignoring is the dao:the decentralized autonomous organization (the dao) was released on th april , but on th may dino mark, vlad zamfir, and emin gün sirer posted a call for a temporary moratorium on the dao, pointing out some of its vulnerabilities; it was ignored. three weeks later, when the dao contained about % of all the ether in circulation, a combination of these vulnerabilities was used to steal its contents. sourcethe graph shows how little of the trading in btc is in terms of actual money, usd. on coinmarketcap.com as i write, usdt has a "market cap" of nearly $ b and the next largest "stablecoin" is usdc, at just over $ . b. usdc is audited and[ ] complies with banking regulations, which explains why it is used so much less. the supply of usdc can't expand enough to meet demand. the total "market cap" of all the cryptocurrencies the site tracks is $ b, an increase of more than % in the last day! so just one day is around the same as bernie madoff's ponzi scheme. the top cryptocurrencies (btc, eth, xrp, usdt) account for $ b ( %) of the total "market cap"; the others are pretty insignificant.david gerard points out the obvious in tether is “too big to fail” — the entire crypto industry utterly depends on it:the purpose of the crypto industry, and all its little service sub-industries, is to generate a narrative that will maintain and enhance the flow of actual dollars from suckers, and keep the party going.increasing quantities of tethers are required to make this happen. we just topped twenty billion alleged dollars’ worth of tethers, sixteen billion of those just since march . if you think this is sustainable, you’re a fool. gerard links to bryce weiner's hopes, expectations, black holes, and revelations — or how i learned to stop worrying and love tether which starts from the incident in april of when bitfinex, the cryptocurrency exchange behind tether, encountered a serious problem:the wildcat bank backing tether was raided by interpol for laundering of criminally obtained assets to the tune of about $ , , . the percentage of that sum which was actually bitfinex is a matter of some debate but there’s no sufficient reason not to think it was all theirs....the nature of the problem also presented a solution: instead of backing tether in actual dollars, stuff a bunch of cryptocurrency in a basket to the valuation of the cash that got seized and viola! a black hole is successfully filled with a black hole, creating a stable asset. at the time, usdt's "market cap" was around $ . b, so assuming tether was actually backed by usd at that point, it lost % of its backing. this was a significant problem, more than enough to motivate shenanigans.weiner goes on to provide a detailed explanation, and argue that tether is impossible to shut down. he may be right, but it may be possible to effectively eliminate the "fiat off-ramp", thus completely detaching usdt and usd. this would make it clear that "prices" expressed in usdt are imaginary, not the same as prices expressed in usd.sourcepostscript: david gerard recounts the pump that pushed btc over $ k:we saw about million tethers being lined up on binance and huobi in the week previously. these were then deployed en masse.you can see the pump starting at : utc on december. btc was $ , . on coinbase at : utc. notice the very long candles, as bots set to sell at $ , sell directly into the pump. see cryptocurrency pump-and-dump schemes by tao li, donghwa shin and baolian wang.sourceki joung yu watched the pump in real time:lots of people deposited stablecoins to exchanges mins before breaking $ k.price is all about consensus. i guess the sentiment turned around to buy $btc at that time....eth block interval is - seconds.this chart means exchange users worldwide were trying to deposit #stablecoins in a single block — seconds. note that " mins" is about one bitcoin block time, and by "exchange users" he means "addresses — it could have been a pre-programmed "smart contract".[ ] david gerard points out that:usdc loudly touts claims that it’s well-regulated, and implies that it’s audited. but usdc is not audited — accountants grant thornton sign a monthly attestation that centre have told them particular things, and that the paperwork shows the right numbers. jonathan rochkind: product management in my career working in the academic sector, i have realized that one thing that is often missing from in-house software development is “product management.” but what does that mean exactly? you don’t know it’s missing if you don’t even realize it’s a thing and people can use different terms to mean different roles/responsibilities. basically, deciding what the software should do. this is not about colors on screen or margins (what our stakeholderes often enjoy micro-managing) — i’d consider those still the how of doing it, rather than the what to do. the what is often at a much higher level, about what features or components to develop at all. when done right, it is going to be based on both knowledge of the end-user’s needs and preferences (user research); but also knowledge of internal stakeholder’s desires and preferences (overall organiational strategy, but also just practically what is going to make the right people happy to keep us resourced). also knowledge of the local capacity, what pieces do we need to put in place to get these things developed. when done seriously, it will necessarily involve prioritization — there are many things we could possibly done, some subset of them we very well may do eventually, but which ones should we do now? my experience tells me it is a very big mistake to try to have a developer doing this kind of product management. not because a developer can’t have the right skillset to do them. but because having the same person leading development and product management is a mistake. the developer is too close to the development lense, and there’s just a clarification that happens when these roles are separate. my experience also tells me that it’s a mistake to have a committee doing these things, much as that is popular in the academic sector. because, well, just of course it is. but okay this is all still pretty abstract. things might become more clear if we get more specific about the actual tasks and work of this kind of product management role. i found damilola ajiboye blog post on “product manager vs product marketing manager vs product owner” very clear and helpful here. while it is written so as to distinguish between three different product management related roles, but ajiboye also acknowledges that in a smaller organization “a product manager is often tasked with the duty of these roles. regardless of if the responsibilities are to be done by one or two or three person, ajiboye’s post serves as a concise listing of the work to be done in managing a product — deciding the what of the product, in an ongoing iterative and collaborative manner, so that developers and designers can get to the how and to implementation. i recommend reading the whole article, and i’ll excerpt much of it here, slightly rearranged. the product managerthese individuals are often referred to as mini ceos of a product. they conduct customer surveys to figure out the customer’s pain and build solutions to address it. the pm also prioritizes what features are to be built next and prepares and manages a cohesive and digital product roadmap and strategy.the product manager will interface with the users through user interviews/feedback surveys or other means to hear directly from the users. they will come up with hypotheses alongside the team and validate them through prototyping and user testing. they will then create a strategy on the feature and align the team and stakeholders around it. the pm who is also the chief custodian of the entire product roadmap will, therefore, be tasked with the duty of prioritization. before going ahead to carry out research and strategy, they will have to convince the stakeholders if it is a good choice to build the feature in context at that particular time or wait a bit longer based on the content of the roadmap.the product marketing managerthe pmm communicates vital product value — the “why”, “what” and “when” of a product to intending buyers. he manages the go-to-market strategy/roadmap and also oversees the pricing model of the product. the primary goal of a pmm is to create demand for the products through effective messaging and marketing programs so that the product has a shorter sales cycle and higher revenue.the product marketing manager is tasked with market feasibility and discovering if the features being built align with the company’s sales and revenue plan for the period. they also make research on how sought-after the feature is being anticipated and how it will impact the budget. they communicate the values of the feature; the why, what, and when to potential buyers — in this case users in countries with poor internet connection.[while expressed in terms of a for-profit enterprise selling something, i think it’s not hard to translate this to a non-profit or academic environment. you still have an audience whose uptake you need to be succesful, whether internal or external. — jrochkind ]the product ownera product owner (po) maximizes the value of a product through the creation and management of the product backlog, creation of user stories for the development team. the product owner is the customer’s representative to the development team. he addresses customer’s pain points by managing and prioritizing a visible product backlog. the po is the first point of call when the development team needs clarity about interpreting a product feature to be implemented.the product owner will first have to prioritize the backlog to see if there are no important tasks to be executed and if this new feature is worth leaving whatever is being built currently. they will also consider the development effort required to build the feature i.e the time, tools, and skill set that will be required. they will be the one to tell if the expertise of the current developers is enough or if more engineers or designers are needed to be able to deliver at the scheduled time. the product owner is also armed with the task of interpreting the product/feature requirements for the development team. they serve as the interface between the stakeholders and the development team. when you have someone(s) doing these roles well, it ensures that the development team is actually spending time on things that meet user and business needs. i have found that it makes things so much less stressful and more rewarding for everyone involved. when you have nobody doing these roles, or someone doing it in a cursory or un-intentional way not recognized as part of their core job responsibilities, or have a lead developer trying to do it on top of develvopment, i find it leads to feelings of: spinning wheels, everything-is-an-emergency, lack of appreciation, miscommunication and lack of shared understanding between stakeholders and developers, general burnout and dissatisfaction — and at the root, a product that is not meeting user or business needs well, leading to these inter-personal and personal problems. islandora: islandora open meeting: february , islandora open meeting: february , manez wed, / / - : body we will be holding another open drop-in session on tuesday, february from : am to : pm eastern. full details, and the zoom link to join, are in this google doc. the meeting is free form, with experienced islandora users on hand to answer questions or give demos on request. please drop in at any time during the four-hour window. registration is not required. if you would like a calendar invite as a reminder, please let us know at community@islandora.ca. hangingtogether: emerging roles for libraries in bibliometric and research impact analysis: lessons learned from the university of waterloo library support for bibliometrics and research impact (bri) analysis is a growing area of library investment and service. not just in the provision of services to researchers, but for the institutions themselves, which increasingly need to quantify research impact for a spectrum of internally and externally motivated purposes, such as strategic decision support, benchmarking, reputation analysis, support for funding requests, and to better understand research performance. research libraries are adopting new roles to support bibliometrics and research impact analysis, and the university of waterloo library’s efforts have caught my attention for some time, and for two specific reasons: they are leaders in the area of bibliometrics and research impact in north americathey have exemplified exceptional cross-institutional collaboration—what i might call “social interoperability”–in developing services, staffing, and programs. alison hitchens, associate university librarian for collections, technology, and scholarly communication, and laura bredahl, bibliometrics and research impact librarian, recently shared about their activities in an oclc research library partnership (rlp) works in progress webinar presentation entitled case study—supporting bibliometric and research impact analysis at the university of waterloo. their efforts also will be described in a forthcoming arl library practice brief on supporting bibliometric data needs at academic institutions. rlp works in progress webinar: case study—supporting bibliometric and research impact analysis at the university of waterloo the waterloo bri story like many institutions, the library at waterloo has been supporting individual researchers with bibliometrics information for reputation management for over a decade. however, around the university recognized that it needed an institutional understanding of bibliometrics because important external stakeholders like funders, governments, and other accountability organizations were using them to evaluate their organization. additionally, as the campus developed a new strategic plan emphasizing transformational research, it also needed indicators to help chart progress. as a result, the provost established a working group on bibliometrics that included cross-institutional representation from the office of research, office of institutional analysis, library, and faculties, with the goal to provide guidance to the university on the effective and appropriate use of bibliometrics. this working group led to a few significant outcomes: first, it led to the realization and recommendation that a campus expert was needed, which led to the creation of a new bibliometrics and research impact librarian role in . this library professional works at the institutional level, providing expertise, leadership, and support for institutional needs.secondly, it led to the publication in of a white paper on measuring research output through bibliometrics. this type of white paper is critical for any institution utilizing bibliometrics for decision making, as it requires institutional investment to understand and document the opportunities and limitations of bibliometric analysis. it also offers transparent guidance for how researchers and administrators can use bibliometric analysis responsibly. another good example of this type of white paper comes from virginia tech, developed as a companion to that institution’s strategic planning efforts. the white paper was followed by the development of a research metrics framework, intended to provide detailed bibliometric indicators related to the work of research institutes supporting key research areas identified in the - strategic plan. and this in turn was followed in by the development of an internal bibliometric assessment tools report, offering an extremely detailed review of existing bibliometrics assessment tools, known use cases, and an overview of other prominent bibliometrics tools. the working group on bibliometrics continues its work today, supporting the current strategic plan, particularly by advising on the definitions of research areas and the responsible use of indicators at the institutional level. so what does a bibliometrics and research impact librarian do? laura described several examples of her work at waterloo: photo by adam nowakowski on unsplash validating university rankings. a variety of rankings, such as cwts leiden, shanghai’s arwu, macleans (canada), qs, and times higher education, are all closely watched by academic institutions and each has their own ranking methodologies. in general, laura works to understand and verify the data from the rankings, and her efforts also serve to monitor institutional performance, and to provide local level analysis to better understand the rankings.support and training to others. laura is not the only person on campus conducting bibliometric analysis. for example, the faculty of engineering has a dedicated research analyst, and laura provides consultation and expert guidance on the use of tools like scival. laura has also developed an informal community of practice, open to anyone on campus, which is intended to support knowledge sharing and discussion.strategic planning. laura supports efforts to identify appropriate bibliometrics indicators that can be used to understand progress on the current strategic plan. importance to the library libraries seem to me to be a natural place for bibliometrics and research impact leadership. librarians have expertise across the research and scholarly communications life cycle, understand disciplinary differences—and how these impact bibliometrics—and also have extensive knowledge with bibliographic data and tools. in general, this type of engagement can also positively impact the library by “raising the profile of the library on campus.” for example, in the webinar alison commented, “it was clear to me that being connected to and known by high level administration in the office of research really had an impact on building partnerships in other areas such as research data management. it was a lot easier to send an email or pick up the phone and call an avp of research because they knew me through the working group on bibliometrics.” overall, this type of activity may result in greater stakeholder appreciation for the value proposition of the library, an improved understanding of the scope of library expertise, and more invitations for the library to participate on campus committees and enterprise-wide projects. at waterloo, for example, this included opportunities for the aul to join the office of research systems advisory group and for the library to contribute to the institutional rim project. as the new strategic planning effort has launched, and seven working groups were formed to develop background papers, the library was in a position to successfully advocate for a librarian on each committee. learn more! continue the conversation with the oclc research library partnership of course there’s much more to discuss, so we are offering affiliates with the research library partnership an opportunity to continue the conversation through informal small group discussions with alison and laura. please join us on the following dates: europe and east coast-friendly time: wednesday, march at am edt (utc- ). this will be pm in the uk.anz and american west coast-friendly time: wednesday, march at pm edt (utc- )/ pm pst (this should be am in melbourne/sydney on thursday the th) we are interested in exploring some of these questions: does your campus have an internal, cross-unit community of practice around bri? what is the role of your library?what products/tools are being used at your institution?how is your rim/cris system also being incorporated into these efforts? what’s the status of institutional data governance and data sharing at your institution? if you send me an email message, i will send you a calendar invitation (which can help to make sure all the time zones align). the post emerging roles for libraries in bibliometric and research impact analysis: lessons learned from the university of waterloo appeared first on hanging together. jez cope: glam data science network fellow travellers updates - - thanks to gene @dzshuniper@ausglam.space for suggesting adho and a better attribution for the opening quote (see comments below for details) see comments & webmentions for details. “if you want to go fast, go alone. if you want to go far, go together.” — african proverb, probbly popularised in english by kenyan church leader rev. samuel kobia (original) this quote is a popular one in the carpentries community, and i interpret it in this context to mean that a group of people working together is more sustainable than individuals pursuing the same goal independently. that’s something that speaks to me, and that i want to make sure is reflected in nurturing this new community for data science in galleries, archives, libraries & museums (glam). to succeed, this work needs to be complementary and collaborative, rather than competitive, so i want to acknowledge a range of other networks & organisations whose activities complement this. the rest of this article is an unavoidably incomplete list of other relevant organisations whose efforts should be acknowledged and potentially built on. and it should go without saying, but just in case: if the work i’m planning fits right into an existing initiative, then i’m happy to direct my resources there rather than duplicate effort. inspirations & collaborators groups with similar goals or undertaking similar activities, but focused on a different sector, geographic area or topic. i think we should make as much use of and contribution to these existing communities as possible since there will be significant overlap. code lib probably the closest existing community to what i want to build, but primarily based in the us, so timezones (and physical distance for in-person events) make it difficult to participate fully. this is a well-established community though, with regular events including an annual conference so there’s a lot to learn here. newcardigan similar to code lib but an australian focus, so the timezone problem is even bigger! glam labs focused on supporting the people experimenting with and developing the infrastructure to enable scholars to access glam materials in new ways. in some ways, a glam data science network would be complementary to their work, by providing people not directly involved with building glam labs with the skills to make best use of glam labs infrastructure. uk government data science community another existing community with very similar intentions, but focused on uk government sector. clearly the british library and a few national & regional museums & archives fall into this, but much of the rest of the glam sector does not. artifical intelligence for libraries, archives & museums (ai lam) a multinational collaboration between several large libraries, archives and museums with a specific focus on the artificial intelligence (ai) subset of data science uk reproducibility network a network of researchers, primarily in heis, with an interest in improving the transparency and reliability of academic research. mostly science-focused but with some overlap of goals around ethical and robust use of data. museums computer group i’m less familiar with this than the others, but it seems to have a wider focus on technology generally, within the slightly narrower scope of museums specifically. again, a lot of potential for collaboration. training several organisations and looser groups exist specifically to develop and deliver training that will be relevant to members of this network. the network also presents an opportunity for those who have done a workshop with one of these and want to know what the “next steps” are to continue their data science journey. the carpentries, aka: library carpentry data carpentry software carpentry data science training for librarians (dst l) the programming historian cdh cultural heritage data school supporters these misson-driven organisations have goals that align well with what i imagine for the glam dsn, but operate at a more strategic level. they work by providing expert guidance and policy advice, lobbying and supporting specific projects with funding and/or effort. in particular, the ssi runs a fellowship programme which is currently providing a small amount of funding to this project. digital preservation coalition (dpc) software sustainability institute (ssi) research data alliance (rda) alliance of digital humanities organizations (adho) … and its libraries and digital humanities special interest group (lib&dh sig) professional bodies these organisations exist to promote the interests of professionals in particular fields, including supporting professional development. i hope they will provide communication channels to their various members at the least, and may be interested in supporting more directly, depending on their mission and goals. society of research software engineering chartered institute of library and information professionals archives & records association museums association conclusion as i mentioned at the top of the page, this list cannot possibly be complete. this is a growing area and i’m not the only or first person to have this idea. if you can think of anything glaring that i’ve missed and you think should be on this list, leave a comment or tweet/toot at me! digital library federation: metadata during covid this post was written by members of the metadata working group, a subgroup of dlf’s assessment interest group. digital collections work has changed in a number of ways during the covid- pandemic. for many libraries and archives, this has meant working remotely and shifting toward tasks that can be done online. within the dlf aig metadata working group, members have discussed a number of ways that organizations have chosen to increase capacity for metadata, transcription, and other tasks related to digital collections as a way of providing work for employees who would normally work in public-serving positions. this post documents some of those projects and activities. university of north texas at the university of north texas, our digital collections use a web-based metadata editing interface and we can add as many users as needed. when the stay-at-home order went into effect right after spring break, many of our library staff members (including full-time librarians/staff and part-time student workers) were no longer able to do their regular jobs and we offered metadata as an alternative. we added about new editors to our system in march . additionally, we added some quickly-drafted documentation to steer people toward easy metadata projects and known issues that require clean-up (like fixing name formatting). to keep oversight manageable, new editors were still attached to their own departments (or assigned to one that needed help), with a central contact person for each department and a specific sub-set of projects. our team of developers rushed an overhaul of the event tracking system that documents who is editing and what records they are changing so that managers could more easily verify if workers were editing when they said they were working. tracking edits has also let us measure how significantly overall editing has increased. multiple times since this started, we have had at least one editor working during every hour of the day. having so many relatively-untrained editors has resulted in a large number of issues that will need to be reviewed, but we have tools built into our system to help identify those issues and have added them to our ongoing list of things to fix. overall, this was still an extremely positive experience since the increase in editors allowed significant progress or completion of work that would not have been done otherwise. – hannah tarver university of utah marriott library at the university of utah, the covid- pandemic pivot to remote work prompted the launch of transcription projects, both with handwritten materials from special collections and newspaper ocr correction. this includes the transcription of , employee records by our digital operations student employees which resulted in the complete transcription of the kennecott miner records collection. we are also using omeka classic with the scripto plug-in as the platform for manuscript transcription projects and are able to find valuable work for people to engage in when they couldn’t physically be at the library. in addition, we developed a born-digital crowdsourced digital collection, the utah covid- digital collection designed to capture what is currently happening during this unusual time. we’ve gotten a great response from the university and larger utah communities, with over contributions so far available in the digital library. the covid- digital collection has enabled us to build new partnerships and provided the library with outreach opportunities. an article detailing the project is forthcoming in a special issue of the journal digital library perspectives. – anna neatrour utah state archives after starting with from the page a few months earlier, moving staff and volunteers to transcription and indexing projects proved to be successful. contributors finished a historical court case (and now working on a second one) and a year’s worth of birth certificates in only a few months using the web-based interface that integrates with contentdm digital collections. with a built-in notes feature, questions can be asked and answered directly on a document’s page, which will then be exported along with the rest of the metadata. we are now preparing to open up the birth certificate indexing to the general public with additional training materials. in addition, new digital collections have been published, even with metadata developed remotely, using tools like google sheets for input and then converting to delimited text files for import. – gina strack university of texas at austin at the start of march, the university of texas libraries collections portal, the public-facing search and discovery interface for our digital asset management system (dams), included approximately , items. shortly after, the ut-austin campus closed and many staff members from the libraries’ first-line customer service, acquisitions and cataloging units found their roles pivoting to create metadata remotely for our dams system. collection curators within ut libraries created large-scale digital projects to help ensure continued remote work and to utilize this unusual time to turn their focus to projects that had been placed on the back burner due to more pressing obligations. our digital asset management system coordinator and staff from our preservation and digital stewardship unit created flexible pathways to support these projects and to ensure successful ingests into the dams. staff at the architecture & planning library and the alexander architectural archives, the nettie lee benson latin american collection, and the perry-castañeda library map collection dedicated themselves to ingesting and describing large amounts of digital items, increasing our total number of items available online to over , by september. digital objects newly available online as a result of this unprecedented, organization-wide collaborative effort include over , digitized architectural drawings and images, historic books from the benson rare book collection and primeros libros de las américas, and , scanned maps. the university of texas libraries documented the experience and provided a more detailed explanation of our dams in texlibris. – mandy ryan colgate university colgate university’s special collections and university archives (scua) is documenting the colgate community’s experiences and stories of covid- . digital contributions can be submitted at any time via a google form and may be added to colgate’s digital collections portal. there have been direct submissions as of october . physical donations of covid- related materials will be accepted once staff return to the library building. colgate’s metadata and cataloging (m&c) staff have been working with scua’s digital collections at home for the first time, describing the work of the university’s longest-serving official photographer, edward h. stone. stone documented life at colgate from the s to the s, and also photographed the people, places, businesses, and industry of the village of hamilton and madison county, new york. m&c staff are creating and editing metadata for more than glass plate negatives scanned by scua staff and students. we anticipate this will be a successful collaboration between the two departments that will serve as a model for other metadata-based remote work projects on campus. m&c staff have also worked with a born-digital lgbtq oral history project curated by students in the explorations in lgbtq studies class. oral history interviews with colgate graduates active in the struggle for lgbtq rights on campus from the s to the s is now available on the digital collections site – rachel white digital library of georgia most of our staff were able to continue doing most of our work from home, though some imaging projects shifted from actively imaging work (which would have had to be done in the office with our cameras) to working on image editing and curation work. we also had to postpone a meeting for our digitization partners. some metadata projects that were waiting on new imaging work were shifted to complete later; metadata staff worked on metadata remediation and metadata harvesting projects. one colleague who works on newspaper imaging was shifted over to a project describing moving image footage for the parade of quartets collection. we set up a student transcription project to keep students teleworking while they had to remain off-campus due to covid- . their transcription work was incorporated into our full-text accessibility feature for some smaller collections. students are now working in the office and from home on newspaper collation and metadata work, and our imaging staff have worked out a schedule to work while social distancing. our full staff meetings shifted from weekly meetings (in person) to daily meetings (via zoom). unit and supervisor meetings continue with the same frequency as they were held pre-covid. our quarter - newsletter and our quarter newsletter both provide more details of what we have worked on throughout the year. – mandy mastrovita university of florida since the middle of march , the digital support services (dss) at the libraries has shifted the focus of its imaging assistant crew. collaborating with the metadata staff, this crew has carried out site-wide metadata cleanup projects for the university of florida digital collection (ufdc) using ufdc’s online metadata edit form. these tasks can all be done at home using a computer connected to the internet with minimum instructions. the projects include adding missing system id numbers, unifying the spelling of language terms, correcting diacritic displays, updating rights statements, transcribing hand-written content, merging genre terms of different spelling variations to selected ones. so far, dss staff has modified over , rights statements and transcribed over , words. these projects improve the overall metadata quality dramatically. for instance, the genre terms in use will then be cut down to about from the original terms gathered from all data contributors over the years. to maintain this smaller selection of genre terms, the dss will also implement steps to assure all incoming content uses terms from the controlled genre list. – xiaoli ma the ohio state university libraries the onset of the covid- pandemic necessitated a shift to telework for university libraries’ employees. in collaboration with metadata initiatives and preservation & digitization, staff and student employees in other units and needing remote work to do were given the opportunity to do metadata telework. these entailed review and description of content for digital collections, a digital repository for digitized and born-digital special collections and archival materials. catalogers worked on remediation of legacy metadata records, particularly audio and image resources. staff and student employees with no prior metadata experience assisted with review and description of digitized audio and video content in the backlog. this group also contributed to metadata gathering and quality review for a large migration of digitized student newspapers. virtual collaboration was conducted with zoom, e-mail, and the university’s instance of box, a cloud-based content management system. this work has made a significant impact on the backlog for dc. however, metadata initiatives and applicable stakeholders are still reviewing the work that was done before making updates to records and ingesting the newly processed content. – annamarie klose the post metadata during covid appeared first on dlf. lucidworks: accelerate time to value for information retrieval with ai we’ve organized the virtuous cycle of our ai and machine learning discipline to make it clear how customers can make the most of the data science innovation at their disposal. the post accelerate time to value for information retrieval with ai appeared first on lucidworks. oclc dev network: planned maintenance: classify api oclc will be performing quarterly maintenance on the experimental classify api on february from : am – : am eastern us (utc - ). terry reese: marcedit . .x/ . .x (beta) updates versions are available at: https://marcedit.reeset.net/downloads information about the changes: . . change log: https://marcedit.reeset.net/software/update .txt . . change log: https://marcedit.reeset.net/software/update .txt if you are using .x – this will prompt as normal for update. . .x is the beta build, please be aware i expect to be releasing updates to this build weekly and also expect to find some issues. questions, let me know. –tr cynthia ng: choosing not to go into management (again) often, to move up and get a higher pay, you have to become a manager, but not everyone is suited to become a manager, and sometimes given the preference, it’s not what someone wants to do. thankfully at gitlab, in every engineering team including support, we have two tracks: technical (individual contributor), and management. progression … continue reading "choosing not to go into management (again)" harvard library innovation lab: new updates to search: advanced filters the caselaw access project offers free, public access to over . million decisions published by state and federal courts throughout american history. because our mission is providing access to legal information, we make these decisions available in a variety of formats through a variety of different access methods. one type of access we've been working hard on recently is our search interface, which you can get to at case.law/search. we've had basic search working for a while, and we're pleased to share our new advanced search filters. advanced filters work exactly as you'd expect. start your search with keywords or phrases, and then use the filters to narrow down jurisdictions, courts, and dates. say you're looking for massachusetts cases from to that contain the word "whaling." you can also access the advanced filters from the search results screen, so that you can fine-tune your search if you're not happy with the initial results. delete or modify any of the filters as you go, and sort the results chronologically or by relevance. there is a lot more we hope to do with search, but we hope you enjoy this improvement. if you have ideas of your own, please share them with us at info@case.law. cap is a project of the library innovation lab at harvard law school library. we make open source software that helps people access legal information, preserve web sources with perma.cc, and create open educational resources with h o. open knowledge foundation: announcing a new partner for open data day mini-grants for open data day on saturday th march, the open knowledge foundation is offering support and funding for in-person and online events anywhere in the world via our mini-grant scheme. today we are pleased to announce an additional partner for the open data day mini-grant scheme: the global facility for disaster reduction and recovery (gfdrr) through the gfdrr labs and its open data for resilience initiative (opendri). gfdrr will be supporting mini-grants in the environmental data track, with a particular focus on ‘data for resilience’. if you need inspiration for your event using data for resilience, some useful resources to check out include: gfdrr labs, opendri, open cities project, thinkhazard, open data for resilience index and the risk data library. we are extremely grateful to gfdrr and all our partners who have provided funding for this year’s mini-grant scheme. these include microsoft, uk foreign, commonwealth and development office, mapbox, latin american open data initiative (ilda), open contracting partnership and datopian. cynthia ng: reflection part : my first year at gitlab and becoming senior about a year ago, i wrote a reflection on summit and contribute, our all staff events, and later that year, wrote a series of posts on the gitlab values and culture from my own perspective. there is a lot that i mention in the blog post series and i’ll try not to repeat myself (too … continue reading "reflection part : my first year at gitlab and becoming senior" cynthia ng: reflection part : my second year at gitlab and on becoming senior again this reflection is a direct continuation of part of my time at gitlab so far. if you haven’t, please read the first part before beginning this one. becoming an engineer ( months) the more time i spent working in support, the more i realized that the job was much more technical than i originally … continue reading "reflection part : my second year at gitlab and on becoming senior again" jonathan rochkind: rails auto-scaling on heroku we are investigating moving our medium-small-ish rails app to heroku. we looked at both the rails autoscale add-on available on heroku marketplace, and the hirefire.io service which is not listed on heroku marketplace and i almost didn’t realize it existed. i guess hirefire.io doesn’t have any kind of a partnership with heroku, but still uses the heroku api to provide an autoscale service. hirefire.io ended up looking more fully-featured and lesser priced than rails autoscale; so the main service of this post is just trying to increase visibility of hirefire.io and therefore competition in the field, which benefits us consumers. background: interest in auto-scaling rails background jobs at first i didn’t realize there was such a thing as “auto-scaling” on heroku, but once i did, i realized it could indeed save us lots of money. i am more interested in scaling rails background workers than i a web workers though — our background workers are busiest when we are doing “ingests” into our digital collections/digital asset management system, so the work is highly variable. auto-scaling up to more when there is ingest work piling up can give us really nice inget throughput while keeping costs low. on the other hand, our web traffic is fairly low and probably isn’t going to go up by an order of magnitude (non-profit cultural institution here). and after discovering that a “standard” dyno is just too slow, we will likely be running a performance-m or performance-l anyway — which likely can handle all anticipated traffic on it’s own. if we have an auto-scaling solution, we might configure it for web dynos, but we are especially interested in good features for background scaling. there is a heroku built-in autoscale feature, but it only works for performance dynos, and won’t do anything for rails background job dynos, so that was right out. that could work for rails bg jobs, the rails autoscale add-on on the heroku marketplace; and then we found hirefire.io. pricing: pretty different hirefire as of now january , hirefire.io has pretty simple and affordable pricing. $ /month/heroku application. auto-scaling as many dynos and process types as you like. hirefire.io by default can only check into your apps metrics to decide if a scaling event can occur once per minute. if you want more frequent than that (up to once every seconds), you have to pay an additional $ /month, for $ /month/heroku application. even though it is not a heroku add-on, hirefire does advertise that they bill pro-rated to the second, just like heroku and heroku add-ons. rails autoscale rails autoscale has a more tiered approach to pricing that is based on number and type of dynos you are scaling. starting at $ /month for - standard dynos, the next tier up is $ for up to standard dynos, all the way up to $ (!) for to dynos. if you have performance dynos involved, from $ /month for - performance dynos, up to $ /month for up to performance dynos. for our anticipated uses… if we only scale bg dynos, i might want to scale from (low) or to (high) or standard dynos, so we’d be at $ /month. our web dynos are likely to be performance and i wouldn’t want/need to scale more than probably , but that puts us into performance dyno tier, so we’re looking at $ /month. this is of course significantly more expensive than hirefire.io’s flat rate. metric resolution since hirefire had an additional charge for finer than -minute resolution on checks for autoscaling, we’ll discuss resolution here in this section too. rails autoscale has same resolution for all tiers, and i think it’s generally seconds, so approximately the same as hirefire if you pay the extra $ for increased resolution. configuration let’s look at configuration screens to get a sense of feature-sets. rails autoscale web dynos to configure web dynos, here’s what you get, with default values: the metric rails autoscale uses for scaling web dynos is time in heroku routing queue, which seems right to me — when things are spending longer in heroku routing queue before getting to a dyno, it means scale up. worker dynos for scaling worker dynos, rails autoscale can scale dyno type named “worker” — it can understand ruby queuing libraries sidekiq, resque, delayed job, or que. i’m not certain if there are options for writing custom adapter code for other backends. here’s what the configuration options are — sorry these aren’t the defaults, i’ve already customized them and lost track of what defaults are. you can see that worker dynos are scaled based on the metric “number of jobs queued”, and you can tell it to only pay attention to certain queues if you want. hirefire hirefire has far more options for customization than rails autoscale, which can make it a bit overwhelming, but also potentially more powerful. web dynos you can actually configure as many heroku process types as you have for autoscale, not just ones named “web” and “worker”. and for each, you have your choice of several metrics to be used as scaling triggers. for web, i think queue time (percentile, average) matches what rails autoscale does, configured to percentile, , and is probably the best to use unless you have a reason to use another. (“rails autoscale tracks the th percentile queue time, which for most applications will hover well below the default threshold of ms.“)here’s what configuration hirefire makes available if you are scaling on “queue time” like rails autoscale, configuration may vary for other metrics. i think if you fill in the right numbers, you can configure to work equivalently to rails autoscale. worker dynos if you have more than one heroku process type for workers — say, working on different queues — hirefire can scale the independently, with entirely separate configuration. this is pretty handy, and i don’t think rails autoscale offers this. (update i may be wrong, rails autoscale says they do support this, so check on it yourself if it matters to you). for worker dynos, you could choose to scale based on actual “dyno load”, but i think this is probably mostly for types of processes where there isn’t the ability to look at “number of jobs”. a “number of jobs in queue” like rails autoscale does makes a lot more sense to me as an effective metric for scaling queue-based bg workers. hirefire’s metric is slightly difererent than rails autoscale’s “jobs in queue”. for recognized ruby queue systems (a larger list than rails autoscale’s; and you can write your own custom adapter for whatever you like), it actually measures jobs in queue plus workers currently busy. so queued+in-progress, rather than rails autoscale’s just queued. i actually have a bit of trouble wrapping my head around the implications of this, but basically, it means that hirefire’s “jobs in queue” metric strategy is intended to try to scale all the way to emptying your queue, or reaching your max scale limit, whichever comes first. i think this may make sense and work out at least as well or perhaps better than rails autoscale’s approach? here’s what configuration hirefire makes available for worker dynos scaling on “job queue” metric. since the metric isn’t the same as rails autosale, we can’t configure this to work identically. but there are a whole bunch of configuration options, some similar to rails autoscale’s. the most important thing here is that “ratio” configuration. it may not be obvious, but with the way the hirefire metric works, you are basically meant to configure this to equal the number of workers/threads you have on each dyno. i have it configured to because my heroku worker processes use resque, with resque_pool, configured to run resque workers on each dyno. if you use sidekiq, set ratio to your configured concurrency — or if you are running more than one sidekiq process, processes*concurrency. basically how many jobs your dyno can be concurrently working is what you should normally set for ‘ratio’. hirefire not a heroku plugin hirefire isn’t actually a heroku plugin. in addition to that meaning separate invoicing, there can be some other inconveniences. since hirefire only can interact with heroku api, for some metrics (including the “queue time” metric that is probably optimal for web dyno scaling) you have to configure your app to log regular statistics to heroku’s “logplex” system. this can add a lot of noise to your log, and for heroku logging add-ons that are tired based on number of log lines or bytes, can push you up to higher pricing tiers. if you use paperclip, i think you should be able to use the log filtering feature to solve this, keep that noise out of your logs and avoid impacting data log transfer limits. however, if you ever have cause to look at heroku’s raw logs, that noise will still be there. support and docs i asked a couple questions of both hirefire and rails autoscale as part of my evaluation, and got back well-informed and easy-to-understand answers quickly from both. support for both seems to be great. i would say the documentation is decent-but-not-exhaustive for both products. hirefire may have slightly more complete documentation. other features? there are other things you might want to compare, various kinds of observability (bar chart or graph of dynos or observed metrics) and notification. i don’t have time to get into the details (and didn’t actually spend much time exploring them to evaluate), but they seem to offer roughly similar features. conclusion rails autoscale is quite a bit more expensive than hirefire.io’s flat rate, once you get past rails autoscale’s most basic tier (scaling no more than standard dynos). it’s true that autoscaling saves you money over not, so even an expensive price could be considered a ‘cut’ of that, and possibly for many ecommerce sites even $ a month might a drop in the bucket (!)…. but this price difference is so significant with hirefire (which has flat rate regardless of dynos), that it seems to me it would take a lot of additional features/value to justify. and it’s not clear that rails autoscale has any feature advantage. in general, hirefire.io seems to have more features and flexibility. until , hirefire.io could only analyze metrics with -minute resolution, so perhaps that was a “killer feature”? honestly i wonder if this price difference is sustained by rails autoscale only because most customers aren’t aware of hirefire.io, it not being listed on the heroku marketplace? single-invoice billing is handy, but probably not worth $ + a month. i guess hirefire’s logplex noise is a bit inconvenient? or is there something else i’m missing? pricing competition is good for the consumer. and are there any other heroku autoscale solutions, that can handle rails bg job dynos, that i still don’t know about? update a day after writing djcp on a reddit thread writes: i used to be a principal engineer for the heroku add-ons program.one issue with hirefire is they request account level oauth tokens that essentially give them ability to do anything with your apps, where rails autoscaling worked with us to create a partnership and integrate with our “official” add-on apis that limits security concerns and are scoped to the application that’s being scaled.part of the reason for hirefire working the way it does is historical, but we’ve supported the endpoints they need to scale for “official” partners for years now.a lot of heroku customers use hirefire so please don’t think i’m spreading fud, but you should be aware you’re giving a third party very broad rights to do things to your apps. they probably won’t, of course, but what if there’s a compromise?“official” add-on providers are given limited scoped tokens to (mostly) only the actions / endpoints they need, minimizing blast radius if they do get compromised. you can read some more discussion at that thread. hugh rundle: automation workflows with github actions and webhooks - library map part this is the third in my series on the library map. part one dealt with why i made the map. part explained how i made it. this post is about strategies i've used to automate some things to keep it up to date. what is a github action? a github action is an automated script that runs on a virtual machine when triggered by some kind of event. triggers for actions are defined in a "workflow" configuration file at .github/workflows in your github repository. the terminology can be a bit confusing, because "github actions" is what github calls the whole system, but an "action" within that system is actually the smallest part in the series: workflow- job - step - action - action - step - action - job - step - action - action github actions are really just github's version of a continuous integration / continuous deployment (ci/cd) tool. i say "just", but it's extremely powerful. unfortunately that does mean that even though github actions are quite extensively documented, the docs aren't necessarily all that clear if you're starting from scratch, and the process is quite confusing for the uninitiated. i spent a couple of days failing to make it work the way i wanted, so that you don't have to. github actions ...in action there are a zillion things you can use github actions for — auto-closing "stale" issues, adding labels automatically, running code linters on pull requests, and so on. if you've read my previous posts, you might remember that i wrote a little python script to merge the data from library_services_information.csv into boundaries.topo.json. but doing that manually every time the csv file is updated is a tedious manual task. wouldn't it be better if we could automate it? well, we can automate it with github actions! what we want to do here is set up a trigger that runs the script whenever the csv file is changed. i originally tried doing this on a push event (every time code is pushed to the default branch), and it worked, but ultimately i decided it would be better to run it whenever someone (including me) makes a pull request. i'm in a reasonably consistent habit of always creating a new git branch rather than committing directly to the default branch, and there's less chance of something going wrong and the topojson file being corrupted if the merge is done at the pull request stage and then manually pulled in — if there can't be a clean merge, github will tell me before i break everything. to set this up, we need to write a workflow configuration file, listing the jobs we want done, and the actions within each job. jobs within each workflow are run concurrently unless the workflow configuration tells them to wait for the previous job, though in our case that doesn't matter, because there is only a single job. the structure is: workflow ('topo auto updater (pr)') - job ('auto-topo-updater') - step : git checkout code - step : add labels - step : merge files - step : git commit updated code the first step uses an action provided by github itself. it runs a git checkout on the repository before anything else happens. this means nothing will happen in the actual repository if anything in the workflow fails, because the virtual machine that checked out your code just gets destroyed without checking the code back in. step will use an action created by christian vuerings, and automatically adds labels to an issue or pull request, based on whatever criteria triggered the workflow. step runs the python script to merge the csv data into the topojson. step (care of stefan zweifel) commits and pushes the updated changes into the pull request that triggered the workflow. this is where the real magic happens, because it simply adds a second commit to the pull request as soon as it is received and before the pr is merged. i initially had set this up to create a second pull request with just the merged topojson changes and then tried to work out how to auto-merge that new pull request, but someone on mastodon helpfully asked me why i would bother creating a pull request if i wanted to auto-merge it anyway. the thought of auto-committing terrified me initially because i had no idea what i was doing, but on reflection a second pr was indeed a bit silly. writing the config file to get all this to happen, we need to write a configuration file. this is written in yaml, and saved in a special directory at the top of the repository, called .github/workflows. you can name this file whatever you want, but it has to end in .yml. first we provide some kind of trigger, and include any conditions we might want to apply. i want this workflow to happen whenever someone creates a pull request that includes changes to the website/data/library_services_information.csv file: name: topo auto updater (pr)on: pull_request: paths: - 'website/data/library_services_information.csv' workflow_dispatch: the on directive lists the different 'events' that can trigger the workflow. the first one is clear enough, but what about workflow_dispatch? this event simply means "when triggered manually by pressing a button". i don't know why it has such an obscure name. once we've told github when we want the workflow to run, we can tell it what we want it to do. first we list our jobs: jobs: auto-topo-updater: runs-on: ubuntu-latest steps: # steps go here the first line under 'jobs' is the name of our job (this can be anything, but without spaces). runs on tells github which runner to use. a 'runner' is a special environment that runs automated continuous integration tools. in this case we're using github actions runners, but runners are also commonly used in other automated testing tools. here we are using the "latest" ubuntu linux runner, which is currently using ubuntu . even though ubuntu . is actually the latest ubuntu lts release. now we've outlined the trigger and where we want to run our steps, it's time to say what those steps are: steps:- uses: actions/checkout@v with: ref: ${{ github.head_ref }}- uses: christianvuerings/add-labels@v with: labels: | auto update data env: github_token: $- name: merge csv to topojson run: | python ./.github/scripts/merge_csv_to_topojson.py- uses: stefanzweifel/git-auto-commit-action@v with: commit_message: merge csv data to topo whoah, that's a lot! you can see there are two ways we describe how to perform an action: uses, or name + run. the uses directive points to an action that someone has publicly shared on github. so uses: actions/checkout@v means "use version of the action at the repository address https://github.com/actions/checkout". this is an official github action. if we want to simply run some commands, we can just give our action a name and use the run directive: - name: merge csv to topojson run: | python ./.github/scripts/merge_csv_to_topojson.py in this example, we use a pipe (|) to indicate that the next lines should be read one after another in the default shell (basically, a tiny shell script). the first step checked out out our code, so we can now use any script that is in the repository. i moved the python merging script into .github/scripts/ to make it clearer how this script is used, and now we're calling it with the python command. to pass data to an action, we use with. the step below passes a list of label names to add to the pull request ('auto update' and 'data'): - uses: christianvuerings/add-labels@v with: labels: | auto update data finally, for the labels step we need to provide an environment variable. for certain activities, github requires actions to use a github_token so that you can't just run an action against any repository without permission. this is automatically stored in the "secret store", to which you can also add other secrets like api keys and so on. the env directive passes this through to the action: env:github_token: ${{ secrets.github_token }} putting the robots to work now when a pull request is sent, it gets tagged auto update and data, and a commit updating the topo.json file is automatically added to it: you can see the full config file in the library map repository. i've also worked out how to reduce the filesize of my geojson file, so i was able to check it in to the repository. this allowed me to automate the transformation from geojson to topojson whenever the geojson file is updated, with a workflow that runs some commands over the geojson and creates a new pull request. one little gotcha with this is that the action i used to process the geojson file into topojson also cleans up the geojson, which means triggering the action on any change to the geojson file creates a recursive loop whereby every time the new pull request is merged, it creates a new one. to get around this, i probably should just make it auto-commit rather than create a pull request, but for now i added an if statement: jobs: processjson: if: "!contains(github.event.head_commit.message, 'from hughrun/geo-to-topo')"... - name: create pull request uses: peter-evans/create-pull-request@v with: commit-message: update topojson boundaries title: update topojson boundaries body: 'clean & minify geojson' branch: geo-to-topo labels: auto update,data the last action creates a pull request on a new geo-to-topo branch, so if the commit message includes "from hughrun/geo-to-topo" the job won't run. recursive pull request problem solved! what is a webhook? i really like cherries, but they're not always season. imagine me sending a text message to the local greengrocer every day in early summer, to ask whether they have any cherries yet. they text me back: usually the answer is "no", but eventually it's a "yes". then i hit on an idea: i call them and ask them to just text me when cherries are in stock. the first approach is how an api call works: you send a request, and the server sends a response. the second is how a webhook works — you get the response without having to even send the request, when a certain criteria is met. i've been playing around with apis and webhooks at work, because we want to connect eventbrite event information to a calendar on our own website. but github also offers webhooks (which actually pre-dates github actions), and this is the final piece of the library map automation pipeline. the big difference of course is that sending an http request and receiving an http request are quite different things. you can send an http request in many different ways: including by just typing a url into a browser. but to receive a request you need some kind of server. especially if you don't know when it will be sent. conveniently i already have a vps that i use for a few things, including hosting this blog. so we have something to receive the webhook (a server), and something to send the webhook (github). now we need to tell those two things how to talk to each other. what we want to do here is automatically update the data on the library map whenever there is an update in the repository. i could make this easier by just publishing the map with github pages, but i don't want to completely rely on github for everything. sending the webhook first of all we need to set up the webhook. in the repository we go to settings - webhooks and then click on add webhook. here we enter the payload url (the url we will set up on our server, to receive the webhook: https://example.com/gh-library-map), the content type (application/json), and a secret. the secret is just a password that can be any text string, but i recommend using something long and hard to guess. you could try one of my favourite urls to create it. we want the trigger to be "just the push event" because we don't want to trigger the webhook every time anything at all happens in the repository. unfortunately there doesn't seem to be a way to trigger it only on a push to the primary branch, but in future we could probably put some logic in at the receiving end to filter for that. make sure the webhook is set to "active", and click "add webhook". receiving the webhook so setting up the webhook to be sent is reasonably straightforward. receiving it is a bit more complicated. we need to set up a little application to hang around waiting to receive http requests. first of all, we set up nginx to serve our domain — in this post i'll refer to that as 'example.com'. then we secure it using certbot so github can send the webhook to https://example.com. because we might want to use other webhooks on other systems for different tasks, we're going to go with a slightly over-powered option and use express. this gives us a bit of control over routing different requests to different functions. express is a nodejs framework for building web apps, so first we need to make sure we have a recent version of nodejs installed. then we create a new package metadata file, and a javascript file: npm inittouch webhooks.js in our empty webhooks.js file we set up some basic routing rules with express: npm install express --save // webhook.jsconst express = require('express')const port = const app = express()app.use(express.json())app.post('/gh-library-map', (req, res, next) => { // do stuff})// everything else should app.use(function (req, res) { res.status( ).send("there's nothing here")})app.listen(port, () => { console.log(`webhooks app listening on port ${port}`)}) this will do something when a post request is received at https://example.com/gh-library-map. all other requests will receive a response. you can test that now. returning to the delicious cherries: what happens if someone else finds out about my arrangement with the greengrocer? maybe a nefarious strawberry farmer wants to entice me to go to the greengrocer and, upon discovering there are no cherries, buy strawberries instead. they could just send a text message to me saying "hey it's your friendly greengrocer, i totes have cherries in stock". this is the problem with our webhook endpoint as currently set up. anyone could send a post request to https://example.com/gh-library-map and trigger an action. luckily github has thought of that, and has a solution. remember the "secret" we set when we set up the webhook? this is where we use it. but not directly. github instead creates a sha hash of the entire payload using your secret and includes the resulting hash in the payload itself. the hash is sent in a header called x-hub-signature- . we know what our secret is, and we can therefore check the hash by running the same process over the payload at the receiving end as github did at the sending end. as long as we use a strong secret and the hashes match, we can be confident the request did indeed come from github, and not a nefarious strawberry farmer. the crypto library is included in nodejs automatically, so we can use that check: // webhook.jsconst crypto = require('crypto')app.post('/gh-library-map', (req, res, next) => { const hmac = crypto.createhmac('sha ', process.env.library_map_gh_secret) hmac.update(json.stringify(req.body)) // check has signature header and the decrypted signature matches if (req.get('x-hub-signature- ')) { if ( `sha =${hmac.digest('hex').tostring()}` === req.get('x-hub-signature- ') ){ // do something } else { console.error("signature header received but hash did not match") res.status( ).send('signature is missing or does not match') } } else { console.error('signature missing') res.status( ).send('signature is missing or does not match') }}) now we just need to "do something" when the hash matches 😆. push and pull so what is the something we're going to do? the library map server simply contains a copy of the repository, sitting behind an nginx web proxy server. what we need to do to update it is run git pull inside that directory, and it will pull in the latest updates from the repository. our webhook will end up calling this action more often than is strictly useful, because a "push" action happens every time someone creates a pull request, for example, but it's pretty harmless to git pull more often than necessary. first we create a new function: // webhook.jsconst util = require('util')const exec = util.promisify(require('child_process').exec) // run child_process.exec as a promise/asyncasync function gitpull(local_repo, res) { try { const { stdout, stderr } = await exec(`cd ${local_repo} && git pull`); let msg = stderr ? stderr : stdout // message is the error message if there is one, else the stdout // do something with message res.status( ).send('ok') } catch (err) { console.error(err) res.status( ).send('server error sorry about that') }} this function is async because we need to await the git pull before we can do something with the output. to make it "awaitable" we use util.promisify() which is another built-in function in nodejs. we call this function back in our express route, where we said we would "do something": // webhook.jsconst local_repo = "/path/to/website/directory"if (req.get('x-hub-signature- ')) { if ( `sha =${hmac.digest('hex').tostring()}` === req.get('x-hub-signature- ') ){ gitpull(local_repo, res) } else { ... }...} sweet! now every time someone does a git push we can do a git pull to add the change to the website! maybe we want to be sure that happened though, so we can add a final piece to this, by sending ourselves an email using emailjs every time the webhook is successfully received: npm install emailjs // webhook.jsconst { smtpclient } = require('emailjs')function sendemail(msg, trigger) { const client = new smtpclient({ user: process.env.email_user, password: process.env.email_password, host: process.env.smtp_domain, ssl: true, }); // send the message and get a callback with an error or details of the message that was sent client.send( { text: `github webhook for ${trigger} has triggered a "git pull" event with the following result:\n\n${msg}`, from: `webhook alerts<${process.env.email_send_address}>`, to: process.env.email_receive_address, subject: `github triggered a pull for ${trigger}`, }, (err, message) => { console.log(err || message); } );}async function gitpull(local_repo, res) { try { const { stdout, stderr } = await exec(`cd ${local_repo} && git pull`); let msg = stderr ? stderr : stdout sendemail(msg, 'mysite.com') res.status( ).send('ok') } catch (err) { console.error(err) res.status( ).send('server error sorry about that') }} we can now test the webhook: node webhooks.js express will start up. we can use curl to send some test payloads from a new console session on our local machine: curl -d '{"key ":"value ", "key ":"value "}' -h "content-type: application/json" -x post https://example.com/gh-library-mapcurl -h "x-hub-signature- : blah" -d '{"key ":"value ", "key ":"value "}' -h "content-type: application/json" -x post https://example.com/gh-library-map both requests should return a with signature is missing or does not match, but in the server console the second one should log a message signature header received but hash did not match. the last thing we need to do is set up our little express app to run automatically as a background process on the server. we can do this using systemd. i personally find the official documentation rather impenetrable, but there are lots of helpful tutorials online. systemd helps us with two tasks: keeping the app running making the environment variables available to the app first we create a "unit file" called webhooks.service at /etc/systemd/system: # /etc/systemd/system/webhooks.servicedescription=keeps the webhooks express server runningafter=network.target[service]type=simpleexecstart=/usr/bin/node webhooks.jsrestart=alwaysrestartsec= user=usernameworkingdirectory=/home/username/webhooksenvironmentfile=/etc/systemd/system/webhooks.env[install]wantedby=multi-user.target the user is your username, and workingdirectory is wherever you installed your express app. since we're responsible server administrators, we have unattended-upgrades running, so occasionally the server will reboot itself to finish installing security updates. we can ensure the webhooks service always comes back up by setting restart to always. next we create the environmentfile mentioned in the unit file: # /etc/systemd/system/webhooks.envlibrary_map_gh_secret="your github secret here"email_user="user@mail.example.com"email_password="top secret password"smtp_domain="smtp.example.com"email_send_address="webhooks@mail.example.com" this is where all those process.env values come from in the webhooks.js file. we could hardcode them, but you might want to share your file in a blog post one day, and you definitely don't want to accidentally leave your hardcoded github secret in the example! make sure we've stopped the app, so we don't have two conflicting installations, then run: sudo systemctl enable webhooks.servicesudo systemctl start webhooks.service our webhooks service should now be running. go back to the github webhooks page in your repository settings and you should see an option to send a "ping event". this simply checks that your webhook is working by sending a test payload. send the ping, wait a few moments, and we should see an email appear in the email_send_address inbox: what's next? that was a pretty long and technical post, sorry not sorry. now that i've set up all that automation, it would be great for library people to help correct and complete the data. as for me, i'll be looking for other things i can do with automation. maybe automatically tooting release notes for ephemetoot. we'll see. mita williams: weeknote ( ) i don’t have much that i can report in this week’s note. you are just going to have to take my word that this week, a large amount of my time was spent at meetings pertaining to my library department, my union, and anti-black racism work. § last year, around this same time, some colleagues from the university and i organized an speaking event called safer communities in a ‘smart tech’ world: we need to talk about amazon ring in windsor.windsor’s mayor proposes we be the first city in canada to buy into the ring network.as residents of windsor, we have concerns with this potential project. seeing no venue for residents of windsor to share their fears of surveillance and loss of privacy through this private-partnership, we hosted an evening of talks on january nd, at the performance hall at the university of windsor’s school of creative arts windsor armories building. our keynote speaker was chris gilliard, heard recently on cbc’s spark.since that evening, we have been in the media raising our concerns, asking questions, and encouraging others to do the same. the city of windsor has yet to have entered an agreement with amazon ring. this is good news. this week, the city of windsor announced that it has entered a one-year deal partnership with ford mobility canada to share data and insights via ford’s safety insights platform. i don’t think this is good news for reasons outlined in this post called safety insights, data privacy, and spatial justice. § this week i learned a neat tweetdeck hack. if set up a search as column, you can limit the results for that term using the number of ‘engagements’: § § i haven’t read this but i have it bookmarked for potential future reference: the weaponization of web archives: data craft and covid- publics: an unprecedented volume of harmful health misinformation linked to the coronavirus pandemic has led to the appearance of misinformation tactics that leverage web archives in order to evade content moderation on social media platforms. here we present newly identified manipulation techniques designed to maximize the value, longevity, and spread of harmful and non-factual content across social media using provenance information from web archives and social media analytics. after identifying conspiracy content that has been archived by human actors with the wayback machine, we report on user patterns of “screensampling,” where images of archived misinformation are spread via social platforms. we argue that archived web resources from the internet archive’s wayback machine and subsequent screenshots contribute to the covid- “misinfodemic” in platforms. understanding these manipulation tactics that use sources from web archives reveals something vexing about information practices during pandemics—the desire to access reliable information even after it has been moderated and fact-checked, for some individuals, will give health misinformation and conspiracy theories more traction because it has been labeled as specious content by platforms. § i’m going to leave this tweet here because i might pick up this thread in the future: this reminds me of a talk given in by data & society founder and president, danah boyd called you think you want media literacy… do you? this essay still haunts me, largely because we still don’t have good answers for the questions that dr. boyd asks of us and the stakes have only gotten higher. david rosenthal: effort balancing and rate limits catalin cimpanu reports on yet another crime wave using bitcoin in as bitcoin price surges, ddos extortion gangs return in force:in a security alert sent to its customers and shared with zdnet this week, radware said that during the last week of and the first week of , its customers received a new wave of ddos extortion emails.extortionists threatened companies with crippling ddos attacks unless they got paid between and bitcoins ($ , to $ , )...the security firm believes that the rise in the bitcoin-to-usd price has led to some groups returning to or re-prioritizing ddos extortion schemes. and dan goodin reports on the latest technique the ddos-ers are using in ddosers are abusing microsoft rdp to make attacks more powerful:as is typical with many authenticated systems, rdp responds to login requests with a much longer sequence of bits that establish a connection between the two parties. so-called booter/stresser services, which for a fee will bombard internet addresses with enough data to take them offline, have recently embraced rdp as a means to amplify their attacks, security firm netscout said.the amplification allows attackers with only modest resources to strengthen the size of the data they direct at targets. the technique works by bouncing a relatively small amount of data at the amplifying service, which in turn reflects a much larger amount of data at the final target. with an amplification factor of . to , gigabytes-per-second of requests directed at an rdp server will deliver roughly gbps to the target. i don't know why it took me so long to figure it out, but reading goodin's post i suddenly realized that techniques we described in impeding attrition attacks in p p systems, a follow-up to our award-winning sosp paper on the architecture of the lockss system, can be applied to preventing systems from being abused by ddos-ers. below the fold, brief details.among the lockss system's defenses against abuse are two relevant to ddos prevention, rate limits and effort balancing.rate limits i've written before about the importance of rate limits, quoting paul vixie:every reflection-friendly protocol mentioned in this article is going to have to learn rate limiting. this includes the initial tcp three-way handshake, icmp, and every udp-based protocol. in rare instances it's possible to limit one's participation in ddos reflection and/or amplification with a firewall, but most firewalls are either stateless themselves, or their statefulness is so weak that it can be attacked separately. the more common case will be like dns [response rate limiting], where deep knowledge of the protocol is necessary for a correctly engineered rate-limiting solution applicable to the protocol. the rdp server being used to ddos sees a flood of authentication requests whose source address has been spoofed to be the target of the ddos. this isn't what they'd see from a real user, so the rdp server should rate-limit sending authentication responses to a client to a reasonable rate for a real client. this would be helpful, but it isn't enough. because the ddos-ers use a large number of systems to mount an attack, even a fairly low rate of reponses can be harmful. effort balancing in our paper we wrote: effort balancing. if the effort needed by a requester to procure a service from a supplier is less than the effort needed by the supplier to furnish the requested service, then the system can be vulnerable to an attrition attack that consists simply of large numbers of ostensibly valid service requests. we can use provable effort mechanisms such as memory-bound functions to inflate the cost of relatively “cheap” protocol operations by an adjustable amount of provably performed but otherwise useless effort. by requiring that at each stage of a multi-step protocol exchange the requester has invested more effort in the exchange than the supplier, we raise the cost of an attrition strategy that defects part-way through the exchange. this effort balancing is applicable not only to consumed resources such as computations performed, memory bandwidth used or storage occupied, but also to resource commitments. for example, if an adversary peer issues a cheap request for service and then defects, he can cause the supplier to commit resources that are not actually used and are only released after a timeout (e.g., syn floods). the size of the provable effort required in a resource reservation request should reflect the amount of effort that could be performed by the supplier with the resources reserved for the request. vixie also noted the economic requirement:engineering economics requires that the cost in cpu, memory bandwidth, and memory storage of any new state added for rate limiting be insignificant compared with an attacker's effort. the reason rdp can be used to amplify a ddos attack is that, as goodin wrote: rdp responds to login requests with a much longer sequence of bits that establish a connection between the two parties. the obvious application of effort balancing would be to require that rdp's login requests be padded with additional bytes to make them longer than the login reponse. thus the rdp server would act to attenuate the attack, not amplify it. this would satisfy vixie's goal:attenuation also has to be a first-order goal—we must make it more attractive for attackers to send their packets directly to their victims than to bounce them off a ddos attenuator. the protocol could specify that the padding bytes not be random, but be computed from the login request parameters by some algorithm making them relatively expensive to generate but cheap to verify (cf. proof-of-work). this would not significantly impact legitimate clients, who issue login requests infrequently, but would increase the cost of using the rdp server to disguise the source of the attack. open knowledge foundation: launching the net zero challenge: a global pitch competition about using open data for climate action open knowledge foundation is excited to launch the net zero challenge, a global pitch competition about using open data for climate action. with a new administration in the usa and the cop meeting in the uk, will be a crucial year for the global climate response. let’s see how open data can play its part. tell us how your idea or project uses open data for climate action – and you could win a $ , usd in the first round of the net zero challenge. full details about the net zero challenge are available at netzerochallenge.info. this project is funded by our partners at microsoft and the uk foreign, commonwealth & development office. we are extremely grateful for their support. how are you advancing climate action using open data? to be eligible for the net zero challenge, your idea or project must do one or more of the following: understand climate risks track climate progress enable informed climate action, or evaluate climate impact. some ways in which you might do this include: making climate relevant data easier to discover, view and understand by the general data user creating a useful passthrough tool or api for climate-relevant data in any country or jurisdiction organising climate data so that potential data users (including those who are less data-literate) can see what’s available, and make use of it we are very open minded about your approach and methodology. what we care about is the outcome, and whether you answer the question. you might consider whether your idea or project is: technically achievable easy to use easily integrated or can be provided as a tool scalable good value for money published under an open licence which allows free use by others explainable (this is the key test of the challenge. can you pitch your project in three minutes to a general audience?) how do i apply? apply now by filling out this form. all applications must be received by pm pacific standard time on friday th march . late submissions will not be accepted. applications will be reviewed and a short list invited to pitch their idea to a panel of experts at a virtual pitch contest. pitches will take the form of a public three-minute presentation via video conference, followed by a question and answer session with our panel of climate data experts. pitches can be live, or prerecorded but the q&a will be live. expert guidance for the net zero challenge is provided by our advisory committee: the open data charter, the innovation and open data team at transport for new south wales and the open data day team at open knowledge foundation. need more information? if you have any questions about the net zero challenge, please check out the faqs on the netzerochallenge.info website. to contact the net zero challenge team directly, email netzero@okfn.org. peter sefton: research data management looking outward from it this is a presentation that i gave on wednesday the nd of december at the aero (australian eresearch organizations) council meeting at the request of the chair dr carina kemp). carina asked: it would be really interesting to find out what is happening in the research data management space. and i’m not sure if it is too early, but maybe touch on what is happening in the eosc science mesh project. the audience of the aero council is aero member reps from aaf, aarnet, qcif, caudit, csiro, ga, tpac, the uni of auckland, reannz, adsei, curtin, unsw, apo. at this stage i was still the eresearch support manager at uts - but i only had a couple of weeks left in that role. in this presentation i’m going to start from a naive it perspective about research data. i would like to acknowledge and pay my respects to the gundungurra and darug people who are the traditional custodians of the land on which i live and work. research data is special - like snowflakes - and i don’t mean that in a mean way, research data could be anything - any shape any size and researchers are also special, not always % aligned with institutional priorities, they align with their disciplines and departments and research teams. it’s obvious that buying storage doesn’t mean you’re doing data management well but that doesn’t mean it’s not worth restating. so "data storage is not data management". in fact, the opposite might be true - think about buying a laptop - do you just get one that fits all your stuff and rely on getting a bigger one every few years? or do you get a smaller main drive and learn how to make sure that your data's actually archived somewhere? that would be managing data. and remember that not all research data is the same “shape” as corporate data - it does not all come in database or tabular form - it can be images, video, text, with all kinds of structures. there are several reasons we don’t want to just dole-out storage as needed. it’s going to cost a lot of money and keep costing a lot of money not everything is stored in “central storage” anyway. there are share-sync services like aarnet’s cloudstor. just keeping things doesn’t mean we can find them again so far we’ve just looked at things from an infrastructure perspective but that’s not actually why we’re here, us with jobs in eresearch. i think we’re here to help researchers do excellent research with integrity, and we need to help our institutions and researchers manage risk. the australian code for the responsible conduct of research which all research organizations need to adhere to if we get arc or nhmrc grants sets out some institutional responsibilities to provide infrastructure and training there are risks associated with research data, reputational, financial and risks to individuals and communities about whom we hold data at uts, we’ve embraced the research data management plan - as a way to assist in dealing with this risk. rdmps have a mixed reputation here in australia - some organizations have decided to keep them minimal and as streamlined as possible but at uts the thinking is that they can be useful in addressing a lot of the issues raised so far. where’s the data for project x - when there’s an integrity investigation. were procedures followed? how much storage are we going to need? inspired by the (defunct?) research data lifecycle project that was conceived by the former organizations that became the australian research data commons (ands, nectar and rdsi) we came up with this architecture for a central research data management system (in our case we use the open source redbox system) loosely linked to a variety of research workspaces, as we call them. the plan is that over time, researchers can plan and budget for data management in the short, medium and long term, provision services and use the system to archive data as they go. (diagram by gerard barthelot at uts) uts has been an early adopter of the ocfl (oxford common file layout) specificiation - a way of storing file sustainably on a file system (coming soon: s cloud storage) so it does not need to be migrated. i presented on this at the open repositories conference and at the same conference, i introduced the ro-crate standards effort, which is a marriage between the datacrate data packaging work we’ve been doing at uts for a few years, and the research object project. we created the arkisto platform to bring together all the work we’ve been doing to standardise research data metadata, and to build a toolkit for sustainable data repositories at all scales from single-collection up to institutional, and potentially discipline and national collections. this is an example of one of many arkisto deployment patterns you can read more on the arkisto use cases page this is an example of an arkisto-platform output. data exported from one content management system into an archive-ready ro-crate package, which can then be made into a live site. this was created for ass prof tamson pietsch at uts. the website is ephemeral - the data will be interoperable and reusable (i and r from fair) via the use of ro-crate. now to higher-level concerns: i built this infrastructure for my chooks (chickens) - they have a nice dry box with a roosting loft. but most of the time they roost on the roof. we know all too well that researchers don’t always use the infrastructure we build for them - you have to get a few other things right as well. one of the big frustrations i have had as an eresearch manager is that the expectations and aspirations of funders and integrity managers and so on are well ahead of our capacity to deliver the services they want, and then when we do get infrastructure sorted there are organizational challenges to getting people to use it. to go back to my metaphor, we can’t just pick up the researchers from the roof and put them in their loft, or spray water on them to get them to move. via gavin kennedy and guido aben from aarnet marco la rosa and i are helping out with this charmingly named project which is adding data management service to storage, syncronization and sharing services. contracts not yet in place so won't say much about this yet. https://www.cs mesh eosc.eu/about eosc is the european open science cloud cs mesh eosc - interactive and agile sharing mesh of storage, data and applications for eosc - aims to create an interoperable federation of data and higher-level services to enable friction-free collaboration between european researchers. cs mesh eosc will connect locally and individually provided services, and scale them up at the european level and beyond, with the promise of reaching critical mass and brand recognition among european scientists that are not usually engaged with specialist einfrastructures. i told carina i would look outwards as well. what are we keeping an eye on? watch out for the book factory. sorry, the publishing industry. the publishing industry is going to “help” the sector look after it’s research data. like, you, know, they did with the copyright in publications. not only did that industry work out how to take over copyright in research works, they successfully moved from selling us hard-copy resources that we could keep in our own libraries to charging an annual rent on the literature - getting to the point where they can argue that they are essential to maintaining the scholarly record and must be involved in the publishing process even when the (sometimes dubious, patchy) quality checks are performed by us who created the literature. it’s up to research institutions whether this story repeats with research data - remember who you’re dealing with when you sign those contracts! in the s the australian national data (ands) service funded investment in metadata stores; one of these was the redbox research data management platform which is alive and well and being sustained by qcif with a subscription maintenance service. but ands didn’t fund development of research data repositories. the work i’ve talked about here was all done with with the uts team. peter sefton: research data management looking outward from it this is a presentation that i gave on wednesday the nd of december at the aero (australian eresearch organizations) council meeting at the request of the chair dr carina kemp). carina asked: it would be really interesting to find out what is happening in the research data management space. and i’m not sure if it is too early, but maybe touch on what is happening in the eosc science mesh project. the audience of the aero council is aero member reps from aaf, aarnet, qcif, caudit, csiro, ga, tpac, the uni of auckland, reannz, adsei, curtin, unsw, apo. at this stage i was still the eresearch support manager at uts - but i only had a couple of weeks left in that role. in this presentation i’m going to start from a naive it perspective about research data. i would like to acknowledge and pay my respects to the gundungurra and darug people who are the traditional custodians of the land on which i live and work. research data is special - like snowflakes - and i don’t mean that in a mean way, research data could be anything - any shape any size and researchers are also special, not always % aligned with institutional priorities, they align with their disciplines and departments and research teams. it’s obvious that buying storage doesn’t mean you’re doing data management well but that doesn’t mean it’s not worth restating. so "data storage is not data management". in fact, the opposite might be true - think about buying a laptop - do you just get one that fits all your stuff and rely on getting a bigger one every few years? or do you get a smaller main drive and learn how to make sure that your data's actually archived somewhere? that would be managing data. and remember that not all research data is the same “shape” as corporate data - it does not all come in database or tabular form - it can be images, video, text, with all kinds of structures. there are several reasons we don’t want to just dole-out storage as needed. it’s going to cost a lot of money and keep costing a lot of money not everything is stored in “central storage” anyway. there are share-sync services like aarnet’s cloudstor. just keeping things doesn’t mean we can find them again so far we’ve just looked at things from an infrastructure perspective but that’s not actually why we’re here, us with jobs in eresearch. i think we’re here to help researchers do excellent research with integrity, and we need to help our institutions and researchers manage risk. the australian code for the responsible conduct of research which all research organizations need to adhere to if we get arc or nhmrc grants sets out some institutional responsibilities to provide infrastructure and training there are risks associated with research data, reputational, financial and risks to individuals and communities about whom we hold data at uts, we’ve embraced the research data management plan - as a way to assist in dealing with this risk. rdmps have a mixed reputation here in australia - some organizations have decided to keep them minimal and as streamlined as possible but at uts the thinking is that they can be useful in addressing a lot of the issues raised so far. where’s the data for project x - when there’s an integrity investigation. were procedures followed? how much storage are we going to need? inspired by the (defunct?) research data lifecycle project that was conceived by the former organizations that became the australian research data commons (ands, nectar and rdsi) we came up with this architecture for a central research data management system (in our case we use the open source redbox system) loosely linked to a variety of research workspaces, as we call them. the plan is that over time, researchers can plan and budget for data management in the short, medium and long term, provision services and use the system to archive data as they go. (diagram by gerard barthelot at uts) uts has been an early adopter of the ocfl (oxford common file layout) specificiation - a way of storing file sustainably on a file system (coming soon: s cloud storage) so it does not need to be migrated. i presented on this at the open repositories conference and at the same conference, i introduced the ro-crate standards effort, which is a marriage between the datacrate data packaging work we’ve been doing at uts for a few years, and the research object project. we created the arkisto platform to bring together all the work we’ve been doing to standardise research data metadata, and to build a toolkit for sustainable data repositories at all scales from single-collection up to institutional, and potentially discipline and national collections. this is an example of one of many arkisto deployment patterns you can read more on the arkisto use cases page this is an example of an arkisto-platform output. data exported from one content management system into an archive-ready ro-crate package, which can then be made into a live site. this was created for ass prof tamson pietsch at uts. the website is ephemeral - the data will be interoperable and reusable (i and r from fair) via the use of ro-crate. now to higher-level concerns: i built this infrastructure for my chooks (chickens) - they have a nice dry box with a roosting loft. but most of the time they roost on the roof. we know all too well that researchers don’t always use the infrastructure we build for them - you have to get a few other things right as well. one of the big frustrations i have had as an eresearch manager is that the expectations and aspirations of funders and integrity managers and so on are well ahead of our capacity to deliver the services they want, and then when we do get infrastructure sorted there are organizational challenges to getting people to use it. to go back to my metaphor, we can’t just pick up the researchers from the roof and put them in their loft, or spray water on them to get them to move. via gavin kennedy and guido aben from aarnet marco la rosa and i are helping out with this charmingly named project which is adding data management service to storage, syncronization and sharing services. contracts not yet in place so won't say much about this yet. https://www.cs mesh eosc.eu/about eosc is the european open science cloud cs mesh eosc - interactive and agile sharing mesh of storage, data and applications for eosc - aims to create an interoperable federation of data and higher-level services to enable friction-free collaboration between european researchers. cs mesh eosc will connect locally and individually provided services, and scale them up at the european level and beyond, with the promise of reaching critical mass and brand recognition among european scientists that are not usually engaged with specialist einfrastructures. i told carina i would look outwards as well. what are we keeping an eye on? watch out for the book factory. sorry, the publishing industry. the publishing industry is going to “help” the sector look after it’s research data. like, you, know, they did with the copyright in publications. not only did that industry work out how to take over copyright in research works, they successfully moved from selling us hard-copy resources that we could keep in our own libraries to charging an annual rent on the literature - getting to the point where they can argue that they are essential to maintaining the scholarly record and must be involved in the publishing process even when the (sometimes dubious, patchy) quality checks are performed by us who created the literature. it’s up to research institutions whether this story repeats with research data - remember who you’re dealing with when you sign those contracts! in the s the australian national data (ands) service funded investment in metadata stores; one of these was the redbox research data management platform which is alive and well and being sustained by qcif with a subscription maintenance service. but ands didn’t fund development of research data repositories. the work i’ve talked about here was all done with with the uts team. hugh rundle: library map part - how this is the second in a series of posts about my new library map. you probably should read the first post if you're interested in why i made the map and why it maps the particular things that it does. i expected this to be a two part series but it looks like i might make a third post about automation. the first post was about why i made the map. this one is about how. the tech stack the map is built with a stack of (roughly in order): original shape (shp) and geojson files qgis geojson a bunch of csv files a tiny python script topojson some html, css and javascript leafletjs and leaflet plugins map box tile service boundary files since i primarily wanted to map things about library services rather than individual library buildings, the first thing i looked for was geodata boundary files. in australia public libraries are usually run by local government, so the best place to start was with local government boundaries. this is reasonably straightforward to get - either directly from data.gov.au or one of the state equivalents, or more typically by starting there and eventually getting to the website of the state department that deals with geodata. usually the relevant file is provided as shapefile, which is not exactly what we need, but is a vector format, which is a good start. i gradually added each state and data about it before moving on to the next one, but the process would basically have been the same even if i'd had all of the relevant files at the same time. there are two slight oddities at this point that may (or may not 😂) be of interest. australian geography interlude the first is that more or less alone of all jurisdictions, queensland provides local government (lga) boundaries for coastal municipalities with large blocks covering the coastal waters and any islands. other states draw boundaries around outlying islands and include the island — as an island — with the lga that it is part of (if it's not "unincorporated", which is often the case in victoria for example). as a result, the national map looks a bit odd when you get to queensland, because the overlay bulges out slightly away from the coast. i'm not sure whether this is something to do with the lga jurisdictions in queensland, perhaps due to the great barrier reef, or whether their cartography team just couldn't be bothered drawing lines around every little island. secondly, when i got to western australia i discovered two things: the cocos (keeling) islands are an overseas territory of australia; and cocos and christmas islands have some kind of jurisdictional relationship with western australia, and are included in the western australia lga files. i hadn't really considered including overseas territories, but since they were right there in the file, i figured i may as well. later this led to a question about why norfolk island was missing, so i hunted around and found a shapefile for overseas territories, which also included cocos and christmas islands. shapefiles are a pretty standard format, but i wanted to use leafletjs, and for that we need the data to be in json format. i also needed to both stitch together all the different state lga files, and merge boundaries where local councils have formed regional library services. this seems to be more common in victoria (which has regional library corporations) than other states, but it was required in victoria, new south wales, and western australia. lastly, it turns out there are significant parts of australia that are not actually covered by any local government at all. some of these areas are the confusingly named national parks that are actually governed directly by states. others are simply 'unincorporated' — the two largest areas being the unincorporated far west region of new south wales (slightly larger than hungary), and the pastoral unincorporated area that consists of almost % of the landmass of south australia (slightly smaller than france). i had no idea these two enormous areas of australia had this special status. there's also a pretty large section of the south of the northern territory that contains no libraries at all, and hence has no library service. if you're wondering why there is a large section of inland australia with no overlays on the library map, now you know. qgis and geojson so, anyway, i had to munge all these files — mostly shape but also geojson — and turn them into a single geojson file. i've subsequently discovered mapshaper which i might have used for this, but i didn't know about it at the time, so i used qgis. i find the number of possibilities presented by qgis quite overwhelming, but there's no doubt it's a powerful tool for manipulating gis data. i added each shapefile as a layer, merged local government areas that needed to be merged, either deleted or dissolved (into the surrounding area) the unincorporated areas, and then merged the layers. finally, i exported the new merged layer as geojson, which is exactly what it sounds like: ordinary json, for geodata. csv data at this point i had boundaries, but not other data. i mean, this is not actually true, because i needed information about library services in order to know which lgas collectively operate a single library service, but in terms of the files, all i had was a polygon and a name for each area. i also had a bunch of location data for the actual library branches in a variety of formats originally, but ultimately in comma separate values (csv) format. i also had a csv file for information about each library service. the question at this point was how to associate the information i was mapping with each area. there was no way i was going to manually update + rows in qgis. luckily, csv and json are two of the most common open file formats, and they're basically just text. python script i'd had a similar problem in a previous, abandoned mapping project, and had a pretty scrappy python script lying around. with a bit more python experience behind me, i was able to make it more flexible and simpler. if we match on the name of the library service, it's fairly straightforward to add properties to each geojson feature (the features being each library service boundaries area, and the properties being metadata about that feature). this is so because the value of properties within each feature is itself simply a json object: {"type": "featurecollection","name": "library_services","crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:epsg:: " } },"features": [{ "type": "feature", "properties" : {"name": "bulloo shire"} "geometry": { "type": "multipolygon", "coordinates": [ [ [ [ . ,- . ],[ . ,- . ] ... ]]} the python script uses python's inbuilt json and csv modules to read both the geojson and the csv file, then basically merge the data. i won't re-publish the whole thing, but the guts of it is: # for each geojson feature, if a field in the json matches a field in the csv, add new properties to the jsonfor feature in json_data['features']: with open(csv_file, newline='') as f: # use dictreader so we can use the header names reader = csv.dictreader(f) for row in reader: # look for match if row[csv_match] == feature['properties'][geojson_match]: # create new properties in geojson for k in row: feature['properties'][k] = row[k] the whole thing is fewer than lines long. this saved me heaps of time, but as you'll discover in my future post on automation, i later worked out how to automate the whole process every time the csv file is updated! topojson geojson is pretty cool — it's specifically designed for web applications to read and write gis files in a native web format. unfortunately, geojson can also get very big, especially with a project like mine where there are lots of boundaries over a large area. the final file was about mb — far too big for anyone to reasonably wait for it to load in their browser (and chrome just refused to load it altogether). because of the way i originally wrote the python script, it actually became nearly three times the size, because i put in a two-space indent out of habit. this created literally hundreds of megabytes of empty spaces. "pretty printing" json is helpful if a human needs to read it, but rather unhelpful if you want to keep the file size down. enter topojson. to be honest i don't really understand the mathematics behind it, but topojson allows you to represent the same information as geojson but in a much, much smaller file. i reduced a mb geojson file (admittedly, about mb being blank spaces) to . mb simply by converting it to topojson! by "quantising" it (essentially, making it less accurate), the file size can be reduced even further, rendering the current file of about . mb - definitely small enough to load in a browser without too much of a wait, albeit not lightning fast. good old html/css/javascript at this point we're ready to start putting together the website to display the map. for this i used plain, vanilla html, css, and javascript. the web is awash with projects, frameworks and blog posts explaining how to use them to create your spa (single page app)™️, but we really don't need any of that. the leaflet docs have a pretty good example of a minimal project, and my map is really not much more complex than that. something that did stump me for a while was how to bring the topojson and csv files into the javascript file as variables. i'm a self-taught javascript coder, and i learned it back to front: initially as a backend scripting language (i.e. nodejs) and then as the front-end browser scripting language it was originally made to be. so sometimes something a front-end developer would consider pretty basic: "how do i import a text file into my javascript and assign it to a variable?" takes me a while to work out. initially i just opened the files in a text editor and copy-pasted the contents between two quote marks, made it the value of a javascript variable, and saved the whole thing as a .js file. but it was obvious even to me that couldn't possibly be the correct way to do it, even though it worked. in nodejs i would use fs.readfile() but the only thing that looked vaguely similar for front end javascript was filereader — which is for reading files on a client, not a server. finally i did a bit of research and found that the answer is to forget that the file is sitting right there in the same directory as all your javascript and html files, and just use ajax like it's a remote file. the modern way to do this is with fetch, so instead of doing this: // index.html<script src="./boundaries.js" type="text/javascript"></script><script src="./branchescsv.js" type="text/javascript"></script><script src="./ikccsv.js" type="text/javascript"></script><script src="./mechanics.js" type="text/javascript"></script><script src="./nslabranches.js" type="text/javascript"></script><script src="./load-map.js" type="text/javascript"></script>// boundaries.jsconst boundaries = `{"contents": "gigantic json string"}`// branchescsv.jsconst branchescsv = `lat,lng,town,address,phone- . , . ,victor harbor public library service, bay road, ... etc`// ikccsv.jsconst ikccsv = `lat,lng,town,address,phone- . , . ,badu island indigenous knowledge centre,nona street , ...etc`// mechanics.jsconst mechanics = `lat,lng,town,address,phone- . , . ,ballaarat mechanics institute, sturt street, ..etc`// nslabranches.jsconst nslabranches = `lat,lng,town,address,phone- . , . ,state library of victoria," swanston street, melbourne", ... etc`// load-map.js // boundaries and the other constants are now globals const loanperiod = new l.topojson(boundaries, options) ...we do this: // index.html<script src="./load-map.js" type="text/javascript"></script>// load-map.jsconst boundaries = fetch('data/boundaries.topo.json').then( response => response.json())const branchescsv = fetch('data/public_library_locations.csv').then( response => response.text());const ikccsv = fetch('data/indigenous_knowledge_centre_locations.csv').then( response => response.text());const mechanics = fetch('data/mechanics_institute_locations.csv').then( response => response.text());const nslabranches = fetch('data/nsla_library_locations.csv').then( response => response.text());// fetch returns a promise so we have to let them all 'settle' before we can use the returned valuepromise.all([boundaries, branchescsv, ikccsv, mechanics, nslabranches]).then( data => { // data is an array with the settled values of the fetch() promises const loanperiod = new l.topojson(data[ ], options)} in the code this doesn't necessarily look much simpler, but in terms of workflow it's a huge improvement that cuts out manually copy-pasting every time a csv or topojson file is updated, and reduces duplication and the total number of files. so now the site consists of: the original data as csv and topojson files an index.html file to display the map a single css file for basic styling a single javascript file to load the map leaflet and friends finally it's time to actually put all of this stuff into a map using leaflet. this is a really great javascript library, with pretty good documentation. leaflet allows us to plot shapes onto a map, and using javascript to make them interactive - including adding popups, zoom to features when they're clicked, and add interactive overlays. i won't try to replicate the leaflet docs here and explain the exact steps to making my map, but i do want to highlight how two leaflet plugins really helped with making the map work nicely. leaflet has a fairly strong plugin collection, and they allow the base library to be fairly lightweight whilst the entire system is still quite flexible and fully featured. i knew from the beginning it would require the whole library community to keep the map up to date over time. there are hundreds of library services across australia, and they don't set their rules or their procurement decisions in stone forever. so it needed to be relatively simple to update the data as it changes. as we've discussed, geojson also takes up a lot of space. ideally, i could store as much data in csv files as possible, and use them directly as data feeding the map. turns out there's a plugin for that - leaflet.geocsv. this allows us to load csv files directly (for library building locations), and it's converted to geojson on the fly. since csv files are much smaller than the equivalent data in json, this is not only easier to maintain, but also loads faster. the second plugin that really helped was leaflet.pattern. the problem this helped me to solve was how to show both the fines layer and the loan period layer at the same time. typically for a chloropleth map, different colours or shades indicate certain values. but if you add a second overlay on top of the first one, the colours no longer necessarily make much sense and combinations can be difficult or impossible to discern. thinking about this, i figured if i could make one layer semi-transparent colours, and the second layer patterns like differently angled stripes or dots, that might do the trick. leaflet.pattern to the rescue! after some alpha-testing by my go-to volunteer quality assurance tester, i worked out how to make the layers always appear in the same order, regardless of which order they were added or removed, making the combination always look consistent: tile service once all of that's complete, we can load the map. but there's a problem: all we have is a bunch of vector points and lines, there's no underlying geography. for this we need a map tile service. we can use one of several options provided by openstreetmap, but i ended up using the commercial map box service on a free plan (or at least, it will be free as long as thousands of people don't suddenly start using the map all at the same time). their dark and light map styles really suited what i was trying to do, with minimal detail in terms of the underlying geography, but with roads and towns marked at the appropriate zoom level. so that's it! it took a while to work it all out, but most of the complexity is in getting the data together rather than displaying the map. once i had that done (though there is still a fair bit of information missing), i was able to pay more attention to maintaining the map into the future. that led me to look into some options for automating the merging of data from the library services csv file (when it's updated) into the topojson file, and also automatically refreshing the data on the actual map when the github repository is updated. in my next post i'll explain how that works. while you're waiting for that, you can help me find missing data and make the map more accurate 😀. jonathan rochkind: managed solr saas options i was recently looking for managed solr “software-as-a-service” (saas) options, and had trouble figuring out what was out there. so i figured i’d share what i learned. even though my knowledge here is far from exhaustive, and i have only looked seriously at one of the ones i found. the only managed solr options i found were: websolr; searchstax; and opensolr. of these, i think websolr and searchstax are more well-known, i couldn’t find anyone with experience with opensolr, which perhaps is newer. of them all, searchstax is the only one i actually took for a test drive, so will have the most to say about. why we were looking we run a fairly small-scale app, whose infrastructure is currently self-managed aws ec instances, running respectively: ) a rails web app ) bg workers for the rails web app ) postgres, and ) solr. oh yeah, there’s also a redis running one of those servers, on # with pg or # with solr, i forget. currently we manage this all ourselves, right on the ec . but we’re looking to move as much as we can into “managed” servers. perhaps we’ll move to heroku. perhaps we’ll use hatchbox. or if we do stay on aws resources we manage directly, we’d look at things like using an aws rds postgres instead of installing it on an ec ourselves, an aws elasticache for redis, maybe look into elastic beanstalk, etc. but no matter what we do, we need a solr, and we’d like to get it managed. hatchbox has no special solr support, aws doesn’t have a solr service, heroku does have a solr add-on but you can also use any solr with it and we’ll get to that later. our current solr use is pretty small scale. we don’t run “solrcloud mode“, just legacy ordinary solr. we only have around , documents in there (tiny for solr), our index size is only mb. our traffic is pretty low — when i tried to figure out how low, it doesn’t seem we have sufficient logging turned on to answer that specifically but using proxy metrics to guess i’d say k- k requests a day, query as well as add. this is a pretty small solr installation, although it is used centrally for the primary functions of the (fairly low-traffic) app. it currently runs on an ec t a.small, which is a “burstable” ec type with only g of ram. it does have two vcpus (that is one core with ‘hyperthreading’). the t a.small ec instance only costs $ /month on-demand price! we know we’ll be paying more for managed solr, but we want to do get out of the business of managing servers — we no longer really have the staff for it. websolr (didn’t actually try out) websolr is the only managed solr currently listed as a heroku add-on. it is also available as a managed solr independent of heroku. the pricing in the heroku plans vs the independent plans seems about the same. as a heroku add-on there is a $ “staging” plan that doesn’t exist in the independent plans. (unlike some other heroku add-ons, no time-limited free plan is available for websolr). but once we go up from there, the plans seem to line up. starting at: $ /month for: million document limit k requests/day index mb storage concurrent requests limit (this limit is not mentioned on the independent pricing page?) next level up is $ /month for: million document limit k requests/day . gb storage concurrent request limit (again concurrent request limits aren’t mentioned on independent pricing page) as you can see, websolr has their plans metered by usage. $ /month is around the price range we were hoping for (we’ll need two, one for staging one for production). our small solr is well under million documents and ~ gb storage, and we do only use one index at present. however, the k requests/day limit i’m not sure about, even if we fit under it, we might be pushing up against it. and the “concurrent request” limit simply isn’t one i’m even used to thinking about. on a self-managed solr it hasn’t really come up. what does “concurrent” mean exactly in this case, how is it measured? with puma web workers and sometimes a possibly multi-threaded batch index going on, could we exceed a limit of ? seems plausible. what happens when they are exceeded? your solr request results in an http error! do i need to now write the app to rescue those gracefully, or use connection pooling to try to avoid them, or something? having to rewrite the way our app functions for a particular managed solr is the last thing we want to do. (although it’s not entirely clear if those connection limits exist on the non-heroku-plugin plans, i suspect they do?). and in general, i’m not thrilled with the way the pricing works here, and the price points. i am positive for a lot of (eg) heroku customers an additional $ * =$ /month is peanuts not even worth accounting for, but for us, a small non-profit whose app’s traffic does not scale with revenue, that starts to be real money. it is not clear to me if websolr installations (at “standard” plans) are set up in “solrcloud mode” or not; i’m not sure what api’s exist for uploading your custom schema.xml (which we’d need to do), or if they expect you to do this only manually through a web ui (that would not be good); i’m not sure if you can upload custom solrconfig.xml settings (this may be running on a shared solr instance with standard solrconfig.xml?). basically, all of this made websolr not the first one we looked at. does it matter if we’re on heroku using a managed solr that’s not a heroku plugin? i don’t think so. in some cases, you can get a better price from a heroku plug-in than you could get from that same vendor not on heroku or other competitors. but that doesn’t seem to be the case here, and other that that does it matter? well, all heroku plug-ins are required to bill you by-the-minute, which is nice but not really crucial, other forms of billing could also be okay at the right price. with a heroku add-on, your billing is combined into one heroku invoice, no need to give a credit card to anyone else, and it can be tracked using heroku tools. which is certainly convenient and a plus, but not essential if the best tool for the job is not a heroku add-on. and as a heroku add-on, websolr provides a websolr_url heroku config/env variable automatically to code running on heroku. ok, that’s kind of nice, but it’s not a big deal to set a solr_url heroku config manually referencing the appropriate address. i suppose as a heroku add-on, websolr also takes care of securing and authenticating connections between the heroku dynos and the solr, so we need to make sure we have a reasonable way to do this from any alternative. searchstax (did take it for a spin) searchstax’s pricing tiers are not based on metering usage. there are no limits based on requests/day or concurrent connections. searchstax runs on dedicated-to-you individual solr instances (i would guess running on dedicated-to-you individual (eg) ec , but i’m not sure). instead the pricing is based on size of host running solr. you can choose to run on instances deployed to aws, google cloud, or azure. we’ll be sticking to aws (the others, i think, have a slight price premium). while searchstax gives you a pricing pages that looks like the “new-way-of-doing-things” transparent pricing, in fact there isn’t really enough info on public pages to see all the price points and understand what you’re getting, there is still a kind of “talk to a salesperson who has a price sheet” thing going on. what i think i have figured out from talking to a salesperson and support, is that the “silver” plans (“starting at $ a month”, although we’ll say more about that in a bit) are basically: we give you a solr, we don’t don’t provide any technical support for solr. while the “gold” plans “from $ /month” are actually about paying for solr consultants to set up and tune your schema/index etc. that is not something we need, and $ +/month is way more than the price range we are looking for. while the searchstax pricing/plan pages kind of imply the “silver” plan is not suitable for production, in fact there is no real reason not to use it for production i think, and the salesperson i talked to confirmed that — just reaffirming that you were on your own managing the solr configuration/setup. that’s fine, that’s what we want, we just don’t want to mangage the os or set up the solr or upgrade it etc. the silver plans have no sla, but as far as i can tell their uptime is just fine. the silver plans only guarantees -hour support response time — but for the couple support tickets i filed asking questions while under a free -day trial (oh yeah that’s available), i got prompt same-day responses, and knowledgeable responses that answered my questions. so a “silver” plan is what we are interested in, but the pricing is not actually transparent. $ /month is for the smallest instance available, and if you prepay/contract for a year. they call that small instance an ndn and it has gb of ram and gb of storage. if you pay-as-you-go instead of contracting for a year, that already jumps to $ /month. (that price is available on the trial page). when you are paying-as-you-go, you are actually billed per-day, which might not be as nice as heroku’s per-minute, but it’s pretty okay, and useful if you need to bring up a temporary solr instance as part of a migration/upgrade or something like that. the next step up is an “ndn ” which has g of ram and gb of storage, and has an ~$ /month pay-as-you-go — you can find that price if you sign-up for a free trial. the discount price price for an annual contract is a discount similar to the ndn %, $ /month — that price i got only from a salesperson, i don’t know if it’s always stable. it only occurs to me now that they don’t tell you how many cpus are available. i’m not sure if i can fit our solr in the g ndn , but i am sure i can fit it in the g ndn with some headroom, so i didn’t look at plans above that — but they are available, still under “silver”, with prices going up accordingly. all searchstax solr instances run in “solrcloud” mode — these ndn and ndn ones we’re looking at just run one node with one zookeeper, but still in cloud mode. there are also “silver” plans available with more than one node in a “high availability” configuration, but the prices start going up steeply, and we weren’t really interested in that. because it’s solrcloud mode though, you can use the standard solr api for uploading your configuration. it’s just solr! so no arbitrary usage limits, no features disabled. the searchstax web console seems competently implemented; it let’s you create and delete individual solr “deployments”, manage accounts to login to console (on “silver” plan you only get two, or can pay $ /month/account for more, nah), and set up auth for a solr deployment. they support ip-based authentication or http basic auth to the solr (no limit to how many solr basic auth accounts you can create). http basic auth is great for us, because trying to do ip-based from somewhere like heroku isn’t going to work. all solrs are available over https/ssl — great! searchstax also has their own proprietary http api that lets you do most anything, including creating/destroying deployments, managing solr basic auth users, basically everything. there is some api that duplicates the solr cloud api for adding configsets, i don’t think there’s a good reason to use it instead of standard solrcloud api, although their docs try to point you to it. there’s even some kind of webhooks for alerts! (which i haven’t really explored). basically, searchstax just seems to be a sane and rational managed solr option, it has all the features you’d expect/need/want for dealing with such. the prices seem reasonable-ish, generally more affordable than websolr, especially if you stay in “silver” and “one node”. at present, we plan to move forward with it. opensolr (didn’t look at it much) i have the least to say about this, have spent the least time with it, after spending time with searchstax and seeing it met our needs. but i wanted to make sure to mention it, because it’s the only other managed solr i am even aware of. definitely curious to hear from any users. here is the pricing page. the prices seem pretty decent, perhaps even cheaper than searchstax, although it’s unclear to me what you get. does “ solr clusters” mean that it’s not solrcloud mode? after seeing how useful solrcloud apis are for management (and having this confirmed by many of my peers in other libraries/museums/archives who choose to run solrcloud), i wouldn’t want to do without it. so i guess that pushes us to “executive” tier? which at $ /month (billed yearly!) is still just fine, around the same as searchstax. but they do limit you to one solr index; i prefer searchstax’s model of just giving you certain host resources and do what you want with it. it does say “shared infrastructure”. might be worth investigating, curious to hear more from anyone who did. now, what about elasticsearch? we’re using solr mostly because that’s what various collaborative and open source projects in the library/museum/archive world have been doing for years, since before elasticsearch even existed. so there are various open source libraries and toolsets available that we’re using. but for whatever reason, there seem to be so many more managed elasticsearch saas available. at possibly much cheaper pricepoints. is this because the elasticsearch market is just bigger? or is elasticsearch easier/cheaper to run in a saas environment? or what? i don’t know. but there’s the controversial aws elasticsearch service; there’s the elastic cloud “from the creators of elasticsearch”. on heroku that lists one solr add-on, there are three elasticsearch add-ons listed: elasticcloud, bonsai elasticsearch, and searchbox elasticsearch. if you just google “managed elasticsearch” you immediately see or other names. i don’t know enough about elasticsearch to evaluate them. there seem on first glance at pricing pages to be more affordable, but i may not know what i’m comparing and be looking at tiers that aren’t actually usable for anything or will have hidden fees. but i know there are definitely many more managed elasticsearch saas than solr. i think elasticsearch probably does everything our app needs. if i were to start from scratch, i would definitely consider elasticsearch over solr just based on how many more saas options there are. while it would require some knowledge-building (i have developed a lot of knowlege of solr and zero of elasticsearch) and rewriting some parts of our stack, i might still consider switching to es in the future, we don’t do anything too too complicated with solr that would be too too hard to switch to es, probably. digital library federation: three new ndsa members since january , the ndsa coordinating committee unanimously voted to welcome three new members. each of these members bring a host of skills and experience to our group. please help us to welcome: arkivum: arkivum is recognized internationally for its expertise in the archiving and digital preservation of valuable data and digitized assets in large volumes and multiple formats. colorado state university libraries: colorado state university libraries’ digital preservation activities has focused on web archiving, targeted born-digital collecting, along with collection development and preservation guidelines for its digital repository. vassar college libraries: vassar college libraries are committed to supporting a framework of sustainable access to our digital collections and to participate locally, nationally, and globally with other cultural and professional organizations and institutions in efforts to preserve, augment, and disseminate our collective documentary heritage. each organization has participants in one or more of the various ndsa interest and working groups, so keep an eye out for them on your calls and be sure to give them a shout out. please join me in welcoming our new members. a complete list of ndsa members is on our website. in future, ndsa is moving to a quarterly process for reviewing membership applications. announcements for new members will be scheduled accordingly. ~ nathan tallman, vice chair of the ndsa coordinating committee the post three new ndsa members appeared first on dlf. duraspace news: fedora migration paths and tools project update: january this is the fourth in a series of monthly updates on the fedora migration paths and tools project – please see last month’s post for a summary of the work completed up to that point. this project has been generously funded by the imls. the grant team has been focused on completing an initial build of a validation utility, which will allow implementers to compare their migrated content with the original fedora .x source material to verify that everything has been migrated successfully. a testable version of this tool is expected to be completed in the coming weeks, at which point the university of virginia pilot team will test and provide feedback on the utility. the university of virginia team has completed a full migration of their legacy fedora . . repository. they also recently contributed improvements to the fedora aws deployer which have been merged into the codebase. the team is now awaiting a testable version of the validation utility so they can validate their migrated content before moving on to testing this content in a newly installed fedora . instance. the whitman college pilot team has completed their metadata remediation and mapping work. their process and lessons learned will be shared in a presentation at the upcoming code lib conference. meanwhile, islandora is currently being tested with an alpha build of fedora . , which will be used as the basis for migration testing for the whitman college pilot. work is currently being done in parallel to install islandora using isle and complete work on a new theme. due to the impending end-of-life date of drupal the team decided to proceed directly to drupal , and the theme needed to be updated accordingly. fortunately, the transition from drupal to is relatively minor. next month we plan to use the validation utility to validate the university of virginia migration before moving on to testing the migrated data in fedora . and updating the application as needed. for the whitman college pilot, once the islandora with fedora . installation is complete we will be able to run a series of test migrations and update the utilities and application as necessary in order to satisfy functional requirements. stay tuned for future updates! the post fedora migration paths and tools project update: january appeared first on duraspace.org. open knowledge foundation: open knowledge justice programme takes new step on its mission to ensure algorithms cause no harm today we are proud to announce a new project for the open knowledge justice programme – strategic litigation. this might mean we will go to court to make sure public impact algorithms are used fairly, and cause no harm. but it will also include advocacy in the form of letters and negotiation. the story so far last year, open knowledge foundation made a commitment to apply our skills and network to the increasingly important topics of artificial intelligence (ai) and algorithms. as a result, we launched the open knowledge justice programme in april . our mission is to ensure that public impact algorithms cause no harm. public impact algorithms have four key features: they involve automated decision-making using ai and algorithms by governments and corporate entities and have the potential to cause serious negative impacts on individuals and communities. we aim to make public impact algorithms more accountable by equipping legal professionals, including campaigners and activists, with the know-how and skills they need to challenge the effects of these technologies in their practice. we also work with those deploying public impact algorithms to raise awareness of the potential risks and build strategies for mitigating them. we’ve had some great feedback from our first trainees! why are we doing this? strategic litigation is more than just winning an individual case. strategic litigation is ‘strategic’ because it plays a part in a larger movement for change. it does this by raising awareness of the issue, changing public debate, collaborating with others fighting for the same cause and, when we win (hopefully!) making the law fairer for everyone. our strategic litigation activities will be grounded in the principle of openness because public impact algorithms are overwhelmingly deployed opaquely. this means that experts that are able to unpick why and how ai and algorithms are causing harm cannot do so and the technology escapes scrutiny. vendors of the software say they can’t release the software code they use because it’s a trade secret. this proprietary knowledge, although used to justify decisions potentially significantly impacting people’s lives, remains out of our reach. we’re not expecting all algorithms to be open. nor do we think that would necessarily be useful. but we do think it’s wrong that governments can purchase software and not be transparent around key points of accountability such as its objectives, an assessment of the risk it will cause harm and its accuracy. openness is one of our guiding principles in how we’ll work too. as far as we are able, we’ll share our cases for others to use, re-use and modify for their own legal actions, wherever they are in the world. we’ll share what works, and what doesn’t, and make learning resources to make achieving algorithmic justice through legal action more readily achievable. we’re excited to announce our first case soon, so stay tuned! sign up to our mailing list or follow the open knowledge justice programme on twitter to receive updates. david rosenthal: isp monopolies for at least the last three years (it isn't about the technology) i've been blogging about the malign effects of the way the faangs dominate the web and the need for anti-trust action to mitigate them. finally, with the recent lawsuits against facebook and google, some action may be in prospect. i'm planning a post on this topic. but when it comes to malign effects of monopoly i've been ignoring the other monopolists of the internet, the telcos.an insightful recent post by john gilmore to dave farber's ip list sparked a response from thomas leavitt and some interesting follow-up e-mail. gilmore was involved in pioneering consumer isps, and leavitt in pioneering web hosting. both attribute the current sorry state of internet connectivity in the us to the lack of effective competition. they and i differ somewhat on how the problem could be fixed. below the fold i go into the details.i've known gilmore since the early days of sun microsystems, and i'm grateful for good advice he gave me at a critical stage of my career. he has a remarkable clarity of vision and a list of achievements that includes pioneering paid support for free software (the "red hat" model) at cygnus support, and pioneering consumer internet service providers (isps) with the little garden. because in those dial-up days internet connectivity was a commercial product layered on regulated infrastructure (the analog telephone system) there was no lock-in. consumers could change providers simply by changing the number their modem dialed.this experience is key to gilmore's argument. he writes:the usa never had "network neutrality" before it was "suspended". what the usa had was , isps. so if an isp did something unfriendly to its customers, they could just stop paying the bad one, and sign up with a different isp that wouldn't screw them. that effectively prevented bad behavior among isps. and if the customer couldn't find an isp that wouldn't screw them, they could start one themselves. i know, because we did exactly that in the s.anyone could start an isp because by law, everyone had tariffed access to the same telco infrastructure (dialup phone lines, and leased lines at kbit/sec or . mbit/sec or mbit/sec). you just called up the telco and ordered it, and they sent out techs and installed it. we did exactly that, plugged it into our modems and routers and bam, we were an isp: "the little garden". i was an early customer of the little garden. a sparcstation, a scsi disk and a modem sat on my window-ledge. the system dialed a local, and thus free, number and kept the call up / , enabling me to register a domain and start running my own mail server. years later i upgraded to dsl with stanford as my isp. as gilmore points out, stanford could do this under the same law: later, dsl lines required installing equipment in telco central offices, at the far end of the wire that leads to your house. but the telcos were required by the fcc to allow competing companies to do that. their central office buildings were / th empty anyway, after they had replaced racks of mechanical relays with digital computers. gilmore explains how this competitive market was killed:the telcos figured this out, and decided they'd rather be gatekeepers, instead of being the regulated monopoly that gets a fixed profit margin. looking ahead, they formally asked the fcc to change its rule that telcos had to share their infrastructure with everybody -- but only for futuristic optical fibers. they whined that "fcc wants us to deploy fiber everywhere, but we won't, unless we get to own it and not share it with our competitors." as usual, the regulated monopoly was great at manipulating the public interest regulators. the fcc said, "sure, keep your fibers unshared." this ruling never even mentioned the internet, it is all about the physical infrastructure. if the physical stuff is wires, regulated telcos have to share it; if it's glass, they don't.the speed of dialup maxed out at kbit/sec. dsl maxed out at a couple of megabits. leased lines worked to mbit/sec but cost thousands of dollars per month. anything over that speed required fiber, not wire, at typical distances. as demand for higher internet speeds arose, any isp who wanted to offer a faster connection couldn't just order one from the telco, because the telco fibers were now private and unshared. if you want a fiber-based internet connection now, you can't buy it from anybody except the guys who own the fibers -- mostly the telcos. most of the , isps could only offer slow internet access, so everybody stopped paying them. the industry consolidated down to just one or a few businesses per region -- mostly the telcos themselves, plus the cable companies that had build their own local monopoly via city government contracts. especially lucky regions had maybe one other competitor, like a wireless isp, or an electrical co-op that ran fibers on its infrastructure.leavitt makes a bigger point than glimore's:the only reason the internet exists as we know it (mass consumer access) was the regulatory loophole which permitted the isp industry to flourish in the s. the telcos realized their mistake, as john said, and made sure that there wasn't going to be a repeat of that, so with each generation (dsl, fiber), they made it more and more difficult to access their networks, with the result that john mention--almost no consumer choice, for consumers or business. last office i rented, there was one choice of internet provider: the local cable monopoly, which arbitrarily wanted to charge me much more ($ /mo) to connect my office than it did the apartments upstairs in the same building ($ ). as is the case in most of that county; the only alternatives were a few buildings and complexes wired up by the two surviving local isps, and a relatively expensive wisp. gilmore concludes:the telcos' elimination of fiber based competition, and nothing else, was the end of so-called "network neutrality". the rest was just activists, regulators and legislators blathering. there never was an /enforceable federal regulatory policy of network neutrality, so the fcc could hardly suspend it. if the fcc actually wanted us customers to have a choice of isps, they would rescind the fiber rule. and if advocates actually understood how only competition, not regulation, restrains predatory behavior, they would ask fcc for the fiber rule to be rescinded, so a small isp company could rent the actual glass fiber that runs from the telco to (near or inside) your house, for the actual cost plus a regulated profit. then customers could get high speed internet from a variety of vendors at a variety of prices and terms. so far neither has happened. leavitt shows the insane lengths we are resorting to in order to deliver a modicum of competition in the isp market:it's ridiculous that it is going to take sending s of thousands of satellites into orbit to restore any semblance of competitiveness to the isp market, when we've had a simple regulatory fix all along. it's not like the telco/cable monopolies suffered as a result of competition... in fact, it created the market they now monopolize. imagine all the other opportunities for new markets that have been stifled by the lack of competition in the isp market over the last two decades! i have been, and still am, an exception to gilmore's and leavitt's experiences. palo alto owns its own utilities, a great reason to live there. in september palo alto's fiber to the home trial went live, and i was one of citizens who got a mbit/s bidirectional connection, with the city utilities as our isp. we all loved the price, the speed and the excellent customer service. the telcos got worried and threatened to sue the utilities if it expanded the service. the city was on safe legal ground, but that is what they had thought previously when they lost a $ . m lawsuit as part of the fallout from the enron scandal. enron's creditors claimed that the utilities had violated their contract because they stopped paying enron. the utilities did so because enron became unable to deliver them electricity.so, when the trial ended after i think five years, we loved it so much that we negotiated with motorola to take over the equipment and found an upstream isp. but the utilities were gun-shy and spent iirc $ k physically ripping out the fiber and trashing the equipment. since then, palo alto's approach to municipal fiber has been a sorry story of ineffective dithering.shortly after we lost our fiber, stanford decided to stop providing staff with dsl, but we again avoided doing business with the telcos. we got dsl and phone service from sonic, a local isp that was legally enabled to rent access to at&t's copper. it was much slower than comcast or at&t, but the upside was sonic's stellar customer service and four static ip addresses. that kept us going quite happily until covid- struck and we had to host our grandchildren for their virtual schools. dsl was not up to the job.fortunately, it turned out that sonic had recently been able to offer gigabit fiber in palo alto. sonic in its north bay homeland has been deploying its own fiber, as has cruzio in its santa cruz homeland. here they rent access to at&t's fiber in the same way that they rented access to the copper. so, after a long series of delays caused by at&t's inability to get fiber through the conduit under the street that held their copper, we have gigabit speed, home phone and sonic's unmatched customer service all for $ /month.as a long-time sonic customer, i agree with what the internet advisor website writes:sonic has maintained a reputation as not only a company that delivers a reliable high-speed connection to its customers but also a company that stands by its ethics. both dane jasper and scott doty have spoken up on numerous occasions to combat the ever-growing lack of privacy on the web. they have implemented policies that reflect this. in , they reduced the amount of time that they store user data to just two weeks in the face of an ever-growing tide of legal requests for its users’ data. that same year, sonic alongside google fought a court order to hand over email addresses who had contacted and had a correspondence with tor developer and wikileaks contributor jacob applebaum. when asked why, ceo dane jasper responded that it was “rather expensive, but the right thing to do.”sonic has made a habit of doing the right thing, both for its customers and the larger world. it’s a conscientious company that delivers on what is promised and goes the extra mile for its subscribers. leavitt explained in e-mail how sonic's exception to gilmore's argument came about:sonic is one of the few independent isps that's managed to survive the regulatory clampdown via stellar customer service and customers willing to go out of their way to support alternative providers, much like cruzio in my home town of santa cruz. they cut some kind of reseller deal with at&t back in that enabled them to offer fiber to a limited number of residents, and again, like cruzio, are building out their own fiber network, but according to [this site], fiber through them is potentially available to only about , customers (in a state with about million households and million businesses); it also reports that they are the th largest isp in the nation, despite being a highly regional provider with access available to only about million households. this says everything about how monopolistic and consolidated the isp market is, given the number of independent cable and telco companies that existed in previous decades, the remaining survivors of which are all undoubtedly offering isp services.i doubt sonic's deal with at&t was much more lucrative than the dsl deals santa cruz area isps were able to cut. gilmore attempted to build a fiber isp in his hometown, san francisco:our model was to run a fiber to about one person per block (what cruzio calls a "champion") and teach them how to run and debug g ethernet cables down the back fences to their neighbors, splitting down the monthly costs. this would avoid most of the cost of city right-of-way crud at every house, which would let us and our champions fiber the city much more broadly and quickly. and would train a small army of citizens to own and manage their own infrastructure. for unrelated reasons it didn't work out, but it left gilmore with the conviction that, absent repeal of the fiber rule, isp-owned fiber is the way to go. especially in rural areas this approach has been successful; a recent example was described by jon brodkin in jared mauch didn't have good broadband—so he built his own fiber isp. leavitt argues:i'd like to see multiple infrastructure providers, both private for profit, and municipally sponsored non-profit public service agencies, all with open access networks; ideally, connecting would be as simple as it was back in the dial up days. i think we need multiple players to keep each other "honest". i do agree that a lot of the barriers to building out local fiber networks are regulatory and process, as john mentions. the big incumbent players have a tremendous advantage navigating this process, and the scale to absorb the overhead of dealing with them in conjunction with the capital outlays (which municipalities also have). i think we all agree that "ideally, connecting would be as simple as it was back in the dial up days". how to make this happen? as gilmore says, there are regulatory and process costs as well as the cost of pulling the fiber. so if switching away from a misbehaving isp involves these costs there is going to a significant barrier. it isn't going to be "as simple as it was back in the dial up days" when the customer could simply re-program their modem.my experience of municipal fiber leads me to disagree with both gilmore and leavitt. for me, the key is to separate the provision of fiber from the provision of internet services. why would you want to switch providers?pretty much the only reason why you'd want to switch fiber providers is unreliability. but, absent back-hoes, fiber is extremely reliable.there are many reasons why you'd want to switch isps, among them privacy, bandwidth caps, price increases.municipal fiber provision is typically cheap, because they are the regulator and control the permitting process themselves, and because they are accountable to their voters. and if they simply provide the equivalent of an ethernet cable to a marketplace of isps, each of them will be paying the same for their connectivity. so differences in the price of isp service will reflect the features and quality of their service offerings. the cost of switching isps would be low, simply reprogramming the routers at each end of the fiber. the reason the telcos want to own the fiber isn't because owning fiber as such is a good business, it is to impose switching costs and thus lock in their customers. we don't want that. but equally we don't want switching isps to involve redundantly pulling fiber, because that imposes switching costs too. the only way to make connecting "as simple as it was back in the dial up days" is to separate fiber provision from internet service provision, so that fiber pets pulled once and rented to competing isps. if we are going to have a monopoly at the fiber level, i'd rather have a large number of small monopolies than the duopoly of at&t and comcast. and i'd rather have the monopoly accountable to voters. lucidworks: cast a smarter net with semantic vector search learn how to power the product discovery experience with semantic vector search to eliminate false zero results and accelerate the path to purchase. the post cast a smarter net with semantic vector search appeared first on lucidworks. digital library federation: virtual ndsa digital preservation recordings available online! session recordings from the virtual ndsa digital preservation conference are now available on ndsa’s youtube channel, as well as on aviary. the full program from digital preservation : get active with digital preservation, which took place online november , , is free and open to the public. ndsa is an affiliate of the digital library federation (dlf) and the council on library and information resources (clir). each year, ndsa’s annual digital preservation conference is held alongside the dlf forum and acts as a crucial venue for intellectual exchange, community-building, development of good practices, and national agenda-setting for digital stewardship. enjoy, tricia patterson; digipres vice-chair, chair the post virtual ndsa digital preservation recordings available online! appeared first on dlf. terry reese: marcedit . .x/macos . .x timelines i sent this to the marcedit listserv to provide info about my thoughts around timelines related to the beta and release. here’s the info. dear all, as we are getting close to feb. (when i’ll make the . beta build available for testing) – i wanted to provide information about the update process going forward. feb. : marcedit . download will be released. this will be a single build that includes both the and bit builds, dependencies, and can install if you have admin rights or non-admin rights. i expect to be releasing new builds weekly – with the goal of taking the beta tag off the build no later than april . marcedit . .x i’ll be providing updates for . .x till . comes out of beta. this will fold in some changes (mostly bug fixes) when possible. marcedit macos . .x i’ll be providing updates for macos . .x till . is out and out of beta marcedit macos . .x beta once marcedit . .x beta is out, i’ll be looking to push a . .x beta by mid-late feb. again, with the idea of taking the beta tag off by april (assuming i make the beta timeline) march marcedit macos . .x beta will be out and active (with weekly likely builds) marcedit . .x beta – testing assessed and then determine how long the beta process continues (with april being the end bookend date) marcedit . .x – updates continue marcedit macos . .x – updates continue april marcedit . .x comes out of beta marcedit . .x is deprecated marcedit macos . .x beta assessed – end bookend date is april th if above timelines are met may marcedit macos . .x is out of beta marcedit macos . .x is deprecated let me know if you have questions. jez cope: a new font for the blog i’ve updated my blog theme to use the quasi-proportional fonts iosevka aile and iosevka etoile. i really like the aesthetic, as they look like fixed-width console fonts (i use the true fixed-width version of iosevka in my terminal and text editor) but they’re actually proportional which makes them easier to read. https://typeof.net/iosevka/ jez cope: training a model to recognise my own handwriting if i’m going to train an algorithm to read my weird & awful writing, i’m going to need a decent-sized training set to work with. and since one of the main things i want to do with it is to blog “by hand” it makes sense to focus on that type of material for training. in other words, i need to write out a bunch of blog posts on paper, scan them and transcribe them as ground truth. the added bonus of this plan is that after transcribing, i also end up with some digital text i can use as an actual post — multitasking! so, by the time you read this, i will have already run it through a manual transcription process using transkribus to add it to my training set, and copy-pasted it into emacs for posting. this is a fun little project because it means i can: write more by hand with one of my several nice fountain pens, which i enjoy learn more about the operational process some of my colleagues go through when digitising manuscripts learn more about the underlying technology & maths, and how to tune the process produce more lovely content! for you to read! yay! write in a way that forces me to put off editing until after a first draft is done and focus more on getting the whole of what i want to say down. that’s it for now — i’ll keep you posted as the project unfolds. addendum tee hee! i’m actually just enjoying the process of writing stuff by hand in long-form prose. it’ll be interesting to see how the accuracy turns out and if i need to be more careful about neatness. will it be better or worse than the big but generic models used by samsung notes or onenote. maybe i should include some stylus-written text for comparison. lucidworks: learning from einstein’s brain einstein’s remarkable brain has an important lesson about balance for all of us in technology and machine learning. the post learning from einstein’s brain appeared first on lucidworks. meredith farkas: making customizable interactive tutorials with google forms in september, i gave a talk at oregon state university’s instruction librarian get-together about the interactive tutorials i built at pcc last year that have been integral to our remote instructional strategy. i thought i’d share my slides and notes here in case others are inspired by what i did and to share the amazing assessment data i recently received about the impact of these tutorials that i included in this blog post. you can click on any of the slides to see them larger and you can also view the original slides here (or below). at the end of the post are a few tutorials that you can access or make copies of. i’ve been working at pcc for over six years now, but i’ve been doing online instructional design work for years and i will freely admit that it’s my favorite thing to do. i started working at a very small rural academic library where i had to find creative and usually free solutions to instructional problems. and i love that sort of creative work. it’s what keeps me going. i’ve actually been using survey software as a teaching tool since i worked at portland state university. there, my colleague amy hofer and i used qualtrics to create really polished and beautiful interactive tutorials for students in our university studies program. i also used qualtrics at psu and pcc to create pre-assignments for students to complete prior to an instruction session that both taught students skills and gave me formative assessment data that informed my teaching. so for example, students would watch a video on how to search for sources via ebsco and then would try searching for articles on their own topic. a year and a half ago, the amazing anne-marie dietering led my colleagues in a day of goal-setting retreat for our instruction program. in the end, we ended up selecting this goal, identify new ways information literacy instruction can reach courses other than direct instruction, which was broad enough to encompass a lot of activities people valued. for me, it allowed me to get back to my true love, online instructional design, which was awesome, because i was kind of in a place of burnout going into last fall. at pcc, we already had a lot of online instructional content to support our students. we even built a toolkit for faculty with information literacy learning materials they could incorporate into their classes without working with a librarian. the toolkit contains lots of handouts, videos, in-class or online activities and more. but it was a lot of pieces and they really required faculty to do the work to incorporate them into their classes. what i wanted to build was something that took advantage of our existing content, but tied it up with a bow for faculty. so they really could just take whatever it is, assign students to complete it, and know students are learning and practicing what they learned. i really wanted it to mimic the sort of experience they might get from a library instruction session. and that’s when i came back to the sort interactive tutorials i built at psu. so i started to sketch out what the requirements of the project were. even though we have qualtrics at pcc, i wasn’t % sure qualtrics would be a good fit for this. it definitely did meet those first four criteria given that we already have it, it provides the ability to embed video, for students to get a copy of the work they did, and most features of the software are ada accessible. but i wanted both my colleagues in the library and disciplinary faculty members to be able to easily see the responses of their students and to make copies of the tutorial to personalize for the particular course. and while pcc does have qualtrics, the majority of faculty have never used it on the back-end and many do not have accounts. so that’s when google forms seemed like the obvious choice and i had to give up on my fantasy of having pretty tutorials. i started by creating a proof of concept based on an evaluating sources activity i often use in face-to-face reading and writing classes. you can view a copy of it here and can copy it if you want to use it in your own teaching. in this case, students would watch a video we have on techniques for evaluating sources. then i demonstrate the use of those techniques, which predate caulfield’s four moves, but are not too dissimilar. so they can see how i would go about evaluating this article from the atlantic on the subject of daca. the students then will evaluate two sources on their own and there are specific questions to guide them. during fall term, i showed my proof of concept to my colleagues in the library as well as at faculty department meetings in some of my liaison areas. and there was a good amount of enthusiasm from disciplinary faculty – enough that i felt encouraged to continue. one anthropology instructor who i’ve worked closely with over the years asked if i could create a tutorial on finding sources to support research in her online biological anthropology classes – classes i was going to be embedded in over winter term. and i thought this was a perfect opportunity to really pilot the use of the google form tutorial concept and see how students do. so i made an interactive tutorial where students go through and learn a thing, then practice a thing, learn another thing, then practice that thing. and fortunately, they seemed to complete the tutorial without difficulty and from what i heard from the instructor, they did a really good job of citing quality sources in their research paper in the course. later in the presentation, you’ll see that i received clear data demonstrating the impact of this tutorial from the anthropology department’s annual assessment project. so my vision for having faculty make copies of tutorials to use themselves had one major drawback. let’s imagine they were really successful and we let a thousand flowers bloom. well, the problem with that is that you now have a thousand versions of your tutorials lying around and what do you do when a video is updated or a link changes or some other update is needed? i needed a way to track who is using the tutorials so that i could contact them when updates were made. so here’s how i structured it. i created a qualtrics form that is a gateway to accessing the tutorials. faculty need to put in their name, email, and subject area. they then can view tutorials and check boxes for the ones they are interested in using. once they submit, they are taking to a page where they can actually copy the tutorials they want. so now, i have the contact information for the folks who are using the tutorials. this is not just useful for updates, but possibly for future information literacy assessment we might want to do. the individual tutorials are also findable via our information literacy teaching materials toolkit. so when the pandemic came just when i was ready to expand this, i felt a little like nostradamus or something. the timing was very, very good during a very, very bad situation. so we work with biology every single term in week to teach students about the library and about what peer review means, why it matters, and how to find peer-reviewed articles. as soon as it became clear that spring term was going to start online, i scrambled to create this tutorial that replicates, as well as i could, what we do in the classroom. so they do the same activity we did in-class where they look at a scholarly article and a news article and list the differences they notice. and in place of discussions, i had them watch videos and share insights. i then shared this with the biology faculty on my campus and they assigned it to their students in week . it was great! [you can view the biology tutorial here and make a copy of it here]. and during spring term i made a lot more tutorials. the biggest upside of using google forms is its simplicity and familiarity. nearly everyone has created a google form and they are dead simple to build. i knew that my colleagues in the library could easily copy something i made and tailor it to the courses they’re working with or make something from scratch. and i knew faculty could easily copy an existing tutorial and be able to see student responses. for students, it’s a low-bandwidth and easy-to-complete online worksheet. the barriers are minimal. and on the back-end, just like with libguides, there’s a feature where you can easily copy content from another google form. the downsides of using google forms are not terribly significant. i mean, i’m sad that i can’t create beautiful, modern, sharp-looking forms, but it’s not the end of the world. the formatting features in google forms are really minimal. to create a hyperlink, you actually need to display the whole url. blech. then in terms of accessibility, there’s also no alt tag feature for images, so i just make sure to describe the picture in the text preceding or following it. i haven’t heard any complaints from faculty about having to fill out the qualtrics form in order to get access to the tutorials, but it’s still another hurdle, however small. this spring, we used google form tutorials to replace the teaching we normally do in classes like biology , writing , reading , and many others. we’ve also used them in addition to synchronous instruction, sort of like i did with my pre-assignments. but word about the google form tutorials spread and we ended up working with classes we never had a connection to before. for example, the biology faculty told the anatomy and physiology instructors about the tutorial and they wanted me to make a similar one for a&p. and that’s a key class for nursing and biology majors that we never worked with before on my campus. lots of my colleagues have made copies of my tutorials and tailored them to the classes they’re working with or created their own from scratch. and we’ve gotten a lot of positive feedback from faculty, which really felt good during spring term when i know i was working myself to the bone. since giving this presentation, i learned from my colleagues in anthropology that they actually used my work as the basis of their annual assessment project (which every academic unit has to do). they used a normed rubric to assess student papers in anthropology and compared the papers of students who were in sections in which i was embedded (where they had access to the tutorial) to students in sections where they did not have an embedded librarian or a tutorial. they found that students in the class sections in which i was involved had a mean score of / and students in other classes had a mean score of / . that is significant!!! i am so grateful that my liaison area did this project that so validates my own work. here’s an excerpt from one email i received from an anatomy and physiology instructor: “i just wanted to follow up and say that the library assignment was a huge success! i’ve never had so many students actually complete this correctly with peer-reviewed sources in correct citation format. this is a great tool.” at the end of a term where i felt beyond worked to the bone, that was just the sort of encouragement i needed. i made copies of a few other tutorials i’ve created so others can access them: finding sources for your reading or writing assignment – make a copy finding articles for your social sciences research assignment – make a copy evaluating sources tutorial – make a copy biology library assignment – make a copy journal of web librarianship: the culture of digital scholarship in academic libraries . hugh rundle: library map part - why this weekend past i ran the generous & open galleries, libraries, archives & museums (go glam) miniconf at linuxconf.au, with bonnie wildie. being a completely online conference this year, we had an increased pool of people who could attend and also who could speak, and managed to put together what i think was a really great program. i certainly learned a lot from all our speakers, and i'll probably share some thoughts on the talks and projects later this year. i also gave a short talk, about my new library map project and some thoughts on generosity in providing open data. unfortunately, alissa is completely right about my talk. the tone was wrong. i spoke about the wrong things and in the wrong way. it was an ungenerous talk on the virtues of generosity. i allowed my frustration at underfunded government bureaucracies and my anxiety about the prospect of giving a "technical" talk that "wasn't technical enough" for lca to overwhelm the better angels of my nature. i won't be sharing the video of my own talk when it becomes available, but here is a short clip of me not long after i delivered it: via giphy so i'm trying again. in this post i'll outline the basic concepts, and the why of library map - why i wanted to make it, and why i made the architecture and design choices i've made. in the next post, i'll outline how i built it - some nuts and bolts of which code is used where (and also, to some extent, why). you may be interested in one, or the other, or neither post 🙂. library map the library map is a map of libraries in australia and its external territories. there are three 'layers' to the map: libraries the libraries layer shows every public library in australia, plus an indicative m radius around it. also mapped on additional overlays are state and national libraries, indigenous knowledge centres, and most still-operating mechanics institutes. rules the rules layer has two overlays. the fines overlay colour-codes each library service area according to whether they charge overdue fines for everyone, only for adults, or not at all. the loan periods overlay uses patterns (mostly stripes) to indicate the standard loan period in weeks ( , , , or as it turns out). library management software the library management software layer works basically the same as the rules layer, except it colour codes library services according to which library management system (a.k.a intergrated library system) they use. what is the library map for? i've wanted something like this map at various times in the past. there is a fair amount of information around at the regional and state level about loan periods, or fine regimes, and even library management systems. but a lot of this is in people's heads, or in lists within pdf documents. i'm not sure i'd call myself a 'visual learner' but sometimes it is much clearer to see something mapped out visually than to read it in a table. the intended audience for the map is actually a little bit "inside baseball". i'm not trying to build a real-time guide for library users to find things like current opening hours. google maps does a fine job of that, and i'm not sure a dedicated site for every public library but only libraries is a particularly useful tool. it would also be a nightmare to maintain. the site ultimately exists because i wanted to see if i could do it, but i had — broadly — two specific use cases in mind: mapping library management systems to visualise network effects my talk at lca was called who else is using it? — in reference to a question library managers often ask when confronting a suggestion to use a particular technology, especially something major like a library management system. this is understandable — it's reassuring to know that one's peers have made similar decisions ("nobody gets fired for buying ibm"), but there are also genuine advantages to having a network of fellow users you can talk to about shared problems or desired features. i was interested in whether these sorts of networks and aggregate purchasing decisions might be visible if they were mapped out, in a different way to what might be clear from a list or table. especially at a national level — i suspected there were strong trends within states in contrasts between them, but didn't have a really clear picture. the state library of queensland was invaluable in this regard, because they have a list of every library service in the state and which library management system they use. when visiting library service websites it turned out that identifying the lms was often the easiest piece of data to find — much easier than finding out whether they charge overdue fines! it turns out there are very strong trends within each state — stronger than i expected — but western australia is a much more fractured and diverse market than i had thought. i also discovered a bunch of library management systems i had never heard of, so that was fun. this layer is the most recent — i only added it today — so there may still be some improvements to be made in terms of how the data is displayed. mapping overdue fines the second thing i wanted to map was whether and how libraries charge overdue fines, but my reason was different. i actually started the map with this layer, as part of a briefing i gave to some incoming victorian local government councillors about what they should know about public libraries. here, the goal is mapping as an advocacy tool, using the peer pressure of "who else is charging it?" to slowly flip libraries to go fine-free. fines for overdue library books are regressive and counter-productive. i have found no compelling or systematic evidence that they have any effect whatsoever on the aggregate behaviour of library users in terms of returning books on time. they disproportionally hurt low income families. they need to go. in victoria there has been a growing movement in the last few years for public libraries to stop charging overdue fines. i wasn't really aware of the situation in other states, but it turns out the whole northern territory has been fine-free for over a decade, and most libraries in queensland seem to also be fine-free. i'm still missing a fair bit of data for other states, especially south and western australia. what i'm hoping the map can be used for (once the data is more complete) is to identify specific libraries that charge fines but are near groups of libraries that don't, and work with the local library networks to encourage the relevant council to see that they are the odd ones out. i've worked in public libraries and know how difficult this argument can be to make from the inside, so this is a tool for activists but also to support library managers to make the case. as if often a problem in libraries, i had to define a few terms and therefore "normalise" some data in order to have it make any sense systematically. so "no fines for children" is defined as any system that has a "younger than" exclusion for library fines or an exclusion for items designated as "children's books". some libraries are fine free for users under , others for those under , some only for children's book loans and so on. on my map they're all the same. the other thing to normalise was the definition of "overdue fine", which you might think is simple but turns out to be complex. in the end i somewhat arbitrarily decided that if there is no fee earlier than days overdue, that is classified as "no overdue fines". some libraries charge a "notice fee" after two weeks (which does count), whilst others send an invoice for the cost of the book after days (which doesn't). colonial mode as the project has progressed, some things have changed, especially how i name things. when i first added the libraries layer, i was only looking at victoria, using the directory of public library services in victoria. this includes mechanics institutes as a separate category, and that seemed like a good idea, so i had two overlays, in different colours. then i figured i should add the national library, and the state libraries, as a separate layer, since they operate quite differently to local public libraries. once i got to queensland, i discovered that the state library of queensland not only provides really good data on public libraries, but also had broadly classified them into three categories: "rlq" for rural libraries queensland, a reciprocal-borrowing arrangement; "ind" for independent library services, and "ikc" for "indigenous knowledge centre". the immediate question for me was whether i would also classify any of these libraries as something different to a "standard" public library. the main thing that distinguishes the rlq network from the "independents" is that it is a reciprocal lending network. in this regard, it's much the same as libraries victoria (formerly the swift consortium), or shorelink. there are other ways that rural libraries in queensland operate differently to urban libraries in queensland, but i don't think these differences make them qualitatively different in terms of their fundamental nature. but what about indigenous knowledge centres? i admit i knew very little about them, and i still only know what i've gleaned from looking at ikc websites. the torres strait island regional council website seems to be fairly representative: our indigenous knowledge centres endeavour to deliver new technology, literacy and learning programs to empower our communities through shared learning experiences. we work with communities to preserve local knowledge and culture and heritage, to keep our culture strong for generations. the big difference between an ikc and a typical public library is that the focus is on preserving local indigenous knowledge and culture, which does happen through books and other library material, but is just as likely to occur through classes and activities such as traditional art and dance. but the more i looked at this difference, the less different it seemed to be. public libraries across the world have begun focussing more on activities and programs in the last two decades, especially in weird countries. public libraries have always delivered new technology, literacy and learning programs. and the ‌directory of public library services in victoria amusingly reports that essentially every library service in victoria claims to specialise in local history. what are public libraries for, if not to "keep our culture strong for generations"? yet it still felt to me that indigenous knowledge centres are operating from a fundamentally different mental model. finally it dawned on me that the word "our" is doing a lot of work in that description. our indigenous knowledge centres, keep our culture strong for generations. i was taken back to a conversation i've had a few times with my friend baruk jacob, who lives in aotearoa but grew up in a minority-ethnicity community in india. baruk maintains that public libraries should stop trying to be universally "inclusive" — that they are fundamentally eurocentric institutions and need to reconcile themselves to staying within that sphere. in this line of thinking, public libraries simply can't serve indigenous and other non-"western" people appropriately as centres of knowledge and culture. i could see where baruk was coming from, but i was troubled by his argument, and the implication that different cultural traditions could never be reconciled. as i struggled to decide whether indigenous knowledge centres were public libraries, or something else, i think i started to understand what baruk meant. i'd been thinking about this back to front. indigenous knowledge centre is a usefully descriptive term. these places are centres for indigenous knowledge. the problem wasn't how to classify ikcs, but rather how to classify the other thing. the activities might be the same, but the our is different. i thought about what a non-indigenous knowledge centre might be. what kind of knowledge does it want to "keep strong for generations"? i thought about all those local history collections full of books about "pioneers" and family histories of "first settlers". if it's not indigenous knowledge, it must be settler knowledge. when i first saw this term being used by aboriginal activists in reference to non-indigenous residents generally, and white australians specifically, i bristled. i mean, sure, the modern culture is hopelessly dismissive of , years of human occupation, culture and knowledge, but how could i be a "settler" when i have five or six generations of australian-born ancestors? but a bit of discomfort is ok, and i have rather hypocritical ideas about other settler-colonial communities. it's exactly the right term to describe the culture most australians live in. so i renamed "public libraries" as "settler knowledge centres". i initially renamed the national & state libraries to "imperial knowledge centres", but later decided it was more accurate to call them "colonial knowledge centres". i also briefly renamed mechanics institutes to worker indoctrination centres, but that's not entirely accurate and i realised i was getting carried away. i wasn't completely oblivious to the fact that this nomenclature could be a bit confusing, so i cheekily created two views: the "general" view which would be the default, and a second view which would appear on clicking "view in white fragility mode". this second mode would show the more familiar names "public libraries" and "national & state libraries". while i was doing some soul searching this morning about my go glam talk, i continued to work on the map. my cheeky joke about "white fragility mode" had made me slightly uncomfortable from the moment i'd created it, but i initially brushed it off as me worrying too much about being controversial. but i realised today that the real problem was that calling it "white fragility mode" sabotages the entire point of the feature. the default language of "settler knowledge centre" and "colonial knowledge centre" sitting next to "indigenous knowledge centre" is intended to invite map users to think about the work these institutions do to normalise certain types of knowledge, and to "other" alternative knowledge systems and lifeworlds. the point is to bring people in to sit with the discomfort that comes from seeing familiar things described in an unfamiliar way. calling it "white fragility mode" isn't inviting, it's smug. it either pushes people away, or invites them to think no more about it because they're already woke enough to get it. so today i changed it to something hopefully more useful. general mode is now called standard mode, and white fragility mode is now called colonial mode. it's the mode of thinking that is colonial, not the reader. flicking to colonial mode is ok if you need the more familiar terms to get your bearings: but hopefully by making it the non-standard view, users of the map are encouraged to think about libraries and about australia in a slightly different way. they don't have to agree that the "standard mode" terminology is better. so that's some background behind why i started building the map and why i made some of the decisions i have about how it works. you can check it out at librarymap.hugh.run and see (most of) the code and data i used to build it on github. next time join me for a walk through how i made it. nick ruest: four fucking years of donald trump nearly four years ago i decided to start collecting tweets to donald trump out of morbid curiosity. if i was a real archivist, i would have planned this out a little bit better, and started collecting on election night in , or inaguration day . i didn’t. using twarc, i started collecting with the filter (streaming) api on may , . that process failed, and i pivoted to using the search api. i dropped that process into a simple bash script, and pointed cron at it to run every days. here’s what the bash script looked like: #!/bin/bash date=`date +"%y_%m_%d"` cd /mnt/vol /data_sets/to_trump/raw /usr/local/bin/twarc search 'to:realdonaldtrump' --log donald_search_$date.log > donald_search_$date.json it’s not beautiful. it’s not perfect. but, it did the job for the most part for almost four years save and except a couple twitter suspensions on accounts that i used for collection, and an absolutely embarassing situtation where i forgot to setup cron correctly on a machine i moved the collecting to for a couple weeks while i was on family leave this past summer. in the end, the collection ran from may , - january , , and collected , , unique tweets; . t of line-delminted json! the final created_at timestamp was wed jan : : + , and the text of that tweet very fittingly reads, “@realdonaldtrump you’re fired!“ the “dehydrated” tweets can be found here. in that dataset i decided to include in a number of derivatives created with twut which, i hope rounds out the dataset. this update is the final update on the dataset. i also started working on some notebooks here where i’ve been trying to explore the dataset a bit more in my limited spare time. i’m hoping to have the time and energy to really dig into this dataset sometime in the future. i’m especially curious at what the leadup to the storming of the united states capitol looks like in the dataset, as well as the sockpuppet frequency. i’m also hopeful that others will explore the dataset and that it’ll be useful in their research. i have a suspicion folks can do a lot smarter, innovative, and creative things with the dataset than i did here, here, here, here, or here. for those who are curious what the tweet volume for the last few months looked like (please note that the dates are utc), check out these bar charts. january is especially fun. - - lucidworks: consider a new application for ai in retail how companies can plan for by weaving ai and machine learning into their digital experiences. the post consider a new application for ai in retail appeared first on lucidworks. mita williams: weeknote ( ) hey. i missed last week’s weeknote. but we are here now. § this week i gave a class on searching scientific literature to a group of biology masters students. while i was making my slides comparing the advanced search capabilities of web of science and scopus, i discovered this weird behaviour of google scholar: a phrase search generated more hits than not. i understand that google scholar performs ‘stemming’ instead of truncation in generating search results but this still makes no sense to me. § new to me: if you belong to an organization that is already a member of crossref, you are eligible to use a similarity check of documents for an additional fee. perhaps this is a service we could provide to our ojs editors. § i’m still working through the canadian journal of academic librarianship special issue on academic libraries and the irrational. long time readers know that i have a fondness for the study of organizational culture and so it should not be too surprising that the first piece i wanted to read was the digital disease in academic libraries. it begins…. though several recent books and articles have been written about change and adaptation in contemporary academic libraries (mossop ; eden ; lewis ), there are few critical examinations of change practices at the organizational level. one example, from which this paper draws its title, is braden cannon’s ( ) the canadian disease, where the term disease is used to explore the trend of amalgamating libraries, archives, and museums into monolithic organizations. though it is centered on the impact of institutional convergence, cannon’s analysis uses an ethical lens to critique the bureaucratic absurdity of combined library-archive-museum structures. this article follows in cannon’s steps, using observations from organizational de-sign and management literature to critique a current trend in the strategic planning processes and structures of contemporary academic libraries. my target is our field’s ongoing obsession with digital transformation beyond the shift from paper-based to electronic resources, examined in a north american context and framed here as the digital disease. i don’t want to spoil the article but i do want to include this zinger of a symptom which is the first of several: if your library’s organizational chart highlights digital forms of existing functions, you might have the digital disease.kris joseph, the digital disease in academic libraries, canadian journal of academic librarianship, vol ( ) ouch. that truth hurts almost as much as this tweet did: jez cope: blogging by hand i wrote the following text on my tablet with a stylus, which was an interesting experience: so, thinking about ways to make writing fun again, what if i were to write some of them by hand? i mean i have a tablet with a pretty nice stylus, so maybe handwriting recognition could work. one major problem, of course, is that my handwriting is awful! i guess i’ll just have to see whether the ocr is good enough to cope… it’s something i’ve been thinking about recently anyway: i enjoy writing with a proper fountain pen, so is there a way that i can have a smooth workflow to digitise handwritten text without just typing it back in by hand? that would probably be preferable to this, which actually seems to work quite well but does lead to my hand tensing up to properly control the stylus on the almost-frictionless glass screen. i’m surprised how well it worked! here’s a sample of the original text: and here’s the result of converting that to text with the built-in handwriting recognition in samsung notes: writing blog posts by hand so, thinking about ways to make writing fun again, what if i were to write some of chum by hand? i mean, i have a toldest winds a pretty nice stylus, so maybe handwriting recognition could work. one major problems, ofcourse, is that my , is awful! iguess i’ll just have to see whattime the ocu is good enough to cope… it’s something i’ve hun tthinking about recently anyway: i enjoy wilting with a proper fountain pion, soischeme a way that i can have a smooch workflow to digitise handwritten text without just typing it back in by hand? that wouldprobally be preferableto this, which actually scams to work quito wall but doers load to my hand tensing up to properly couldthe stylus once almost-frictionlessg lass scream. it’s pretty good! it did require a fair bit of editing though, and i reckon we can do better with a model that’s properly trained on a large enough sample of my own handwriting. tara robertson: diversity, equity and inclusion core competencies: get cross functional projects done (part of ) photo from wocintech chat, cc-by this is the last post in a weeklong series exploring dei professional competencies. again, i believe the five key competencies for dei professionals are: be strategic translate academic research into action and measure the impact of initiatives meet people where they are at and help them move to be more inclusive influence others get cross functional projects done. yesterday’s post was about influencing others. this post will explore getting cross functional projects done. i’ll also share some other dei career resources. great ideas without action are totally meaningless. as a dei leader you’ll be working across departments and functions to get stuff done. strong project management skills and collaboration are key in making change to existing processes and developing new ways of doing things. here’s two examples to illustrate this competency. one of my first projects at mozilla was working with people ops and a tableau expert in it to build a dashboard to track our diversity metrics, which was more difficult and time consuming than i first thought. when i started the project was off the rails, so i suggested we restart by introducing ourselves, what we thought we brought to the table and then developed a rasci for the project. with these foundations in place we shifted us to be a very effective team. we completed the project and became friends. having a dashboard for diversity metrics was important as leaders owned accountability goals and needed to know how they were doing. engineers started mozilla’s first mentorship program. i joined the team and was the only non-technical person and marvelled at some of the skills and ways of thinking that the others brought. it was one of those wonderful experiences where we were more than the sum of our parts. we were a small group of people with different backgrounds, doing different jobs, at various job levels and we were able to stand up and support a mentorship program for about people. i credit the leadership of melissa o’connor, senior manager of data operations. she often said “tell me what i’m missing here” to invite different options and ran the most efficient meetings i’ve ever attended in my life. great ideas without action are totally meaningless. turning thoughts into actions as a leader in dei is a necessary art–to get things done you’ll need to effectively collaborate with people at different levels and in different functions. additional resources on dei careers i’m excited to be one of the panelists for andrea tatum’s dei careers panel tomorrow, january . the event is sold out but she’ll be simulcasting live on youtube at january at am pacific. andrea also introduced me to russell reynold’s list of competencies of a chief diversity officer. aubrey blanche’s post how can i get a job in d&i? starts by trying to talk the reader out of going into this line of work then gets into five key areas of expertise. dr. janice gassam’s dirty diversity podcast has an episode where she interviews lambert odeh, diversity and inclusion manager at olo inc. on how to land a career in dei. the post diversity, equity and inclusion core competencies: get cross functional projects done (part of ) appeared first on tara robertson consulting. open knowledge foundation: how to run your open data day event online in for open data day on saturday th march, the open knowledge foundation is offering support and funding for in-person and online events anywhere in the world via our mini-grant scheme. open data day normally sees thousands of people getting together at hundreds of events all over the world to celebrate and use open data in their communities but this year has not been a normal year. with many countries still under lockdown or restricted conditions due to the covid- pandemic, we recognise that many people will need to celebrate open data day by hosting online events rather than getting together for in-person gatherings. to support the running of events, anyone can apply to our mini-grant scheme to receive $ usd towards the running of your open data day event whether it takes place in-person or online. applications must be submitted before pm gmt on friday th february by filling out this form. if you’re applying for a mini-grant for an online event, we will accept applications where the funds are allocated to cover any of the following costs: fees for online tools needed to help with the running of your event essential equipment needed to help with the running of your event reimbursing speakers or participants for mobile data costs incurred during event paying for the printing and posting of physical materials to event participants other costs associated with running the event it might feel challenging to plan a great online event if you are used to running events in the real world. but many people and organisations have overcome these challenges this year, and there are many tools that can help you plan your event. here are some tips and tools that we use for remote events that we think will help with your preparations. open knowledge foundation is a remote working organisation with our team spread around the world. we use zoom, google meet or slack to host our internal and external video meetings and rely on google docs, github, gitter and discourse to allow us to share documents and talk in real-time. many of these tools are free and easy to set up. two members of our team are also on the organisation team of csv,conf, an annual community conference for data makers which usually hosts several hundred people for a two-day event. for csv,conf,v in may , the team decided to make their event online-only and it proved to be a great success thanks to lots of planning and the use of good online tools. read this post – https://csvconf.com/ /going-online – to learn more about how the team organised their first virtual conference including guidance about the pros and cons of using tools like crowdcast, zenodo, zoom and spatial chat for public events. other organisations – including the center for scientific collaboration and community engagement and the mozilla festival team – have also shared their guidebooks and processes for planning virtual events. we hope some of these resources will help you in your planning. if you have any further questions relating to an open data day mini-grant application, please email opendataday@okfn.org. ed summers: trump's tweets tldr: trump’s tweets are gone from twitter.com but still exist spectrally in various states all over the web. after profiting off of their distribution twitter now have a responsibility to provide meaningful access to the trump tweets as a read only archive. this post is also published on the documenting the now medium where you can comment, if the mood takes you. so trump’s twitter account is gone. finally. it’s strange to have had to wait until the waning days of his presidency to achieve this very small and simple act of holding him accountable to twitter’s community guidelines…just like any other user of the platform. better late than never, especially since his misinformation and lies can continue to spread after he has left office. but isn’t it painful to imagine what the last four years (or more) could have looked like if twitter and the media at large had recognized their responsibility and acted sooner? when twitter suspended trump’s account they didn’t simply freeze it and prevent him from sending more hateful messages. they flipped a switch that made all the tweets he has ever sent disappear from the web. these are tweets that had real material consequences in the world. as despicable as trump’s utterances have been, a complete and authentic record of them having existed is important for the history books, and for holding him to account. twitter’s suspension of donald trump’s account has also removed all of his thousands of tweets sent over the years. i personally find it useful as a reporter to be able to search through his tweets. they are an important part of the historical record. where do they live now? — olivia nuzzi (@olivianuzzi) january , where indeed? one hopes that they will end up in the national archives (more on that in a moment). but depending on how you look at it, they are everywhere. twitter removed trump’s tweets from public view at twitter.com. but fortunately, as shawn jones notes, embedded tweets like the one above persist the tweet text into the html document itself. when a tweet is deleted from twitter.com the text stays behind elsewhere on the web like a residue, as evidence (that can be faked) of what was said and when. it’s difficult to say whether this graceful degradation was an intentional design decision to make their content more resilient, or it was simply a function of twitter wanting their content to begin rendering before their javascript had loaded and had a chance to emboss the page. but design intent isn’t really what matters here. what does matter is the way this form of social media content degrades in the web commons. kari kraus calls this process “spectral wear”, where digital media “help mitigate privacy and surveillance concerns through figurative rather than quantitative displays, reflect and document patterns of use, and promote an ethics of care.” (kraus, ). this spectral wear is a direct result of tweet embed practices that twitter itself promulgates while simultaneously forbidding it developer terms of service: if twitter content is deleted, gains protected status, or is otherwise suspended, withheld, modified, or removed from the twitter applications (including removal of location information), you will make all reasonable efforts to delete or modify such twitter content (as applicable) as soon as possible. fortunately for history there has probably never been a more heavily copied social media content than donald trump’s tweets. we aren’t immediately dependent on twitter.com to make this content available because of the other other places on the web where it exists. what does this copying activity look like? i intentionally used copied instead of archived above because the various representations of trump’s tweets vary in terms of their coverage, and how they are being cared for. given their complicity in bringing trump’s messages of division and hatred to a worldwide audience, while profiting off of them, twitter now have a responsibility to provide as best a representation of this record for the public, and for history. we know that the trump administration have been collecting the @realdonaldtrump twitter account, and plan to make it available on the web as part of their responsibilities under the presidential records act: the national archives will receive, preserve, and provide public access to all official trump administration social media content, including deleted posts from @realdonaldtrump and @potus. the white house has been using an archiving tool that captures and preserves all content, in accordance with the presidential records act and in consultation with national archives officials. these records will be turned over to the national archives beginning on january , , and the president’s accounts will then be made available online at nara’s newly established trumplibrary.gov website. nara is the logical place for these records to go. but it is unclear what shape these archival records will take. sure the library of congress has (or had) it’s twitter archive. it’s not at all clear if they are still adding to it. but even if they are lc probably hasn’t felt obligated to collect the records of an official from the executive branch, since they are firmly lodged in the legislative. then again they collect gifs so, maybe? reading between the lines it appears that a third party service is being used to collect the social media content: possibly one of the several e-discovery tools like archivesocial or hanzo. it also looks like the trump administration themselves have entered into this contract, and at the end of its term (i.e. now) will extract their data and deliver it to nara. given their past behavior it’s not difficult to imagine the trump administration not living up to this agreement in substantial ways. this current process is a slight departure from the approach taken by the obama administration. obama initiated a process where platforms [migrate official accounts] to new accounts that were then managed going forward by nara (acker & kriesberg, ). we can see that this practice is being used again on january , when biden became president. but what is different is that barack obama retained ownership of his personal account @barackobama, which he continues to use.nara has announced that they will be archiving trump’s now deleted (or hidden) personal account. a number of trump administration officials, including president trump, used personal accounts when conducting government business. the national archives will make the social media content from those designated accounts publicly available as soon as possible. the question remains, what representation should be used, and what is twitter’s role in providing it? meanwhile there are online collections like the trump archive, the new york times’ complete list of trump’s twitter insults, propublica’s politwoops and countless github repositories of data which have collected trump’s tweets. these tweets are used in a multitude of ways including things as absurd as a source for conducting trades on the stock market. but seeing these tweets as they appeared in the browser, with associated metrics and comments is important. of course you can go view the account in the wayback machine and browse around. but what if we wanted a list of all the trump tweets? how many times were these tweets actually archived? how complete is the list? after some experiments with the internet archive’s api it’s possible to get a picture of how the tweets from the @realdonaldtrump account have been archived there. there are a few wrinkles because a given tweet can have many different url forms (e.g. tracking parameters in the url query string). in addition just because there was a request to archive a url for something that looks like a realdonaldtrump tweet url doesn’t mean it resulted in a successful response. success here means a ok from twitter.com when resolving the url. factoring these issues into the analysis it appears the wayback machine contains (at least) , , snapshots of trump’s tweets. https://twittter.com/realdonaldtrump/status/{tweet-id} of these millions of snapshots there appear to be , unique tweets. this roughly correlates with the k total tweets suggested by the last profile snapshots of the account. the maximum number of times in one day that his tweets were archived was , times on february , . here’s what the archive snapshots of trump’s tweets look like over time (snapshots per week). it is relatively easy to use the csv export from the [trump archive] project to see what tweets they know about that the internet archive does not and vice-versa (for the details see the jupyter notebook and sqlite database here). it looks like there are tweet ids in the trump archive that are missing from the internet archive. but further examination shows that many of these are retweets, which in twitter’s web interface, have sometimes redirected back to the original tweet. removing these retweets to specifically look at trump’s own tweets there are only tweets in the trump archive that are missing from the internet archive. of these are in fact retweets that have been miscategorized by the trump archive. one of the three is this one, which is identified in the trump archive as deleted, and wasn’t collected quick enough by the internet archive before it was deleted: roger stone was targeted by an illegal witch hunt tha never should have taken place. it is the other side that are criminals, including the fact that biden and obama illegally spied on my campaign - and got caught!" sure enough, over at the politwoops project you can see that this tweet was deleted seconds after it was sent: flipping the table it’s also possible to look at what tweets are in the internet archive but not in the trump archive. it turns out that there are , tweet identifiers in the wayback machine for trump’s tweets which do not appear in the trump archive. looking a bit closer we can see that some are clearly wrong, because the id itself is too small a number, or too large. and then looking at some of the snapshots it appears that they often don’t resolve, and simply display a “something went wrong” message: yes, something definitely went wrong (in more ways than one). just spot checking a few there also appear to be some legit tweets in the wayback that are not in the trump archive like this one: notice how the media will not play there? it would take some heavy manual curation work to sort through these tweet ids to see which ones are legit, and which ones aren’t. but if you are interested here’s an editable google sheet. finally, here is a list of the top ten archived (at internet archive) tweets. the counts here reflect all the variations for a given tweet url. so they will very likely not match the count you see in the wayback machine, which is for the specific url (no query paramters). thank you alabama! #trump #supertuesday , make america great again! , thank you georgia!#supertuesday #trump , such a beautiful and important evening! the forgotten man and woman will never be forgotten again. we will all come together as never before , watching the returns at : pm. #electionnight #maga🇺🇸 https://t.co/hfujerzbod , #electionday https://t.co/mxraxyntjy https://t.co/fzhoncih , i will be watching the election results from trump tower in manhattan with my family and friends. very exciting! , i love you north carolina- thank you for your amazing support! get out and https://t.co/hfihperfgz tomorrow!watch:… https://t.co/jzzfquznyh , still time to #votetrump! #ivoted #electionnight https://t.co/uztyay ba , watching my beautiful wife, melania, speak about our love of country and family. we will make you all very proud…. https://t.co/dikmsntlc the point of this rambling data spelunking, if you’ve made it this far, is to highlight the degree to which trump’s tweets have been archived (or collected), and how the completeness and quality of those representations is very fluid and difficult to ascertain. hopefully twitter is working with nara to provide as complete a picture as possible of what trump said on twitter. as much as we would like to forget, we must not. references acker, a., & kriesberg, a. ( ). tweets may be archived: civic engagement, digital preservation and obama white house social media data. proceedings of the association for information science and technology, ( ), – . kraus, k. ( ). the care of enchanted things. in m. k. gold & l. f. klein (eds.), debates in the digital humanities . retrieved from https://www.jstor.org/stable/ . /j.ctvg hk. lucidworks: lucidworks announces updated integration with drupal updated fusion integration with drupal provides easier setup and additional monitoring. the post lucidworks announces updated integration with drupal appeared first on lucidworks. opendri open data for resilience initiative projects news resources about opendri brings the philosophies and practices of the global open data movement to the challenges of reducing vulnerability and building resilience to natural hazards and the impacts of climate change across the globe. sharing datacollecting datausing data opendri in sec explore projects news tout le monde a un rôle à jouer : comment les différents acteurs contribuent à accroître la résilience urbaine au cameroun auteurs : bontje zangerling, vivien deparday, michel tchotsoua, mira gupta, et tamilwai kolowa article original à world bank blogs. (english) photo: michel tchotsoua comme de nombreux centres urbains en afrique, la ville de ngaoundéré, au cameroun, a connu un accroissement rapide de sa population, qui est passée d’environ habitants en à près de en . largement non planifiée, cette croissance urbaine s’est souvent traduite par l’occupation de zones exposées aux inondations et aux chutes de pierres, notamment par les migrants ruraux. de telles catastrophes font régulièrement des ravages au sein de la population et, entraînent une exposition accrue aux maladies, aux pénuries alimentaires ainsi qu’une vulnérabilité financière. ce lien entre une croissance urbaine rapide et non planifiée et une exposition accrue aux risques naturels est particulièrement problématique pour les gouvernements locaux, qui manquent de données pour élaborer des réponses politiques précises et efficaces. comment faire face à un problème sans pouvoir le mesurer ? au cours des deux dernières années, les contributions d’un large éventail d’acteurs ont permis d’accroître la disponibilité des données et de produire un atlas des risques. ce dernier est désormais utilisé par la communauté urbaine de ngaoundéré (cun) pour informer la gestion et la planification urbaines. combler le déficit d’information en , ngaoundéré a été sélectionnée comme l’une des villes prenant part à l’initiative open cities africa, lancée par le fonds mondial de prévention des catastrophes et de relèvement (gfdrr). son objectif ? appuyer la collecte d’informations sur les risques à travers l’engagement des citoyens et favoriser le développement d’outils pour la prise de décision locale. À ngaoundéré, cette approche a permis à l’équipe de cartographier plus de km² de zone urbaine, en combinant les contributions des résidents locaux avec des données de la municipalité, et de nouvelles images de drones. ce travail est actuellement disponible en ligne, via l’atlas des risques et dans openstreetmap. l’initiative a été coordonnée avec le projet des villes inclusives et résilientes du cameroun (pdvir), financé par la banque mondiale et mis en œuvre par le ministère de l’habitat et du développement urbain. photo: mohamadou arabo l’engagement local permet à la communauté de se sentir concerner et d’agir en s’appropriant les actions locales dès le début, le projet open cities ngaoundéré a donné la priorité à l’engagement des différents acteurs locaux. son principal exécutant, l’association pour la cartographie de la gestion des ressources (acager), a consulté groupes locaux, dont les ministères, les organisations de la société civile et des associations communautaires opérant dans des zones à risque. les membres des communautés, qui n’avaient jamais été impliqués dans la gestion urbaine, sont désormais plus conscients des défis. de plus, ils s’approprient et assument la responsabilité des actions déployées dans les zones où ils travaillent et vivent. nous avons pu leur montrer le nombre de maisons dans chaque quartier, celles qui sont à risque et leur expliquer les mesures qui peuvent être prises. les communautés organisent maintenant des « jeudi propre » : des rassemblements hebdomadaires pour ramasser les ordures, afin d’éviter que les systèmes de drainage ne se bouchent. elles ont également demandé que des drones soient utilisés pour surveiller régulièrement les nouvelles constructions dans les zones inondables. comment les étudiants ont revitalisé la collecte de données municipales ? en menant des enquêtes auprès des ménages et en sensibilisant les habitants à la cartographie, les étudiants de l’université de ngaoundéré ont constitué la pièce maîtresse du dispositif de collecte de données géospatiales. ce travail a permis de renforcer leurs compétences techniques et leur a donné l’occasion de contribuer concrètement aux efforts de développement de la communauté. le recours aux étudiants a été particulièrement efficace, du fait de leur connaissance de la langue locale, le fulfuldé, et des normes sociales et culturelles des ménages. les contributions de ces jeunes cartographes ne sont pas passées inaperçues aux yeux des fonctionnaires locaux. photo: gaëlle nodjignemal goltobs. ahmad barngawi mohammad, le responsable de l’urbanisme, de l’architecture et des permis de construire à la cun, nous a dit avoir été impressionné par l’engagement des jeunes. alors que pendant très longtemps le projet n’avait pas réussi à rassembler ces données, il constate que maintenant le projet prend de l’ampleur grâce à son approche facile et accessible à tous. de meilleures données pour une prise de décision ciblée les données actualisées ont aidé les autorités locales à démontrer les difficultés rencontrées à ngaoundéré, et à plaider avec succès auprès du gouvernement central afin qu’il investisse dans l’atténuation des risques, en ajustant par exemple les voies d’eau pour améliorer le ruissellement des eaux de pluie. la cun envisage également d’utiliser ces données à des fins plus larges, telles que le développement d’un système de recettes pour gérer les impôts fonciers. photo: gabriel amougou s’appuyant sur le succès rencontré à ngaoundéré, un projet similaire de cartographie des zones à risque a été lancé dans la capitale yaoundé. le département d’État américain a également soutenu la cartographie participative à douala, à travers son programme secondary cities. il ne fait aucun doute que le succès du projet open cities ngaoundéré tient à son approche collaborative qui a su mettre en valeur et impliquer de nombreux acteurs : lorsqu’il s’agit de gestion et de résilience urbaines, tout le monde a un rôle à jouer. note : open cities africa (a) est financé par l’union européenne à travers le programme acp-ue africa disaster financing et géré par l’équipe gpurl afrique. feb everybody has a role: how stakeholders are contributing to increase urban resilience in cameroon by bontje zangerling, vivien deparday, michel tchotsoua, mira gupta, and tamilwai kolowa this article was originally published on world bank blogs. (français) new housing construction at risk of rock falls in ngaoundéré. photo: michel tchotsoua like many urban areas in africa, the city of ngaoundéré, cameroon has seen a rapid increase in its population, from roughly , in to almost , in . this urban growth has been largely unplanned, with rural migrants often occupying areas within flood plains or along mountain slopes, vulnerable to flooding and rock falls. such disasters regularly wreak havoc on citizens’ lives and lead to increased exposure to disease, food shortages, and financial vulnerabilities. this nexus of rapid, unplanned urban growth and increased exposure to natural hazards is especially problematic for local governments who lack data to formulate accurate and efficient policy responses. how do you address a problem you cannot measure? over the last two years, inputs from a diverse range of stakeholders have increased the availability of data and led to the production of a tangible risk atlas, which the ngaoundéré city council (ncc) is now using to inform urban management and planning. closing the information gap in , ngaoundéré was selected as one of cities to participate in open cities africa, an initiative of the global facility for disaster reduction and recovery (gfdrr) that supports the collection of open-source risk information through citizen engagement and the development of data products to support local decision-making. in ngaoundéré, this approach allowed the team to map over km² of urban area combining inputs from local residents, with data from the municipality and new drone imagery. all this is now available online through the risk atlas and in openstreetmap. the initiative was coordinated with the world bank-financed cameroon inclusive and resilient cities project (circp) that is being implemented by the ministry of housing and urban development (minhdu). community members providing feedback on draft maps. photo: mohamadou arabo local engagement leads to community ownership the open cities ngaoundéré project prioritized stakeholder engagement from the beginning. its lead implementer, the association for resource management mapping (acager), consulted with local groups, including government departments, civil society organizations and community associations operating in risk-prone areas. community members who had never been engaged in urban management are now more aware of the challenges and take ownership and responsibility for actions that affect the areas where they work and live. we were able to show them the number of houses in each neighborhood, which ones were at risk and explain what actions can be taken. communities now organize weekly “jeudi propre” (clean thursday) gatherings to pick up trash, to prevent drainage systems from clogging, and they have requested that drone flights be used to regularly monitor new construction in flood-prone areas. how students revived municipal data collection at the heart of the geospatial data collection were students from the university of ngaoundéré, who carried out household surveys and facilitated the mapping with residents. this role strengthened their technical skills and provided them with a hands-on opportunity to contribute to community development efforts. the use of students was especially effective because they spoke the local language, fulfuldé, and understood the social and cultural norms around approaching households. community members providing feedback on draft mapa student conducting household surveys. photo: gaëlle nodjignemal goltobs. photo: mohamadou arabo the contributions of these young mappers did not go unnoticed by local government officials. ahmad barngawi mohammad, the urban planning, architecture and building permit manager at the ncc told us that the commitment of the young people was the thing that impressed him most. while for a very long time the project couldn’t gather this data, today, the project is gaining momentum thanks to its easy and accessible approach. better data for targeted decision-making the updated data has helped local authorities demonstrate the challenges in ngaoundéré and successfully advocate for central government funding to invest in risk mitigation, for instance in adjusting waterways to improve stormwater runoff. the ncc is also exploring the use of the data for broader purposes, such as the development of an own-source revenue system to manage property taxes. as a result of improved data, the ncc mobilized funding to upgrade its drainage infrastructure. photo: gabriel amougou based on the success in ngaoundéré, a similar project to map risk-prone areas was launched in cameroon’s capital, yaoundé. the u.s. department of state also supported participatory mapping in douala, through its secondary cities program. the accomplishments of the open cities ngaoundéré project can be attributed to its collaborative approach that recognized the value of numerous stakeholders: when it comes to urban management and resilience, everybody has a role. many local stakeholders benefited from open cities africa activities in ngaoundéré. note: open cities africa is financed by the eu-funded acp-eu africa disaster financing program. feb gfdrr supports data for resilience at open data day every year, the global open data community gathers for hundreds of local events for the annual open data day, organized by open knowledge foundation. groups host hackathons, workshops, webinars and more to show the benefits of open data in their local community, and to encourage the adoption of open data policies in government, business and civil society. the next open data day is saturday, march th, . gfdrr is pleased to announce its support for open data day mini-grants to provide funding for local events—online or in-person—showcasing open data in their communities. the deadline to submit your mini-grant application is midday gmt on friday, february th, . apply today! lilongwe university of agriculture and natural resource students look at a map of the location of dumpsites around their campus at open data day . image from the open data day report. the gfdrr open data for resilience initiative (opendri) will be supporting the environmental data track of mini-grants focused on data for resilience. environmental data events use open data to illustrate the urgency of the climate emergency and spur people into action to take a stand or make changes in their lives to help the world become more environmentally sustainable. last year’s events were impressive; highlights include a youth-led effort to map waste dumping sites in malawi and an interactive exploration of climate change public data platforms in costa rica, among many more. if you need inspiration for your event using data for resilience, useful resources from the gfdrr labs include: opendri, open cities project, thinkhazard, open data for resilience index and the risk data library. if you have started planning your open data day event already, please add it to the global map on the open data day website using this form. need more information? if you have any questions, you can reach out to the open knowledge foundation team by emailing opendataday@okfn.org or on twitter via @okfn. there’s also the open data day google group where you can connect with others interested in taking part, share ideas for your event or ask for help. feb quand résilience des villes rime avec épanouissement professionnel des jeunes auteurs : dina ranarifidy et tamilwai kolowa article original à world bank blogs. (english) quelques membres de l’équipe de jeunes congolais recrutés pour cartographier les zones vulnérables de brazzaville en utilisant la technologie street view. ici en compagnie des experts en analyse de données géospatiales de la société mindearth. © mindearth lorsque nous avons lancé, en , les activités villes ouvertes (open cities) (a) en république du congo, notre principal objectif était d’aider les municipalités de brazzaville et de pointe-noire à mieux se prémunir contre les risques d’inondation et d’érosion en milieu urbain. nous étions loin d’imaginer alors l’impact que ce projet aurait sur la jeunesse congolaise. en plus de renforcer la résilience des villes aux chocs climatiques, le projet villes ouvertes a suscité chez les jeunes congolais une véritable passion pour l’urbanisme et la cartographie participative tout en leur donnant l’occasion d’apprendre et de trouver des débouchés professionnels . nous avons eu le privilège d’assister à cet élan d’enthousiasme et d’observer les changements dans ces deux villes. jusque-là, les données urbanistiques étaient rares, souvent obsolètes et difficilement accessibles. elles étaient de ce fait une denrée coûteuse, ce qui contrariait les efforts pour planifier durablement et efficacement l’espace urbain. lorsque les étudiants congolais ont appris que le projet villes ouvertes mobilisait les ressources locales pour pallier cette pénurie de données, en promouvant ainsi la production d’informations gratuites, accessibles et collaboratives, ils y ont vu une opportunité. ces jeunes ont réalisé qu’ils pouvaient collecter des données inaccessibles autrement, les partager librement et les analyser afin d’améliorer la situation de leurs quartiers, et se sont investis avec ferveur et fierté dans ce défi. À pointe-noire, les mapathons (a) organisés régulièrement dans le cadre des activités villes ouvertes ont donné naissance à une communauté de cartographes bénévoles et passionnés . des exercices qui ont également renforcé l’engagement citoyen et permis aux jeunes d’acquérir de nouvelles compétences. christian massama ganga, fraîchement diplômé de l’université marien ngouabi de brazzaville, nous a confié apprécier de pouvoir utiliser des outils en libre accès pour recueillir des données sur son quartier. grâce à cette initiative, il s’est plongé dans la cartographie ouverte et envisage désormais de mettre à profit ses nouvelles connaissances pour décrocher son doctorat. fort de ses compétences cartographiques certifiées, il s’est porté candidat pour faire partie des équipes chargées du recensement national. À terme, christian rêve d’une carte couvrant « tout le congo » pour que les habitants et les touristes puissent s’orienter facilement. la municipalité de pointe-noire a elle aussi tiré directement parti de cette collaboration : ayant recruté certains de ces cartographes enthousiastes, elle a renforcé les liens entre une jeunesse dynamique et les autorités traditionnelles, pour le bien de la ville. et c’est là où les universités jouent notamment un rôle décisif de passerelle. une équipe de jeunes cartographes équipés de leurs appareils photo arpente des quartiers non cartographiés de brazzaville, au congo. © mindearth rapprocher les étudiants des opportunités professionnelles À pointe-noire, l’université catholique d’afrique centrale et l’institut catholique des arts et métiers (ucac-icam), qui possèdent aussi un campus à douala et à yaoundé au cameroun, servent de tremplin professionnel aux ingénieurs et techniciens. partenaire de l’initiative open cities africa (a), l’ucac-icam forme ses étudiants à la collecte et l’analyse de données et les fait participer directement à ces opérations. en intégrant les activités de l’initiative dans le cursus universitaire, l’institut élargit les options professionnelles proposées aux étudiants. promouvoir l’aménagement urbain grâce à un vivier de cartographes cette nouvelle cohorte de cartographes compétents et maîtrisant les nouvelles technologies constitue un précieux atout pour les municipalités lors des projets collaboratifs. leurs compétences ont également été mobilisées pour accompagner les activités de réhabilitation de quartiers précaires menés au titre du projet de développement urbain et de restructuration des quartiers précaires (durquap) (a) financé par la banque mondiale. certains étudiants ont été embauchés par le durquap pour collecter et analyser les données d’appui aux investissements d’infrastructure (pavage et ouvrages de drainage) financés au titre du projet. plus récemment, des cartographes locaux à brazzaville et pointe-noire ont pris part à la collecte, à l’analyse et à la classification de données en vue d’expérimenter un nouvel exercice de cartographie fondé sur la technologie street view et destiné à évaluer la vulnérabilité des quartiers et les risques d’inondation. au congo, le projet villes ouvertes a su s’appuyer sur les talents et les compétences de jeunes gens passionnés et, pour ces étudiants prometteurs comme pour les villes auxquelles ils consacrent leur énergie, le meilleur est à venir ! dec latest tweets a twitter list by gfdrr view all news projects open cities africa carried out in cities in sub-saharan africa to engage local government, civil society, and the private sector to develop the information infrastructures necessary to meet st century urban resilience challenges. the project is implemented through a unique partnership between gfdrr and the world bank, city governments across the continent, and a partner community comprised of regional scientific and technology organizations, development partners, and technology companies. website counterpartscities opencitiesproject.org national and provincial ministries, municipal offices and local development committees accra, ghana antananarivo, madagascar brazzaville, republic of congo kampala, uganda kinshasa, democratic republic of congo monrovia, liberia ngaoundÉrÉ, cameroon niamey, niger pointe-noire, republic of congo saint-louis, senegal seychelles zanzibar city, tanzania overview as urban populations and vulnerability grow, managing urban growth in a way that fosters cities’ resilience to natural hazards and the impacts of climate change becomes a greater challenge that requires detailed, up-to-date geographic data of the built environment. addressing this challenge requires innovative, open, and dynamic data collection and mapping processes that support management of urban growth and disaster risk. success is often contingent on local capacities and networks to maintain and utilize risk information, enabling policy environments to support effective data management and sharing, and targeted tools that can help translate data into meaningful action. building on the success of the global open data for resilience initiative, its work on open cities projects in south asia, and gfdrr’s code for resilience, open cities africa is carried out in cities in sub-saharan africa to engage local government, civil society, and the private sector to develop the information infrastructures necessary to meet st century urban resilience challenges. following an application process, a small team of mappers, technologists, designers, and risk experts in each of the selected cities receive funding, targeted training, technical support, and mentorship throughout the year of work to: i) create and/or compile open spatial data on the built environment, critical infrastructure, and natural hazards; ii) develop targeted systems and tools to assist key stakeholders to utilize risk information; and iii) support local capacity-building and institutional development necessary for designing and implementing evidence-driven urban resilience interventions. phases of implementation . plan and assess in the first phase, open cities teams establish what data already exists and its openness, relevance and value. project target area and data to collect are finalized. this phase is also when teams identify project partners and stakeholders to ensure that efforts are a participatory process. at the open cities kick off meeting, teams meet with open cities leadership and the other open cities teams in their cohort and receive training on project components. . map in this second phase, teams roll out the findings and data capture strategy developed in the first phase to address critical data gaps relevant to their specific problem statements. on the ground, teams coordinate field data collection according to the approach developed and agreed upon in consultation with project stakeholders. depending on needs, tools for data collection may include smartphones or tablets, drones for the collection of high resolution imagery, or handheld gps. as the project team is training team members to collect data for the project, efforts are made to develop, and/or strengthen the local openstreetmap community within the selected city working in partnership with local stakeholders. project teams may hold trainings, mapathons, or community town halls in coordination with a local university, ngo or government counterparts. . design in this third phase of the project, teams use the data collected in the map phase to design a tool or product to communicate the data to their stakeholders to support decision-making. products vary widely depending on city context and may include a database and visualization tool, an atlas, a map series, or a mobile application. . develop and present in the final phase of the project, teams develop their tools/products and share results with targeted end user populations and other relevant stakeholders. once final products are shared, teams work with project mentors and open cities africa leadership to establish a sustainability plan and to explore opportunities for expansion or extension. this could include convening meetings with the world bank, government counterparts, or the nongovernmental organization and donor communities. it may also include the development of concept notes, proposals or additional user research. learn more more information about the project and team activities can be found on the open cities africa site. collecting data niger in niger, the world bank is supporting the government reduce the vulnerability of populations at risk of flooding, while taking into account the requirements of community development and capacity building of national structures both at central and local level. data sharing platform http://risques-niger.org counterpart pgcr-du (projet de gestion des risques de catastrophes et de développement urbain – disaster risk management and urban development project) number of geonode layers understanding niger’s risks despite its semi-arid climate, niger is regularly stricken by floods that destroy housing, infrastructure and croplands everywhere in the country. while flood damages usually occur in the vicinity of permanent water bodies such as the niger and komadougou rivers, more and more damages and casualties have been reported as linked to intense precipitations and runoff in urban areas. despite the recurrent losses, little is known about the number of people who are living in flood-prone areas or the value of properties at risk. furthermore, the vast majority of stations in the meteorological and hydrological collection network does not have the ability to transmit data in real-time and therefore cannot be fully exploited in emergency situations. collecting data with the support of the world bank, the pgrc-du is supporting the nigerien ministry of water and sanitation to retrofit the hydrometric station network with new water level gauges with real-time data transmission capability. the new gauges will make hydrometric data collection more efficient and more reliable while allowing for a faster detection of flood risk. at the same time, the pgrc-du is funding the collection of critical socio-economic information and building characteristics in all areas of niamey (the capital of niger) that are deemed vulnerable to floods. uavs are being used to acquire high-resolution images of potentially flooded areas that would help better identify buildings characteristics and develop a digital terrain model (dtm) with cm vertical resolution, which will help better predict water movement in the area. sharing data the collected hydrometric data will be available to selected users in an online portal, along with various other data sets from regional and global sources. part of the data collected in niamey is expected to contribute to the openstreetmap project. the rest of the data will be analyzed and converted into vulnerability maps and reports available to the public. using data it is expected that the network of real-time hydrometric stations will be used to feed a flood warning system that will provide authorities a better estimate of flood risk at any given time. the acquired dtm is being used to develop computer models that can simulate flood propagation in the city of niamey and evaluate the effects of existing of planned flood protection infrastructures. finally, the collected socio-economic data combined with flood simulations will provide decision-makers an accurate estimation of flood risk in terms of exposed populations and expected economic damages. collecting data sharing data using data uganda in uganda, the world bank is supporting the government to develop improved access to drought risk related information and quicken the decision of scaling up disaster risk financing (drf) mechanisms counterpart national emergency coordination and operations center (necoc) project overview in the context of the third northern uganda social action fund project (nusaf iii), the world bank is supporting the government of uganda to develop improved access to drought risk related information and quicken the decision of scaling up disaster risk financing (drf) mechanisms. the opendri team is providing technical assistance to uganda’s national emergency coordination and operations center (necoc) in determining requirements for collecting, storing and analyzing satellite data used for monitoring drought conditions. understanding uganda’s risk in recent years uganda has been impacted by drought, with more than % of the population being at risk. the northern sub-region of karamoja is one of the most severely hit, with a consequent increase in food insecurity. currently the government of uganda (gou) faces challenges in the collection and analysis of information upon which they can base a decision to respond and mitigate such risk. without transparent, objective and timely data, times in mobilizing and financing responses can be delayed. collecting data the world bank is supporting gou to strengthen its disaster risk management strategy and response mechanisms. the current engagement looks to develop a more systematic, robust system for collecting, storing and analyzing drought risk related information to enable gou to make more timely decisions. by retrieving satellite data systematically, necoc will be able to analyze current crop and vegetation conditions with historic information, and quickly detect early warning signs of drought. uganda has a vibrant openstreetmap community, which has been mapping the country since . a pilot community mapping project funded by gfdrr with support from the government of belgium, is being conducted in the city of kampala. sharing data the opendri team provides support and advice to gou in developing best practices for sharing and managing risk related information. interoperability of data sources produced by various ministries and non-government organizations is critical to ensure timely access to data by necoc and conduct effective drought risk analysis. a geospatial data sharing platform will be deployed by gou to facilitate exchange of such critical information and adoption of data standards. using data a technical committee, composed of experts from the government and partner organizations, has agreed to use a satellite derived indicator known as normalized difference vegetation index (ndvi) as the primary dataset to inform decisions for triggering the disaster risk financing mechanism. initially the system will be exclusively dedicated to monitoring drought risk in the northern sub-region of karamoja. in the following years, it is expected to expand operations and cover other regions exposed to drought risk, integrating additional data sources which will become accessible thanks to improved data collection strategies and sharing mechanisms. collecting data sharing data using data zanzibar the revolutionary government of zanzibar (rgoz) with the support of the world bank has been developing evidence-based and innovative solutions to better plan, mitigate, and prepare for natural disasters. zanzibar is part of the southwest indian ocean risk assessment and financing initiative (swio rafi) which seeks to address high vulnerability of the southwest indian ocean island states to disaster losses from catastrophes such as cyclones, floods, earthquakes and tsunamis. these threats are exacerbated by the effects of climate change, a growing population and increased economic impacts. data sharing platform project page zan sea facebook page http://zansea-geonode.org www.zanzibarmapping.org https://www.facebook.com/zansea/ understanding zanzibar’s risk zanzibar’s disaster events are mainly related to rainfall, and both severe flooding and droughts have been experienced. sharing data island map: openstreetmap data collected through swio rafi activities will be shared on a geonode. the zansea geonode currently contains maps and layers of geospatial data for zanzibar. collecting data the zanzibar mapping initiative is creating a high resolution map of the islands of zanzibar and pemba, over square km, using low-cost drones instead of satellite images or manned planes. the zanzibar commission for lands will use the maps for better planning, land tenure and environmental monitoring. data is being collected in collaboration with the rgoz. using data data collected can be used for risk assessment and planning activities. collecting data sharing data using data pacific islands: cook islands, fiji, kiribati, marshall islands, federated states of micronesia, nauru, niue, palau, papua new guinea, samoa, solomon islands, timor-leste, tonga, tuvalu, vanuatu pacific catastrophe risk assessment and financing initiative (pcrafi) is a joint initiative of sopac/spc, world bank, and the asian development bank with the financial support of the government of japan, the global facility for disaster reduction and recovery (gfdrr) and the acp-eu natural disaster risk reduction programme, and technical support from air worldwide, new zealand gns science, geoscience australia, pacific disaster center (pdc), opengeo and gfdrr labs. data sharing platform http://pcrafi.spc.int/beta/ number of layers understanding risks in pacific island countries the pacific island countries are highly exposed to the adverse effects of climate change and natural hazards, which can result in disasters affecting their economic, human, and physical environment and impacting their long-term development agenda. since , natural disasters have affected approximately . million people in the pacific region, causing , reported deaths. sharing data throughout the pacific islands launched in december , the pacific risk information system enhances management and sharing of geospatial data within the pacific community. the system enables the creation of a dynamic online community around risk data by piloting the integration of social web features with geospatial data management. exposure, hazard, and risk maps for pacific countries were produced as part of the pacific catastrophe risk assessment and financing initiative (pcrafi) and are accessible through this platform as powerful visual tools for informing decision-makers, facilitating communication and education on disaster risk management. thumbnail image by samoa department of foreign affairs and trade licensed under cc by . sharing data using data sri lanka the disaster management centre of sri lanka (dmc) with the support of the world bank has been developing the open data for resilience initiative (opendri) to support evidence-based methods to better plan for, mitigate, and respond to natural disasters. counterpart disaster management centre, ministry of disaster management number of buildings mapped , with attributes each roads mapped > km understanding sri lanka’s risks since , flood and drought events have cumulatively affected more than million people across sri lanka. regular flooding, drought, and landslides are natural hazards that threaten the long-term growth and development of the country. in sri lanka, nearly $ million in unplanned expenditures resulting from flooding in and has strained government budgets and required reallocation from other planned development priorities. the impacts of these events are growing due to increased development and climate change, both of which put more assets at risk. sharing data to enable better disaster risk modeling, the government of sri lanka partnered with gfdrr, undp and ocha on the development of an opendri program in november . this branch of the initiative focused on the south asia region and was dubbed the open cities project. a component of the opendri open cities mission in sri lanka was to collate data around hazards and exposure and prepare them to be uploaded into a geonode which serves as a disaster risk information platform. working with the dmc, the national survey department, department of the census and statistics, nation building research organization, information and communication technology agency, department of irrigation, several universities and the international partners, the opendri team supported dmc with the aggregation of data that had been stored in static pdfs, old paper maps and several databases onto the geonode. the data on the geonode is currently available to authorized users in the opendri network, in preparation for launch. this transitional state is typical for open data projects, as the partnership reviews data with the parties and affirms that it is ready for release to the open public. some layers may restrict access only to authorized users. collecting data the project has also built technical capacity and awareness in sri lanka through training sessions on open data and crowdsourced mapping in batticaloa city and gampaha district. as a result of the open data for resilience initiative, government and academic volunteers have mapped over , buildings and kilometers of roadways on the crowdsourced openstreetmap database. this enables the country to plan ahead and be prepared for future disaster and climate risks. it also helps planning during disaster responses: the data was used to assess flooding impacts in real time and direct government resources during the may floods in gampaha district. collecting data sharing data using data view all projects resources at opendri we are committed to increasing information that can empower individuals and their governments to reduce risk to natural hazards and climate change in their communities. we’ve compiled a database of relevant resources to share what we have learned through our own projects and from the work of others. view all resources open data for resilience initiative field guide planning an open cities mapping project opendri principles open cities africa final report oct open cities ai challenge: segmenting buildings for disaster resilience may perspectives on responsible ai for disaster risk management may sustainability in openstreetmap feb an initiative of sign up for our newsletter: projects resources about news © copyright open dri privacy policy sign up for our newsletter: > projects resources about news an initiative of opensolr - hosted solr cloud solutions, worldwide home pricing enterprise contact support faq blog documentation security compliance submit new support ticket users login register reset password solr hosting, worldwide. enjoy the ultimate customisation and flexibility, while harnessing the full power of solr on any scale. either its a billion requests per second, or ten billion documents, we are fully managing your solr cloud, while you focus on what matters most. get started free trial login setup your solr index in just easy steps . create your opensolr index, in your preferred region and provider. . edit or upload your solr configuration files, according to your applicaton needs. . connect your application to your solr index by using the opensolr control panel information. schedule a meeting to learn more. drupal ready search api opensolr is the official opensolr-drupal plugin, that helps you setup your opensolr index with drupal in just a few clicks. extends the main search api solr module and provides functionality for connecting and managing solr using the opensolr services. wordpress ready wp-solr opensolr plugin , helps you integrate your opensolr index with wordpress easily. simply follow the wp-solr website instructions, and you'll be able to start using your opensolr index with wordpress, in just a few clicks. team management made easy we realize that you may not be alone in your hosted solr ventures. therefore, it is now very easy to manage your teams with us. get started with your new solr cloud team, in just a few easy clicks, directly from your opensolr control panel. live action log be informed, be safe, and... stay updated with your team's progress, and actions inside your opensolr account, at all times, with our new live action log. we now provide a comprehensive history, of all the actions taken inside your account, and on your solr indexes, for any of our account types. schedule a meeting to learn more. provision we are provisioning and setting everything up for you, so you don't have to. from a shared solr index, to a high-availability solr cluster, we will set it all up for you in no time. deploy deployment and automation has never been easier. we provide automation api that will make deployment very easy, so you can focus on what really matters. enjoy sit back, relax, and let us fully assist you throughout your initial setup, of your solr cluster. talk to us, to see how you can get started right away. ”opensolr has turned our entire business around, with their great support and restless passion for solr. supporting our entire resilient, fail-over solr cloud, it is truly a pleasure to work with such great professionals!” t. tomlinson director of operations ”we would not be where we are today, had it not been for the great product and service opensolr has provided us with these past two years. professional support and state of the art solr services makes opensolr # in the industry.” j. eberle managing director you are never alone! our support professionals, are highly trained in solr, and full stack web development technologies, and we will assist you, every step of the way. simply create your opensolr shared index, or resilient cluster, and contact us, if you need help with your initial setup. you can also submit a support ticket, or even request a free callback, if you need to talk to our sales, billing, or tech support representatives. talk to an expert back to top faq terms & conditions privacy policy blog support © - opensolr.com - all rights reserved microsoft word - strategic planning appendix b_metricsrankingspibb.docx appendix b: metrics, rankings, and partnership for incentive-based budget appendix b: metrics, rankings, and partnership for incentive-based budget published june appendix b: metrics, rankings, and partnership for incentive-based budget metrics white paper appendix b: metrics, rankings, and partnership for incentive-based budget on th e de si gn a nd u se o f m etr i cs by the strategic planning metrics subcommittee june , the purpose of this document is two-fold. first, it serves as a guide for the strategic planning committee as it drafts the - virginia tech strategic plan, particularly in terms of the metrics the committee will choose to assess progress towards the strategic objectives. second, it provides some guidelines for the larger virginia tech community, particularly administrators responsible for defining and implementing metrics throughout other parts of the academic enterprise, on how to design and use metrics. in writing this, we take the point of view that in any large organization key metrics are indispensable for understanding and communicating organizational performance: they help report progress and guide decision making. furthermore, we recognize that some metrics will be used, whether virginia tech likes it or not, by external organizations in such things as university rankings. given the impact of these rankings on the university, it is thus critical that such metrics are not ignored and, in fact, perhaps actively managed. on the other hand, we are also cognizant that there is a proliferation of metrics throughout society that follows from a frequently misplaced faith that metrics: ( ) can be used to fully characterize an individual’s or organization’s performance, and ( ) that they are useful for properly and positively incentivizing behavior. as wilsdon et al. ( ) say in their report, the metric tide: report of the independent review of the role of metrics in research assessment and management, metrics evoke a mixed reaction from the research community. a commitment to using data and evidence to inform decisions makes many of us sympathetic, even enthusiastic, about the prospect of granular, real-time analysis of our own activities. if we as a sector can’t take full advantage of the possibilities of big data, then who can? yet we only have to look around us, at the blunt use of metrics such as journal impact factors, h-indices and grant income targets to be reminded of the pitfalls. some of the most precious qualities of academic culture resist simple quantification, and individual indicators can struggle to do justice to the richness and plurality of our research. too often, poorly designed evaluation criteria are “dominating minds, distorting behaviour and determining careers.” at their worst, metrics can contribute to what rowan williams, the former archbishop of canterbury, calls a “new barbarity” in our universities. the tragic case of stefan grimm, whose suicide in september led imperial college to launch a review of its use of performance metrics, is a jolting reminder that what's at stake in these debates is more than just the design of effective management systems. metrics hold real power: they are constitutive of values, identities and livelihoods. as this paper should make clear, it is critical to carefully select and define metrics, as well as ensure the quality of the data upon which the metrics are calculated. it is equally critical that consumers of the metrics have a nuanced understanding of what each metric does and does not measure as well as how a metric may incentivize behavior, potentially with both intended and unintended consequences. executive summary in the following, we explain what constitutes a good metric. several factors are paramount: ease of measurement, direct correlation to institutional success, predictive of good performance, control by the group being measured, and comparableness to competitors’ measures. we also identify key characteristics of quality data: relevance, accuracy, timeliness, accessibility, interpretability, coherence, and credibility. finally, we foreground key principles for developing metrics. these encompass careful definition, paucity in number, reliable data, isomorphic comparison, cost sensitivity, meaningful ratio expression, minimization of perverse incentives, distinction between target and measurement, and care to ensure ease of measurement does not determine target of measure. definitions in this section, we provide a few key definitions, including defining the term “metric,” then we discuss the characteristics of a good metric, and finally we distinguish between “direct” and “proxy” metrics/measures. to begin, the use of the word “metric” in the context of strategic planning or organizational management is somewhat more specific than the typical dictionary definition. for example, merriam-webster ( ) defines a metric either as “a standard of measurement” or in terms of its formal use in mathematics. the oxford living dictionary ( ) comes closer to our usage amplifying the main definition of “a system or standard of measurement” with “(in business) a set of figures or statistics that measure results.” for our purposes, we use the following definition: metric: a quantifiable measure used to track or assess an individual’s, organization’s, or process’s progress towards a specific objective. citation-based metrics are often referred to as bibliometrics, and the term altmetrics refers to alternative metrics that focus on trying to measure the impact of research in alternative forums such as social media. metrics can be direct or proxy measures of progress towards an objective. a direct measure is one that, as the name suggests, is based on data that directly measure the objective. for example, for an objective focused on achieving a particular enrollment target, a direct metric is the number of students enrolled in the university at, say, the start of the fall semester. on the other hand, a proxy measure is one that is based on data that only indirectly measure the objective. for example, spot scores directly measure student perceptions of teaching but are intended to be proxy measures of actual teaching performance. metrics can be used to assess performance and communicate preferences or as a way to influence organizational behavior. as is discussed in more detail below, designing metrics to influence behavior is the more difficult of the two, both because the measurement becomes less reliable over time as behavior adapts and because it can have unintended consequences potentially leading to unforeseen outcomes. defining a good metric not all metrics are good in the sense that they can be ill-defined and/or ill-applied in any number of ways. see, for example, muller ( a). and, while it is impossible to catalog all the ways that metrics can be misapplied and misused, there are some guidelines about what makes a good metric. appendix b: metrics, rankings, and partnership for incentive-based budget we begin by paraphrasing the five characteristics of a good metric by trammell ( ): • easily measurable: a good metric should be relatively simple to measure. if you have to build a new system or implement a complicated process just to measure the metric, it's probably not worth measuring in the first place. • directly correlated to institutional performance: the metric should be tied to institution-oriented goals you establish for the department, group, or company. the right metric will tell you if you are successfully executing the fundamentals. • predictive of future performance: the best metrics do not tell you just how well you've done (for a business, financials provide that measure); they tell you how well you're going to do - in the next month, semester, or year. • isolated to factors controlled by the group it is measuring: it's difficult to do, but identifying those fundamentals pertaining to a particular team will tell you much more about their strengths and performance. • comparable to competitors' metrics: it’s helpful to track your progress against peer institutions. this will help judge how well you're building or maintaining an operational advantage, holding on to top talent, and retaining students. inherent in these characteristics is the quality of the data, since easily measured poor data is still just poor data, and that to have comparable metrics one must also have data on one’s competitors that is directly comparable. as godfrey ( ) said, “data quality is a critically important subject. unfortunately, it is one of the least understood subjects in quality management and, far too often, is simply ignored.” we return to the question data quality in the next section. building on these, yoskovitz ( ) says a good metric is: • comparative: being able to compare a metric across time periods, groups of users, or competitors helps you understand which way things are moving.” • understandable: take the numbers you're tracking now...if people can't remember the numbers you're focused on and discuss them effectively, it becomes much harder to turn a change in the data into a change in the culture.” • a ratio or a rate: ratios and rates are inherently comparative. for example, if you compare a daily metric to the same metric over a month, you'll see whether you're looking at a sudden spike or a long-term trend.” • changes the way you behave: this is by far the most important criterion for a metric: what will you do differently based on changes in the number? if you don't know, it’s a bad metric.” appendix b: metrics, rankings, and partnership for incentive-based budget of course, the assumption in the last characteristic above is that the metric changes behavior in a positive way (which could also mean reinforcing current behavior) and not a negative one. in choosing metrics, it is critical to assess this, and particularly the issue of whether the metric could drive unintended changes in behavior (perverse incentives). see muller ( a) for examples of unintended consequences and edwards and roy ( ) for illustrations of how well intended metrics can result in perverse incentives. we return to this point later. the twdi blog ( ) lists twelve characteristics of effective metrics. some of these are redundant with the previous characteristics, so here we list the unique ones: • strategic: to create effective performance metrics, you must start at the end point--with the goals, objectives or outcomes you want to achieve--and then work backwards. a good performance metric embodies a strategic objective.” • timely: actionable metrics require timely data. performance metrics must be updated frequently enough so the accountable individual or team can intervene to improve performance…” • referenceable: for users to trust a performance metric, they must understand its origins. this means every metric should give users the option to view its metadata, including the name of the owner, the time the metric was last updated, how it was calculated, systems of origin, and so on.” • accurate: it is difficult to create performance metrics that accurately measure an activity. part of this stems from the underlying data, which often needs to be scanned for defects, standardized, deduped, and integrated before displaying to users. poor systems data creates lousy performance metrics that users won’t trust. garbage in, garbage out.” • correlated: performance metrics are designed to drive desired outcomes. many organizations create performance metrics but never calculate the degree to which they influence the behaviors or outcomes they want.” • relevant: a performance metric has a natural life cycle.” when first introduced, the performance metric energizes the institution and performance improves. over time, the metric loses its impact and must be refreshed, revised, or discarded. some of these characteristics have to do with the quality of the data, including the , referenceable, and accurate characteristics, and we will delve into the question of defining data quality more in the next section. the correlated characteristic makes the point that metrics intended to influence behaviors should influence the desired behaviors and the relevant characteristic connects back to the notion of continuous planning. a good metric is one that is well defined, quantifiably measurable, and if we model it in the form of a “key result” as described by doerr ( ), it has a numeric goal. as described in doerr ( , p. ), andy grove, the former ceo of intel, described his system of objectives and key results as follows: now the two key phrases...are objectives and the key result. and they match the two purposes. the objective is the direction: “ we want to dominate the mid-range microcomputer component business.” that’s an o bjective. that’s where we’re going t o go. key results for this quarter: “win ten new designed for the ” is one key result. it’s a milestone. the two are not the same… timely appendix b: metrics, rankings, and partnership for incentive-based budget the key result has to be measurable. but at the end you can look, and without any arguments: did i do that or did i not do it? yes? no? simple. no judgements in it. now, did we dominate the mid-range microcomputer business? that’s for us to argue in the years to come, but over the next quarter we’ll know whether we’ve won ten new designs or not. what is interesting in this approach is the combination of a numeric goal with the metric itself to form a “key result.” in so doing, this unburdens the objective from having to have a numeric goal and so it can simply express the desired organizational direction. such a system has the potential to help the strategic plan align better with the notion of continual planning where, say, the strategic plan can specify six-year objectives and perhaps key results but the key results (and thus intermediate goals) can be updated more frequently. in the context of a university, the educational advisory board (eab) defined the following seven metrics characteristics in their report “academic vital signs: aligning departmental evaluation with institutional priorities.” their intention is to ensure that “[b]road institutional metrics [can be] translated into clear, actionable goals for academic departments in order to motivate improvement” (eab, , p. ): • aligned: do department-level changes in the metric reflect the relevant institutional goal(s)? • measurable: can the institution collect longitudinal information about the metric? • realistic/fair: does the metric control for variables outside departmental influence? • actionable: does the department have direct influence over this metric? • time-bound: can the department significantly influence the metric in the given time frame? • difficult to game: does the metric eliminate “perverse incentives” to avoid true improvement? • simplified: is the metric easy to understand and not an amalgamation of many calculations? while these are framed in terms of department-level metrics, they clearly apply at all levels of a university. and, finally, wilsdon et al. ( ) refined responsible metrics as having the following dimensions: • robustness: basing metrics on the best possible data in terms of accuracy and scope; • humility: recognizing that quantitative evaluation should support – but not supplant – qualitative, expert assessment; • transparency: keeping data collection and analytical processes open and transparent, so that those being evaluated can test and verify the results; • diversity: accounting for variation by field, and using a range of indicators to reflect and support a plurality of research and researcher career paths across the system; • reflexivity: recognizing and anticipating the systemic and potential effects of indicators and updating them in response. in lieu of adopting the objectives/key results approach, we have proposed the adoption of the eab and wilsdon et al. characteristics as the most relevant to academia. in addition, in concert with the measures of data quality in the next section, they largely capture the previous sets of metric characteristics. appendix b: metrics, rankings, and partnership for incentive-based budget defining “quality data” the international monetary fund ( ) and the organisation for economic co-operation and development (oecd, ) specify seven dimensions of data quality. here we paraphrase their definitions within the context of strategic planning and other types of organizational performance metrics. • relevance: the degree to which the data are useful in a metric for quantifying progress towards a goal or objective. • accuracy: the degree to which the data, via the metric, correctly estimate or describe the characteristics that they are intended to measure. • timeliness: the temporal relevance of the data, generally in the sense that the data are available sufficiently quickly so that the resulting metric is of value and may still acted upon. • accessibility: the ease with which the data can be obtained, including the ease with which the data can be accessed. • interpretability: the ease with which the user may understand and properly use the data for the calculation of the metric or metrics. • coherence: the degree to which the data are logically connected and mutually consistent so that they can be successfully brought together with other statistical information within the framework of the metrics and over time. • credibility: the confidence that users place in the data, where an important aspect is trust in the objectivity of the data. other aspects of data quality include the completeness of the data or, conversely, the lack of missing values in a dataset. incomplete data may result in biased metrics, meaning metrics that systematically under- or over-estimate the quantity of interest. quality data should not be based on convenience samples, meaning incomplete data sets that are assembled simply because they are easy to collect. for example, spot scores based only on those students who choose to submit scores are convenience samples. data scraped off the web and only from select databases by academic analytics are convenience samples. instead, metrics using internal data should be based on census sampling, meaning all the data that is available or, in consultation with a statistician, an appropriate sampling scheme. properly designed, these methods should help ensure that the metric is accurately estimating the characteristics they are intended to measure (per the accuracy dimension above). finally, no data set is perfectly complete, nor will the resulting metric perfectly measure the characteristic of interest. thus, a final measure of quality is the extent to which the data and associated metric are transparent about what they do not measure. for additional concerns about academic analytics, see the american association of university professors march , “statement on ‘academic analytics’ and research metrics.” appendix b: metrics, rankings, and partnership for incentive-based budget key principles this section presents and describes key principles for properly defining and applying metrics. they assume that those defining and selecting metrics will take into consideration the previous discussions on what it means for a metric to be “good” as well as what it means for data to be of high quality. principle # : the very first step should always be careful definition of the objective or goal. only after careful definition of the objective or goal should the metric or metrics be selected. the most important consideration when selecting the metric or metrics is how well the metric or metrics will characterize progress towards the goal or objective. thus, it is key that the metric or metrics be selected with the particular goal or objective in mind. corollaries: • metrics that do not measure progress towards the goal or objective are of no use. • selecting metrics in advance of defining the goal or objective to be achieved is potentially a waste of time. principle # : the number of metrics affiliated with any given goal or objective should be kept as small as possible; more is not always better. one s hould always select the s mallest number of metrics that adequately characterize performance towards the goal or objective. “in general, each objective should be tied to five or fewer key results [i.e., metrics]” (doerr, , p. ). too many metrics make it easy to lose sight of the objective, perhaps to game the system, and to understand what action to take. corollaries: • complexity is the enemy of understanding. when in doubt, apply the kiss principle: keep it simple, stupid. • if it’s not possible to characterize performance towards the goal or objective with a reasonably small number of metrics, it may be that the goal or objective is either too complicated or ill-defined. appendix b: metrics, rankings, and partnership for incentive-based budget principle # : a metric based on weak or poor data, no matter how well defined and intentioned, should not be used. the quality of the data upon which a metric is based is critical and it is not possible to have a good metric that is based on poor or weak data. when using proxy measures, because direct measurement is not possible for some reason, it is equally important to base the proxy measurements on good data. m ost importantly, the notion that a metric based on poor data will lead to good decision making is simply wishful thinking. corollaries: • just as we require rigorous data collection leading to good data in our academic research, so we should require equally good data practices in the management of our academic enterprise. • it is easier to collect good data on our own operations and internal processes than on external processes or entities. principle # : when using metrics to compare between two or more organizations, the data upon which the metrics are calculated must be equivalent between the organizations. this is nothing more than common sense for avoiding apples-to-oranges comparisons. it is possible that two different sets of data will be highly correlated, and thus it may be possible to at least compare trends over time between organizations, but without equivalent data no direct performance comparisons can be made. corollaries: • this means that in general it will be difficult at best, and likely impossible, to compare metrics based on internal data with external entities since the equivalent data for the external entities is unlikely to be available. principle # : the cost of calculating the metric, either in terms of dollars and/or time, should be taken into consideration. all things be equal or nearly equal, the metric that costs less or that can be calculated quicker or easier should be preferred. as the section on defining a good metric discussed, metrics should be relatively simple to measure and thus be both inexpensive and quick to calculate, again relatively speaking. fo r metrics based on internal data, the costs may be in terms of staff time to compile the data if it is already routinely collected. corollaries: • if the data are not already being collected, then the cost in terms of dollars and/or time will likely be significant. appendix b: metrics, rankings, and partnership for incentive-based budget principle # : metrics should be defined in the appropriate units and with the proper denominator (in the case of a ratio) so that they reflect the desired organizational performance and do not confound that performance with exogenous factors. corollaries: • financial data displayed as trends over time must be presented in constant dollars. using real dollars confounds the e ffect of inflation with actual performance a nd should be avoided. for example, showing growth over time without adjusting for inflation overstates the actual growth. • metrics that are a function of organizational size or some aspect of size should be reported on a per capita basis. for example, reporting the number of schs delivered should be per capita because changes in total schs will be confounded with changes in faculty size. principle # : metrics should be crafted to minimize the tendencies toward perverse incentives. this means metrics should always be subjected to anticipatory analysis to discern likely problems that might emerge as a manifestation of perverse incentivization. as discussed in edwards and roy ( , see table i), well-intended metrics can result in perverse incentives. thus, to the extent possible, metrics should be chosen or crafted that minimize these perverse incentives. appendix b: metrics, rankings, and partnership for incentive-based budget principle # : metrics for assessing research and scholarship must follow the ten principles of the leiden manifesto. the leiden manifesto was written as a “distillation of best practice in metrics-based research assessment so that researchers can hold evaluators to account, and evaluators can hold their indicators to account” (hicks et al, , p. ). these practices should be applied in all aspects of university operations that use metrics to assess research and scholarship, including the partnership for incentive-based budget as well as promotion and tenure. the leiden manifesto specifies the following ten principles: . quantitative evaluation should support qualitative, expert assessment. . measure performance should be against the research missions of the institution, group, or researcher. . protect excellence in locally relevant research. . keep data collection and analytical processes open, transparent and simple. . allow those evaluated to verify data and analysis. . account for variation by field in publication and citation practices. . base assessment of individual researchers on a qualitative judgment of their portfolio. . avoid misplaced concreteness and false precision. . recognize the systemic effects of assessment and indicators. . scrutinize indicators regularly and update them. of course, many of these a lso apply to the u se o f metrics for other purposes, including strategic planning, particularly items , , and - . indeed, had these six not been included in the leiden manifesto, then this section would have consisted of principles. principle # : the process of measurement should not influence the objects being measured, else the measurement is made less valid. ( muller, , p. ). goodhart’s law states that ‘‘when a measure becomes a target, it ceases to be a good measure’’ which means that systems will tend to optimize performance in terms of the metrics, often in spite of the consequences (koehrsen, ). this effect can be particularly pernicious when the metric or measurement is tied to funding, but it can also arise in teaching evaluation scores and other systems where the object of measurement is a person’s or organization’s performance. corollary: • if the goal of a metric or metrics is to influence performance, then significant care must be taken to avoid negative outcomes, including perverse incentivization (see principle # ), “metric fixation,” and “short-termism” (see muller, b). other useful references related to scholarship metrics include mcnutt ( ), ioannidis and khoury ( ), carpenter, cone, and sarli ( ), jump ( ), wilsdon et al. ( ), benedictus and miedema ( ), ferguson ( ), edwards and roy ( ), and moher ( ). appendix b: metrics, rankings, and partnership for incentive-based budget principle # : “not everything that can be counted counts, and not everything that counts can be counted” (cameron, , p. ). we conclude with this principle to underscore the point that reducing complex issues/objectives to summary metrics may not always the best strategy. in particular, metrics are not a substitute for management and, particularly when assessing performance, qualitative information can be critically important for understanding and putting the quantitative metrics results in an appropriate context. appendix b: metrics, rankings, and partnership for incentive-based budget discussion: metrics and the strategic plan the definition, application, and use of metrics in our strategic planning process should also be consistent with the following points. • strategic planning is a continual process and should be approached as such by scheduling periodic review of active objectives and implementation. as time passes and new opportunities emerge, it will become essential to adjust for these developments. • the strategic planning should distinguish between metrics for assessment and metrics for incentivization, particularly incentive metrics in the partnership for incentive-based budget model. in addition, strategic planning should distinguish between stretch/aspirational goals and actual/essential goals. • metrics should support the key objectives of the strategic planning process while simultaneously being consistent with and supportive of the strategic plan’s core values. in no case, should a metric contradict a core value or incentivize behavior that would violate a core value. • metrics should be aligned between all levels of the organization. in particular, metrics for the partnership for incentive-based budget should flow from and support the objectives and core values of the strategic plan. • metrics should be conscious of the broader context and work in concert to address negative externalities and secondary effects that undermine other objectives. for instance, a metric basis that incentivizes offering large-enrollment courses will need to be paired with another metric that incentivizes teaching small courses that privilege experiential learning and seminar-style engagement. • per the leiden manifesto, metrics should be based on the plurality of ways that excellence is manifested across multiples colleges and disciplines throughout the university. for instance, research metrics in stem disciplines should not be applied as a universal norm across the entire university (alliance for the arts in research university, ). disciplinary norms for research or creative discovery in performing arts or literary fields should be applied with equal sensitivity as that associated with traditional norms in stem fields. • metrics should be designed for iteration over the long-term so that the institution can recalibrate progress toward metrics over the long-term. examples include periodically assessing fundraising/advancement progress and recalibrating the targets for enrollment growth. • core values and strategic objectives should drive the balance between internal and external factors for metrics. • iterative, continuous assessment with feedback is essential to successful continual planning. in the context of metrics, it is important to create constructive ways to retool objective and metrics when targets are missed. appendix b: metrics, rankings, and partnership for incentive-based budget references american association of university professors ( ). “statement on ‘academic analytics’ and research metrics” dated march , . accessed online at www.aaup.org/file/ academicanalytics_statement.pdf on june , . alliance for the arts in research universities. ( ). what is research? practices in the arts, research, and curricula. ann arbor, mi: university of michigan. benedictus, r., and f. miedema ( ). fewer numbers, better science, nature, , - . cameron, w.b. ( ). informal sociology: a casual introduction to sociological thinking, random house, new york, ny. carpenter, c.r., cone, d.c., and c.c. sarli ( ). using publication metrics to highlight academic productivity and research impact, academic emergency medicine, , - . doerr, j. ( ). measure what matters, portfolio/penguin, new york, ny. eab ( ). academic vital signs: aligning departmental evaluation with institutional priorities. accessed online at https://www.eab.com/research-and-insights/academic-affairs- forum/studies/ /academic-vital-signs on june , . edwards, m.a., and s. roy ( ). academic research in the st century: maintaining scientific integrity in a climate of perverse incentives and hypercompetition, environmental engineering science, , - . ferguson, m.w.j. ( ). treat metrics only as surrogates, nature, , . godfrey, a.b. ( ) eye on data quality, six sigma forum magazine, , – . hicks, d., p. wouters, l. waltman, s. de rijcke, and i. rofols ( ). the leiden manifesto for research metrics, nature, , - . imf ( ). data quality assessment framework (dqaf), chapter ix. data quality and metadata, accessed online at https://unstats.un.org/oslogroup/meetings/ og- /docs/oslo-group-meeting- --escm-ch - draft .pdf on june , . ioannidis, j.p.a., and m.j. khoury ( ). assessing value in biomedical research: the pqrst of appraisal and reward, journal of the american medical association, , - jump, p. ( ), “metrics: how to handle them responsibly,” times higher education. accessed online at www.timeshighereducation.com/features/metrics-how-to- handle-them-responsibly on june , . koehrsen, w. ( ). unintended consequences and goodhart’s law: the importance of using the right metrics, dated february . accessed online at https://towardsdatascience. com/unintended- consequences-and-goodharts-law- d a c on june , . mcnutt, m. ( ). the measure of research merit, nature, , . appendix b: metrics, rankings, and partnership for incentive-based budget https://towardsdatascience www.timeshighereducation.com/features/metrics-how-to-handle-them-responsibly https://unstats.un.org/oslogroup/meetings https://www.eab.com/research-and-insights/academic-affairs www.aaup.org/file merriam-webster ( ). definition of metric, updated on may . accessed at www.merriam- webster.com/dictionary/metric on june , . moher, d., naudet, f., cristea, i.a., miedema, f., ioannidis, j.p.a., and s.n. goodman ( ). assessing scientists for hiring, promotion, and tenure, plos biology, https://doi.org/ . /journal.pbio. . muller, j.z. ( a). the tyranny of metrics, princeton university press, princeton, nj. muller, j.z. ( b). against metrics: how measuring performance by numbers backfires, aeon. accessed online at https://aeon.co/ideas/against-metrics- how-measuring-performance-by-numbers- backfires on june , . oecd ( ). quality dimensions, core values for oecd statistics and procedures for planning and evaluation statistical activities, std/qfs( ) , accessed online at www.oecd.org/sdd/ .pdf on june , . oxford living dictionary ( ). main definitions of metric in english. accessed at http://en.oxforddictionaries.com/definition/metric on june , . tremmell, j. ( ). the characteristics of an effective business metric, inc., september . accessed online at www.inc.com/joel-trammell/the- - characteristics-of-an-effective-business-metric.html on june , . tdwi blog ( ). characteristics of effective metrics, april . accessed online at https://tdwi.org/blogs/tdwi-blog/ / /effective-metrics.aspx on june , . wilsdon, j. et al. ( ). the metric tide: report of the independent review of the role of metrics in research assessment and management. doi: . /rg. . . . yoskovitz, b.. ( ). measuring what matters: how to pick a good metric, onstartups.com, march . accessed online at www.onstartups.com/tabid/ /bid/ /measuring-what-matters-how-to- pick-a-good-metric.aspx on june , . appendix b: metrics, rankings, and partnership for incentive-based budget www.onstartups.com/tabid https://onstartups.com https://tdwi.org/blogs/tdwi-blog/ / /effective-metrics.aspx www.inc.com/joel-trammell/the- -characteristics-of-an-effective-business-metric.html http://en.oxforddictionaries.com/definition/metric www.oecd.org/sdd/ .pdf https://aeon.co/ideas/against-metrics-how-measuring-performance-by-numbers https://doi.org/ . /journal.pbio. https://webster.com/dictionary/metric www.merriam appendix b: metrics, rankings, and partnership for incentive-based budget rankings white paper a top nati on al ly an d glob al ly re co gn iz ed u ni ve r si ty virginia tech aspires to be a top nationally and globally recognized public land-grant university. specifically, our goal is to be a member of that rarified set of universities that are recognized nationally and globally for their excellence in research and education, for their superiority in creativity and innovation, and for their worldwide outreach and service. evaluating our progress towards becoming a top recognized public land-grant university will be partially based on various rankings such as the times higher education (the) world university rankings and wall street journal/times higher education (wsj/the) u.s. college rankings. however, we recognize that these rankings are, at best, proxy measures that neither fully reflect our unique aspirations as a university nor all the relevant and important dimensions of our reputation. that said, we also recognize that each of the university ranking schemes captures some important aspects of a university’s performance. we further recognize that, broadly speaking, both tangible and intangible benefits accrue to universities that are highly ranked. for example, global reputation is important for both international partnerships, collaborations, and enrollments. similarly, prestigious international institutions, governments, and corporations are increasingly considering global rankings as they look for the institutions, academic programs, and faculty with whom they would like to partner. furthermore, qualified international students look to rankings in making their enrollment decisions. however, while we will use the various university rankings as one way to assess our progress towards becoming a nationally and globally recognized top public land-grant university, virginia tech will not change who we are to match or optimize our performance in the rankings. we are proud of who we are, particularly of our land-grant heritage, and we seek to bring that reputation to the world. in this strategic plan, we take the point of view that tracking and managing metrics related to university rankings need not come at the expense of compromising virginia tech’s values and core identity, particularly ut prosim (that i may serve). the key idea is not to pursue rankings at the expense of our identity – it is to improve our ranking while maintaining our unique identity. we will accurately reflect university activities to improve our standing in the various rankings, and we will align activities and practices within university operations to maximally support and promote our research and creative enterprise, and we will do so without compromising our principles or our unique identity as a university. appendix b: metrics, rankings, and partnership for incentive-based budget times higher education world university rankings and wall street journal/times higher education u.s. college rankings, in brief rankings are based on a variety of measures, all quantified and weighted differently by the various ranking schemes. the charts below summarize the measures used by the two ranking organizations previously mentioned: figure . wall street journal/times higher education u.s. college rankings methodology appendix b: metrics, rankings, and partnership for incentive-based budget figure . times higher education world university rankings methodology as the charts show, wall street journal/times higher education u.s. college rankings is undergraduate and teaching oriented, focusing on the following pillars: outcomes, resources, engagement, and environment. while times higher education world university rankings is a more comprehensive ranking scheme that is research oriented, focusing on the following pillars: teaching, research, citations, industry income, international outlook. when considered collectively, these rankings provide a comprehensive view of the of the land-grant mission on both a national and global scale. there are two common factors that cut across both ranking systems. they are: • reputation measures based on surveys of academics and students to gather their subjective judgements; • research measures, including citation measures, based on data collected from the institution and elsevier’s scopus, a database of peer-reviewed literature. appendix b: metrics, rankings, and partnership for incentive-based budget closely related to, and indeed underlying the citations measures, are publications, where quality and high impact publications typically and ideally drive the citation rates. other factors that contribute to a global reputation include: • national and international visibility, including faculty participation, particularly speaking at international conferences and other events; • an effective communications strategy that raises the visibility of the institution in a variety of national and international media, including both traditional media and emerging new forms of media; • and an effective strategy for engaging and leveraging alumni networks. a naive approach to improving a university’s reputation, particularly as measured by the number of publications and the citation rate, would be to encourage and/or incentivize the faculty to increase their output as measured by these metrics. this strategic plan explicitly rejects this approach for the following two reasons. first, simply encouraging faculty to increase publication output and/or citation rates is an exercise fraught with perverse incentives. the objective of research conducted at virginia tech is impactful, high quality scholarship; publication output and citation rates are but proxy measures for this type of activity, not the end goals. second, incentivizing publication and/or citation rates may yield short-term improvements, but it will not likely result in sustained output. quality scholarly publications and high citation rates are output measures of a faculty engaged in impactful research. the inputs are the critical drivers of long-term success: recruitment and retention of a world class faculty supported by systems, processes, and resources that facilitate the conduct of high-quality research. next steps the university has set the following milestones within the strategic planning framework: virginia tech will be a top u.s. public land-grant according to wall street journal/times higher education u.s. college rankings and a top u.s. public land-grant according to times higher education world university rankings by . immediate steps that virginia tech will take, indeed is already taking, to make sure that the university’s current performance is appropriately and properly reflected in the rankings include: . ensuring the various rankings organizations are capturing all the university’s scholarly and creative activities and publications. . similarly, ensuring that those databases used to quantify citation counts, such as scopus, are fully utilized so that all citations are captured. for example, ensuring that faculty have the necessary access and training to access scopus so that they may correct any errors in their data. . significantly increasing placements in both national and international media that promote the university and increase virginia tech’s research profile. similarly, redoubling efforts to increase brand recognition more broadly worldwide. . undertaking a study of those typically surveyed by the ranking organizations to understand how virginia tech is perceived in terms of scholarship and other measures contributing to our national and international reputation. implement appropriate measures to address any shortcomings. to substantially improve the university’s national and international reputation, in order for virginia tech to meet the above milestones, requires significant and sustained investment in faculty and infrastructure. appendix b: metrics, rankings, and partnership for incentive-based budget appendix b: metrics, rankings, and partnership for incentive-based budget partnership for an incentive-based budget (pibb) model pa rtne r sh ip fo r i nc ent i ve- ba sed b ud ge t (p ibb ) m ode l beyond boundaries imagines a university with greater financial resilience, funded by a diverse resource base and supported by budget models that enable adaptability and innovation in an increasingly dynamic academic environment and shifting financial landscape. the partnership for incentive-based budget model is one of the new funding models established within the university to realize this vision. guiding principles as the main funding model for the university’s academic programs, the partnership for incentive-based budget model is intended to integrate university strategic planning with the budget process to ensure that resources are allocated in a manner that supports the university’s core mission and vision. primary principles guiding the development of the partnership for incentive-based budget model are as follows: strategic: • the budget model must connect resource allocations to accomplishing objectives of university strategic plan • the budget model should promote growth and diversification of university resources • the budget model should reward performance outputs and outcomes that are relevant, clearly defined, and easily measured inclusive: • the set of chosen performance metrics should reflect both shared and distinctive strategic outcomes expected from a comprehensive university • performance goals and milestones should be established in collaboration with units being assessed • the budget model should encourage and reward inter- and transdisciplinary instruction, research, and outreach predictable: • the budget model should promote institutional decision-making based on valid data accessible to units being assessed • the budget model and associated budget development processes must ensure transparency in resource decision-making • the budget model must foster the ability to conduct long-range planning responsive: • the budget model must enable the university to manage resources effectively in a dynamic academic and financial environment • the budget model must enable adjustments to resource allocations based on actual performance structure of the partnership for incentive-based budget model the office of the executive vice president and provost continues to work with degree-granting colleges to develop a model that sufficiently resources the academic enterprise, while incenting activities in strategically important directions. to accomplish this, the partnership for incentive-based budget model has been structured around three major budget components that are combined to calculate the overall budget for appendix b: metrics, rankings, and partnership for incentive-based budget academic areas as shown in figure : unit allocations, scorecard allocations, and earmarked allocations. the unit allocations and scorecard allocations are part of a formulaic distribution of resources based on the achievement of annually established milestones across a broad range of performance metrics. earmarked allocations are funding reserved to support specific activities in certain areas of the university. figure : major budget components of the partnership for incentive-based budget model unit allocations metrics associated with unit allocations of the partnership for incentive-based budget model are primarily intended to incentivize growth in major revenue generating activities of the institution. these include, student credit hours and enrollments to reflect the institution’s increased reliance on tuition to support educational costs. also reflected in this portion of the budget model are metrics associated with growing the external funding that the university receives to support operations, including new gifts and commitments provided through fundraising, extramural grant and contract funding for sponsored expenditures, and ancillary income generated from university sales and services. these metrics are termed “unit allocation” metrics because they are assigned a unit of value for each unit of output. for example, in fiscal year - , the partnership for incentive-based budget model allocated $ . per student credit hour to colleges as part of the unit allocations budget component. some unit allocation metrics have an additional budget value, or premium, attached to a subset of the metric’s output in order to incentivize strategically important activities that go beyond revenue generation. for example, in fiscal year - student credit hours delivered to students whose majors were outside of the instructing college received a premium of $ . per student credit hour to incentivize interdisciplinary instruction. in this example, this $ . premium is added to the baseline student credit hour value of $ . , increasing the per unit budget value to $ . per student credit hour. similar premiums are or will be provided for courses within targeted class sizes, honors courses, pathways to general education courses, courses supporting the university’s destination areas initiative, out-of-state enrollments, students enrolled in appendix b: metrics, rankings, and partnership for incentive-based budget more than one major, industry-funded sponsored expenditures, and new gifts and commitments that support scholarship or professorship endowments. additional premiums may be developed over time to incentivize activities that support the university’s strategic plan. in the unit allocation portion of the partnership for incentive-based budget model long-range goals and short- term milestones are established for each metric annually in consultation with the colleges and university administration to ensure that the university’s infrastructure can accommodate projected enrollments, that sufficient instructional resources can be deployed to teach projected class loads, and that external funding targets are attainable. the goal-based nature of the partnership for incentive-based budget model and its intentional connections to university strategic priorities differentiates it from the pure revenue-sharing budget models currently established at many peer institutions. scorecard allocations another characteristic that distinguishes the partnership for incentive-based budget model from other university budget models is its reliance on a broader array of outcomes and activities expected from a comprehensive university. the metrics in the scorecard allocation portion of the partnership for incentive- based budget model captures these outcomes in three broad categories: faculty success, student success, and administrative effectiveness. the scorecard will include summarized measures of faculty activity and faculty composition. these metrics will be drawn from teaching data, faculty activity reporting systems, and other sources to include broad categories like faculty teaching, scholarship, engagement, and diversity. the scorecard will also include a broad range of commonly understood student outcomes. these will include admissions process metrics, progress s to degree metrics, outcomes for graduating students, and student participation in the broad range of curricular and extracurricular experiences that promote the “vt-shaped” student goals of the strategic plan. it will also include measures of student diversity and the opportunity to look at a broad range of outcomes for various populations of underrepresented and underserved students. a third area of metrics will be related to administrative effectiveness. these metrics will monitor institutional efforts in continuous improvement and compliance with important external regulatory requirements. the scorecard portion of the budget is not a metric-by-metric calculation but rather treated as a block grant with a portion subject to a review and allotment process. annually, the provost, the college dean and related vice provosts or vice presidents will jointly review scorecard goals and achievements toward those goals. when progress towards expected outcomes in a scorecard metric is not made, a cooperative, qualitative evaluation of the activity will be undertaken and funds from the college, academic administration and, where appropriate, central resources will be applied to a jointly developed plan addressing outcomes in the particular area. fiscal year - will be the first year that detailed scorecard metrics are incorporated in the partnership for incentive-based budget. the first scorecard metrics will focus on gender and racial diversity among tenured/tenured track and non-tenured instructional faculty, the -year graduation rates for students who enter the university as freshmen, the -year graduation rates for students who enter the university as transfers, and disparities in the graduation rates for underrepresented minority and underserved students. appendix b: metrics, rankings, and partnership for incentive-based budget earmarked allocations the third component of the partnership for incentive-based budget model is earmarked allocations that support specific university activities. these include program and course fees allocated directly to colleges to cover extraordinary costs of instruction associated with some degrees and courses; special session revenues for courses taught during winter and summer terms that are shared with the colleges (approximately % returned to the college and % retained centrally); self-supporting, professionally oriented programs that are anticipated to charge a market rate of tuition and enroll sufficient students to generate net income that will enhance the resources for the college, department and the institution; external income from ancillary operations who charge for services that also support their instructional activities (e.g., the veterinary teaching hospital, the adult day care center, the child development center); and other college-specific allocations in support of strategic academic programs and initiatives. next steps the office of the executive vice president and provost will continue work with the colleges and the university’s administration to refine and, as appropriate, develop new metrics that support the strategic goals of the university. in parallel with this effort, the university is continuously improving the information systems necessary to support the new budget model and other associated strategic decision-making processes and structures (e.g., undergraduate enrollment management, faculty activity reporting, graduate program management, strategic planning metric tracking, and other ad hoc analyses). appendix b: metrics, rankings, and partnership for incentive-based budget structure bookmarks open cities africa open cities africa menu home about partners resources open cities abidjan accra antananarivo bamako brazzaville dar es salaam kampala kinshasa monrovia niamey ngaoundéré pointe-noire saint-louis seychelles yaounde zanzibar city open cities africa open data today to build resilient and sustainable societies tomorrow about learn more building skills, data, and networks to support disaster risk management and urban planning in africa. participating cities abidjan, côte d’ivoire accra, ghana antananarivo, madagascar bamako, mali brazzaville, roc dar es salaam, tanzania kinshasa, drc monrovia, liberia niamey, niger ngaoundéré, cameroon pointe-noire, roc saint-louis, senegal seychelles kampala, uganda yaounde, cameroon zanzibar city, tanzania a collaborative approach data collected through open cities africa will use the openstreetmap platform and a collaborative approach that will provide technical and institutional advantages. photos: © ilab liberia (left), © acager (right) learn more blog posts on open cities africa more stories... videos nd regional meeting – tool design for urban resilience – dar es salaam, tanzania st regional meeting – kickoff for community mapping – kampala, uganda understanding flood vulnerability in niger through drones, open street mapping and flood modeling open cities ngaoundéré: la cartographie libre pour aider à s'adapter au changement climatique open cities africa - saint-louis english subtitles open cities in brazzaville more videos... contact us for any questions or concerns, please send an email to opendri [at] gfdrr [dot] org photo: mark iliffe / © world bank contact learn more an initiative of supported by implementing partners website design based on html up librarian of things skip to content librarian of things weeknote ( ) § last friday i was interviewed for the podcast the grasscast — a game-themed podcast named after the book, the grasshopper: games, life, and utopia. i ramble a little bit in the episode as i tried to be more open and conversational than concise and correct. but i also spoke that way because for some of the questions, no pat answer came immediately to mind. there was one question that stumped me but in my trying to answer, i think i found something i had not considered before. the question was, what is one bad thing about games? and i tried to convey that, unlike video games where you can play with strangers, most tabletop games are generally constrained by the preferences of your social circles. in order to convince others to spend time on a game that might think is too complicated for them or not for them, you need to have be a successful evangelist. also the episode drifts into chatter about libraries, copyright and ebooks. § this week, i reviewed and published another batch of works for our institutional repository from our department of history that was prepared by our library assistants at leddy at this point, we have reviewed and uploaded the works of half the faculty from this department. i’m hoping to finish the rest this month but i think i have some outstanding h p work that might push the end of this project til march. § this morning i assisted with an online workshop called data analysis and visualization in r for ecologists that was being lead by a colleague of mine. r version . . (“bunny-wunnies freak out”) was released on - - . the release of r . . (“lost library book”) is scheduled for monday - - . § on sunday, i published a short response to “windsor works – an economic development strategy” which is going to city council on monday. why am i writing about this document here? i am mention this here because the proposed strategy (l.i.f.t.) lists the following as potential metric for measuring the strategy’s success… take it from me, someone who knows a quite a bit about citations — the city should use another metric — perhaps one pertaining to local unemployment levels instead. § a viral post from resurfaced on my fb feed this week and unlike most of the posts i read there, this one did spark joy: and it struck me how much i loved that the anti-prom was being at the library. so i started doing some research! it appears to me that some anti-proms are technically better described as alternative proms. these proms have been established as an explicitly safe place where lgbtq young people can enjoy prom. other anti-proms are true morps. i now wonder what other anti-traditions should find a home at the public library. author mita williamsposted on february , february , categories weeknotesleave a comment on weeknote ( ) weeknote ( ) i don’t have much that i can report in this week’s note. you are just going to have to take my word that this week, a large amount of my time was spent at meetings pertaining to my library department, my union, and anti-black racism work. § last year, around this same time, some colleagues from the university and i organized an speaking event called safer communities in a ‘smart tech’ world: we need to talk about amazon ring in windsor. windsor’s mayor proposes we be the first city in canada to buy into the ring network. as residents of windsor, we have concerns with this potential project. seeing no venue for residents of windsor to share their fears of surveillance and loss of privacy through this private-partnership, we hosted an evening of talks on january nd, at the performance hall at the university of windsor’s school of creative arts windsor armories building. our keynote speaker was chris gilliard, heard recently on cbc’s spark. since that evening, we have been in the media raising our concerns, asking questions, and encouraging others to do the same. the city of windsor has yet to have entered an agreement with amazon ring. this is good news. this week, the city of windsor announced that it has entered a one-year deal partnership with ford mobility canada to share data and insights via ford’s safety insights platform. i don’t think this is good news for reasons outlined in this post called safety insights, data privacy, and spatial justice. § this week i learned a neat tweetdeck hack. if set up a search as column, you can limit the results for that term using the number of ‘engagements’: § § i haven’t read this but i have it bookmarked for potential future reference: the weaponization of web archives: data craft and covid- publics: an unprecedented volume of harmful health misinformation linked to the coronavirus pandemic has led to the appearance of misinformation tactics that leverage web archives in order to evade content moderation on social media platforms. here we present newly identified manipulation techniques designed to maximize the value, longevity, and spread of harmful and non-factual content across social media using provenance information from web archives and social media analytics. after identifying conspiracy content that has been archived by human actors with the wayback machine, we report on user patterns of “screensampling,” where images of archived misinformation are spread via social platforms. we argue that archived web resources from the internet archive’s wayback machine and subsequent screenshots contribute to the covid- “misinfodemic” in platforms. understanding these manipulation tactics that use sources from web archives reveals something vexing about information practices during pandemics—the desire to access reliable information even after it has been moderated and fact-checked, for some individuals, will give health misinformation and conspiracy theories more traction because it has been labeled as specious content by platforms. § i’m going to leave this tweet here because i might pick up this thread in the future: this reminds me of a talk given in by data & society founder and president, danah boyd called you think you want media literacy… do you? this essay still haunts me, largely because we still don’t have good answers for the questions that dr. boyd asks of us and the stakes have only gotten higher. author mita williamsposted on january , january , categories weeknotesleave a comment on weeknote ( ) weeknote ( ) hey. i missed last week’s weeknote. but we are here now. § this week i gave a class on searching scientific literature to a group of biology masters students. while i was making my slides comparing the advanced search capabilities of web of science and scopus, i discovered this weird behaviour of google scholar: a phrase search generated more hits than not. i understand that google scholar performs ‘stemming’ instead of truncation in generating search results but this still makes no sense to me. § new to me: if you belong to an organization that is already a member of crossref, you are eligible to use a similarity check of documents for an additional fee. perhaps this is a service we could provide to our ojs editors. § i’m still working through the canadian journal of academic librarianship special issue on academic libraries and the irrational. long time readers know that i have a fondness for the study of organizational culture and so it should not be too surprising that the first piece i wanted to read was the digital disease in academic libraries. it begins…. though several recent books and articles have been written about change and adaptation in contemporary academic libraries (mossop ; eden ; lewis ), there are few critical examinations of change practices at the organizational level. one example, from which this paper draws its title, is braden cannon’s ( ) the canadian disease, where the term disease is used to explore the trend of amalgamating libraries, archives, and museums into monolithic organizations. though it is centered on the impact of institutional convergence, cannon’s analysis uses an ethical lens to critique the bureaucratic absurdity of combined library-archive-museum structures. this article follows in cannon’s steps, using observations from organizational de-sign and management literature to critique a current trend in the strategic planning processes and structures of contemporary academic libraries. my target is our field’s ongoing obsession with digital transformation beyond the shift from paper-based to electronic resources, examined in a north american context and framed here as the digital disease. i don’t want to spoil the article but i do want to include this zinger of a symptom which is the first of several: if your library’s organizational chart highlights digital forms of existing functions, you might have the digital disease. kris joseph, the digital disease in academic libraries, canadian journal of academic librarianship, vol ( ) ouch. that truth hurts almost as much as this tweet did: author mita williamsposted on january , january , categories weeknotesleave a comment on weeknote ( ) weeknote ( ) this week’s post is not going to capture my ability to be productive while white supremacists appeared to be ushered in and out of the us capitol building by complicit police and covid- continued to ravage my community because our provincial government doesn’t want to spend money on the most vulnerable. instead, i’m just going to share what i’ve learned this week that might prove useful to others. this week i added works to three faculty member’s orcid profiles using orcid’s trusted individual functionality. one of these professors was works in the field of psychology and i found the most works for that researcher using base (bielefeld academic search engine) including apa datasets not found elsewhere. similarly, i found obscure eric documents using the lens.org. unfortunately, you can’t directly import records into the lens into an orcid profile unless you create a lens profile for yourself. i’ve added the lens to my list of free resources to consult when looking for research. this list already includes google scholar and dimensions.ai. fin author mita williamsposted on january , february , categories weeknotesleave a comment on weeknote ( ) weeknote ( ) § it looks like andromeda yelton is sharing weeknotes (“this week in ai“). i can’t wait to see what she shares with us all in . § earlier this fall, clarivate analytics announced that it was moving toward a future that calculated the journal impact factor (jif) based on the date of electronic publication and not the date of print publication… this discrepancy between how clarivate treated traditional print versus online-only journals aroused skepticism among scientists, some of whom… cynically suggested that editors may be purposefully extending their lag in an attempt to artificially raise their scores. changes to journal impact factor announced for , scholarly kitchen, phil davis, dec , i don’t think there is anything cynical about the observation that journal publishers picked up a trick from those booksellers who actively engage in promoting pre-publication book sales because those weeks of sales are accumulated and counted in the first week of publication which results in a better chance of landing on the new york times bestseller list. § in , a team at georgia state university compiled a report on virtual learning best practices. while evidence in the field is “sparse” and “inconsistent,” the report noted that logistical issues like accessing materials—and not content-specific problems like failures of comprehension—were often among the most significant obstacles to online learning. it wasn’t that students didn’t understand photosynthesis in a virtual setting, in other words—it was that they didn’t find (or simply didn’t access) the lesson on photosynthesis at all. that basic insight echoed a study that highlighted the crucial need to organize virtual classrooms even more intentionally than physical ones. remote teachers should use a single, dedicated hub for important documents like assignments… the most significant education studies of , edutopia, by youki terada, stephen merrill, december , § i’m pleased to say that with some much appreciated asssistance, our ojs instances are now able to allow to connect authors with their orcid profiles. this means that all authors who have articles accepted by these journals will receive an email asking if they would like to connect to orcid. i was curious how many authors from one of our existing journals had existing orcid profiles and so i did a quick check. this is how i did it. first, i used ojs’s export function to download all the metadata available at an article level. next, i used the the information from that .csv file to create a new spreadsheet of full names. i then opened this file using openrefine. then, through the generosity from jeff chiu, i was able check these last names with the orcid api using the openrefine reconciliation service and chiu’s smartname server: http://refine.codefork.com/reconcile/orcid/smartnames. using the smart name integration, i can limit the list to those names very likely to match. with this set of likely suspects in hand, i can locate the authors in the ojs backend and then send invitations from the ojs server from their author profile (via the published article’s metadata page): § i can’t wait to properly tuck into this issue of the canadian journal of academic librarianship with its special focus on academic libraries and the irrational § happy solstice, everyone. author mita williamsposted on december , december , categories weeknotesleave a comment on weeknote ( ) weeknote ( ) § i don’t have much to report in regards to the work i’ve been doing this week. i tried to get our orcid-ojs plugin to work but there is some small strange bug that needs to be squished. luckily, next week i will have the benefit of assistance from the good people of crkn and orcid-ca. what else? i uploaded a bunch of files into our ir. i set up a site for an online-only conference being planned for next year. and i finally got around to trying to update a manuscript for potential publication. but this writing has been very difficult as my attention has been sent elsewhere many times this week. § unfortunately i wasn’t able to catch the live teach-in #againstsurveillance on tuesday but luckily the talks have been captured and made available at http://againstsurveillance.net/ so many of our platforms are designed to extract user data. but not all of them are. our institutions of higher education could choose to invest in free range ed-tech instead. § bonus links! making a hash out of knitting with data shannon_mattern’s library | zotero mystery file! author mita williamsposted on december , december , categories weeknotesleave a comment on weeknote ( ) weeknote ( ) § first off is this recommended read from the november th issue of the new yorker, the rise and fall of getting things done by cal ‘deep work’ newport. as newport himself describes his work, it’s not, however, really about david allen’s productivity system, which longtime readers (and listeners) know i really admire. it’s instead about a deeper question that i hadn’t heard discussed much before: why do we leave office workers to figure out on their own how to get things done? with the notable exception of agile software development teams, companies in this sector largely leave decisions about how work is assigned, reviewed, and organized up to individuals. we promulgate clear objectives and construct motivating corporate cultures, but when it comes to actually executing these tasks, we just hook everyone up to an email address or slack channel and tell them to rock and roll. this has led to a culture of overload and fragmented attention that makes everyone involved miserable. i don’t want to spoil the conclusions of this article, but i will tip you off that i’m filling this article away in my notebook about visualizing workflow. § i discovered this work from carl’s e-alert newsletter, thinking politically about scholarly infrastructure (a.j. boston, lpc blog – fellows journal, november ). parts of it hit a little too close to home for my liking… i’m sure i’m being unfair in my stance. to capture a diverse constituency, a big-tent approach can be effective. compromise can cause cynicism about our politics, but sometimes a little progress can be better than a lot of regression. that’s the story i’ve told myself, at least, while making my daily compromise as a scholcomm librarian who manages our elsevier-owned institutional repository service, digital commons. my school contracted with bepress (then an independent company) shortly before hiring me to manage it, and my values felt fully aligned as i made the pitch across campus to deposit green oa manuscripts there. but that feeling changed with the announcement of elsevier acquiring bepress in august (mackenzie, ). since , the digital commons service hasn’t worsened, but the premise that many customers initially bought into, of supporting an independent platform in the scholarly communication ecosystem, has eroded. and what do people do when they face a deterioration of goods and services? for a.o. hirschman ( ), there are three choices (which later scholars have revised upon): exit, voice, and loyalty. in my case, exit seems out of the question: a diverse constituency of groups on my campus have now integrated the software, and a swap would be overly-costly and damage relationships in the process. i don’t know whether i’d categorize what i am doing now as voice or loyalty, but what i do know is that there is a strong glimmer of recognition when sen. harris walks her fracking-issue tightrope, or when grant-funding institutions rock the boat just lightly enough that it doesn’t risk a capsize. § also from aforementioned e-alert, aap and ccc end georgia state ‘e-reserves’ copyright litigation (p. anderson, publishing perspectives, november ) after a -year fight, the association of american publishers and copyright clearance center have declined to pursue any further appeals in their lawsuit against georgia state university regarding their reliance on fair use in making materials available via e-reserves. read more @pubperspectives i used to refer to the georgia state e-reserves case as an example of selective enforcement of copyright by publishers in which educational use of works behind an authentication system was vigorously challenged in court, while rampant open distribution of works under copyright via academia.edu and researchgate was ignored for years. § i only read the headline and the abstract of this article but i am sharing it anyway because i liked the conclusion that tyler cowan [ht] drew from it: open access improves the quality of citations. § earlier this week hugh rundle published a blog post called empathy daleks that gave me life: her studies indicate that diversifying the authors, perspectives, representations and examples in standard textbooks is not simply “more inclusive” or “just” in an abstract way (though that would be good anyway). students who feel they belong — who feel validated as members or potential members of a profession or academic discipline — are more likely to succeed and complete their degrees. that is, lambert suggests that diversifying the authors and even the examples or hypothetical actors in university textbooks by itself has a positive effect on completion rates, engagement, and student satisfaction with courses. amy nusbaum shows in a recent article that oer is an effective way to accelerate this, because with licenses allowing “remixing” of content the examples used within open textbooks can be updated to suit local needs without having to rewrite the entire text…. but it was lambert uttering the magic words about diverse texts improving “student success” that suddenly felt quite subversive. to understand why, we need to interrogate what universities usually mean when they talk about “student success”, and particularly the infrastructures universities have been building around it. hugh rundle, empathy daleks, november , and on that note… i liked this tweet about university rankings some days ago. "none of these ‘flagship’ rankings considered #openaccess, equality, diversity, sustainability or other society-focused agendas." https://t.co/am ctepuom — peter suber (@petersuber) november , speaking of society-focused agendas, while i was doing some of the more rote collection development tasks this week (reviewing lists of duplicate titles, finding missing titles that were of need of replacing), i listened to a number of episodes of terry greene’s getting air: the open pedagogy podcast and i enjoyed them very much. i’ve had the pleasure of knowing and spending time with some of the guests on his show and it is such a treat to hear them speak about the careful thought and thoughtful care they put into their work of teaching. author mita williamsposted on november , november , categories weeknotesleave a comment on weeknote ( ) weeknote ( ) i had a staycation last week. it took me two days just to catch up on email i received while i was gone. and the only reason i was able to do that in two days is because i had booked the days off as meeting-free so i could attend an online conference. said conference was the indigenous mapping workshop. i was not able to attend many of the sessions but the ones that i did rekindled my affection for web-maps and inspired me to make two proof-of-concept maps. the first one is of bike parking in my neighbourhood. the location and photos were collected using a web form through kobotoolbox. i then downloaded a csv from the site and paired with this leaflet-omnivore powered map. bike parking in my neighbourhood the second map i made was a more mischievous creation in which i used mapbox studio to rename the world. other things i did this week: chair our monthly information services department meeting, selected a set of duplicate books as part of a larger weeding project, ordered a lot of books using exlibris’ rialto, did a librarychat shift, contributed to some collection management work, did some ojs support, attended several meetings, and wrote many emails. one day i would like write a piece that applies the concept of technical debt to library services / the library as an organization. i didn’t do much reading this week but i did read one article which i think has an exceptional title: public libraries are doing just fine, thank you: it’s the “public” in public libraries that is threatened this is a project that is close to my civic interests: introducing the civic switchboard data literacy project! we’re pleased to announce the receipt of an imls laura bush st century librarian program grant, which will support the next piece of civic switchboard project – the civic switchboard data literacy project! this project builds on the civic switchboard project’s exploration of civic data roles for libraries and will develop instructional materials to prepare mlis students and current library workers for civic data work. through the civic switchboard project, we’ve learned about common barriers to entry that libraries are navigating with civic data work. we regularly heard library workers say that they feel unqualified to participate in their civic data ecosystems. with this barrier in mind, the civic switchboard data literacy project will build and pilot instructional material that mlis instructors can integrate in coursework and that can be used in professional development training in library settings. author mita williamsposted on november , categories weeknotesleave a comment on weeknote ( ) weeknote ( ) some things i was up to this past week: i registered for the indigenous mapping workshop which will run nov. - ; had meetings pertaining to servers and surveys; attended regular meetings including that of the university library advisory committee, leddy library department heads, my bi-weekly meeting with library admin, and the wufa grievance committee uploaded another batch of etds to the repository uploaded another batch of final edits to the ossa conference repository ordered books that have gone missing from the library (including steal like an artist natch) as well titles to support the school of the environment discussed apcs, video streaming, and the potential structure of the new leddy library website with various colleagues; and did an evening shift of our livechat research help service. i don’t think i’ve said this publicly but the weekly updates from the carl e-alert newsletter are excellent and are put together so well. from last week’s alert, i learned of this amazing project: community members living in vancouver’s downtown eastside (dtes) have been the focal point of countless scholarly research studies and surveys over the years. up until recently, this research has remained largely out of reach to participants and community organizations, locked away in journals and other databases that require paid subscriptions to access. community members have said they would benefit from access to that data for evaluating program and service effectiveness, for example, or for grant writing. the recently launched downtown eastside research access portal (dtes rap), a project led by the ubc learning exchange in partnership with ubc library’s irving k. barber learning centre, is designed to change that. the dtes rap provides access to research and research-related materials relevant to vancouver’s downtown eastside through an easy-to-use public interface. the portal was developed in consultation with dtes residents and community organizations through focus groups and user experience testing, and in collaboration with a number of university units. new downtown eastside research access portal takes collaborative approach to open access (ubc) i love that this collection is centred around the needs of those who have been studied and not the needs of the researcher. and not to center my own work but (but) i was hoping to explore similar work during my last sabbatical but for a variety of reasons, it did not come to pass. no weeknote update next week because i’m taking a staycation! author mita williamsposted on november , november , categories weeknotesleave a comment on weeknote ( ) weeknote ( ) this is my third week of weeknotes and i have to say that the format is agreeing with me. i did a quick search online to see if any other librarians have adopted this particular format and i couldn’t find anyone from the librarian profession so i have yet to become an influencer (*snerk*). i did find a data scientist from the house of commons library who employs the practice quite well. this is consistent with my hunch that the weeknotes format is still largely an expression of uk civic computing types. many people use weeknotes to report on their last week’s activities as its originator intended… started on the blog of design company berg a few months back, weeknotes detailed what they were up to that week, what had been going well, what hadn’t. they were just blog entries, updated weekly, nothing more remarkable than that. except they struck a little chord with people — and other companies and individuals started doing the same thing. russell m davies: on the structure of time, wired uk, may for what it’s work, this is what i’ve been up to during the past week: worked with our cataloguing team to process a detailed shelf-reading list of our theses and dissertations, met with a small group of scholars who are working on establishing a new journal on our ojs system, helped a student with a thorny research question, collected and delivered a bibliography of works for the university’s anti-black racism office, worked on a draft statement of publication ethics for two of our ojs journals, worked on the reference chat schedule for the month, worked with colleagues on a potential survey, attended several online meetings, researched how we could promote our collections using online book-carousels, uploaded some of the final manuscripts of the ossa conference, uploaded a batch of etds into our repository, and answered a truckload of email. i’m not sure if i’m going to report on my workings every week. i’m more interested in using the weeknotes format to help me keep up with my reading. speaking of which, this morning i spent some time with the latest issue of portal: libraries and the academy which has a variety of articles that touch on the role of the library liaison and of digital scholarship. i saved for later this excerpt from survey of digital humanities online guides in canadian academic research libraries: in the digital scholar, martin weller argues that new digital tools are “necessary, but not sufficient, for any substantial change in scholarly practice” that they might help to bring about. his contention is that for these technologies to be truly transformative, three factors must converge: digital content, networks, and openness. when high-quality scholarly content can be shared digitally via online networks without legal restrictions, we enter an era of scholarship—digital scholarship—that differs substantially from the traditional one. an amplification of the scope of available academic content and the ability to instantly publish and share one’s content online challenges the fundamental assumptions about the nature of scholarly practice. along this line, robin goodfellow and mary lea define digital scholarship as “the relatively recent invention of cross-disciplinary groups of individual scholars … who have begun to use technology to disseminate their own work outside the formal academic publishing system.” for at least the last twenty years, the academic library has been licensing collections of digital objects from commercial vendors for the private use of those only belonging to the campus. the work to maintain these collections is considered the work of a electronic resources librarian and is not considered digital scholarship. expanding a bit from what i read from the above, digital scholarship is used to designate labour that is dedicated to the transformation of scholarship by making available collections of material openly licensed on the internet and structured for use and re-use at both the item level and at the level in which the collection itself is data. as a scholcomm librarian, i particularly like this framing. also, as a scholcomm librarian, i appreciated and enjoyed this presentation on the promise and peril of transformative agreements by brianne selman. 🎃 happy hallowe’en! 🎃 author mita williamsposted on october , categories weeknotesleave a comment on weeknote ( ) posts navigation page page … page next page about me librarian of things is a blog by me, mita williams, who used to blog at new jack librarian until blogger.com finally gave up the ghost. if you don’t have an rss reader, you can subscribe for email delivery through mailchimp. you can learn more about my work at aedileworks.com as well as my other blogs and my weekly newsletter. if you are an editor of a scholarly journal and think that a post could be expanded into a more academic form, please let me know. search for: search recent posts weeknote ( ) weeknote ( ) weeknote ( ) weeknote ( ) weeknote ( ) archives february january december november october september june may april october june may april march january july june may april december june may april november august july meta log in entries feed comments feed wordpress.org librarian of things proudly powered by wordpress pricing - hosted solr as a service solutions home pricing enterprise contact support faq blog documentation security compliance submit new support ticket users login register reset password please wait... we are now processing your transaction. please do not navigate away from this page. opensolr cloud pricing get started with your opensolr cloud service, and receive a staggering, % discount, for your yearly orders. playground go ahead and try it out. free for days! free register now solr cluster solr index mb disk space / index mb bandwidth / index https & auth security no uptime sla limited email support shared infrastructure - go dedicated basic for small to mid-sized websites and applications € /mo. (billed yearly) buy now solr clusters solr index mb disk space / index gb bandwidth / index https & auth security limited uptime sla email support shared infrastructure - go dedicated entrepreneur extra disk space, and bandwdith. suitable for mid-large websites/apps. € /mo. (billed yearly) buy now solr clusters solr indexes mb disk space / index gb bandwidth / index https & auth security limited uptime sla email & helpdesk support shared infrastructure - go dedicated executive high availability and load balancing guarateed uptime sla & support € /mo. (billed yearly) buy now solr cluster solr indexes gb disk space / index gb bandwidth / index https & auth security . % uptime guarantee email & helpdesk support shared infrastructure - go dedicated customize & checkout still not sure? schedule a meeting with us, to learn more. resources: solr clusters clusters € / cluster / month solr indexes indexes € / index / month index documents , documents € / k documents index disk (mb) mb € / mb / index / month bandwidth (gb) gb € / gb / index / month web crawler urls url(s) € / url do you want to use the opensolr webcrawler® ? analytics (is now free!) do you want to use our opensolr index analytics® ? € / month) backup do you want to use the opensolr backup manager® ? € / index / month plan: pay monthly pay yearly ( % discount!) price: € / month € / year tax: % € / month € / year total: € / month € / year - save € register now login to your account full featured control panel the opensolr control panel, will provide you with the full functionality to get your solr cloud running in just minutes. want to know more? we are always here, either via our support ticket system, phone or email, you can always count on our support! get in touch now! free playground back to top faq terms & conditions privacy policy blog support © - opensolr.com - all rights reserved common questions - rails autoscale pricingbloghelp & docsupdates homepricingbloghelp & docsupdates common questions plans & pricing getting started using the dashboard basic settings scheduled autoscaling advanced settings long-running jobs troubleshooting api getting help product updates common questions (faq) if i haven't answered your question here, email me! plans & installation what kinds of apps can use rails autoscale? you must have a ruby web application running on heroku. it doesn't have to be rails, but it does need to use rack. how do i install rails autoscale? just run heroku addons:create rails-autoscale from a terminal. check out the getting started guide for step-by-step instructions. what's the difference between rails autoscale plans? every rails autoscale plan supports the same features and the same service. the only difference is the maximum number of dynos supported. for example, if you might need to autoscale to four dynos or more, you'll at least need the silver plan. see the autoscale range docs for more on this. how does the free trial work? the "trial" plan is the default when installing via the heroku cli (heroku addons:create rails-autoscale). with the trial plan, you have a week of unlimited autoscaling. at the end of seven days, the add-on will remain installed, but autoscaling will be disabled until you upgrade to a paid plan. for more details, see the plans & pricing docs. how do i use rails autoscale on multiple heroku apps? rails autoscale does not support attaching to multiple apps. you must install the add-on separately for each app. can i use other autoscalers together with rails autoscale? you can use different autoscalers for different processes. for example, you could use heroku's native autoscaling for web dynos and rails autoscale for worker dynos. do not use multiple autoscalers on the same process. this results in very unpredictable scaling behavior. how is rails autoscale different from heroku's autoscaler? heroku offers a native autoscaling solution that's worth a try if you run performance dynos and you only need to autoscale web dynos. here's what makes rails autoscale different: web autoscaling based on request queue time instead of total response time. this means more reliable autoscaling. worker autoscaling for sidekiq, delayed job, and que. works great on standard and performance dynos. personalized customer support from the developer who built it. can i use rails autoscale with a non-rails app? you must be running a rack-based ruby app on heroku. if your rack-based app is not running rails, see these instructions on setting up rails_autoscale_agent. what is the performance impact of the middleware agent? the agent has no noticeable impact on response time. it collects the queue time for each request in memory—a very simple operation—and an async reporter thread periodically posts those queue times to the rails autoscale service. check out the middleware code on github if you're interested. why is the autoscale agent not running? perhaps you're running a worker-only app, or an app with very little web traffic? if not, check out the troubleshooting guide. what happens if i change my rails_autoscale_url config var? bad things will happen. the rails autoscale add-on manages this config var, so it's best to leave it alone. also note that if you fork a heroku app, it will copy the config vars, including rails_autoscale_url. this also results in bad things, because rails autoscale doesn't know about the forked app. if you do fork a heroku app with rails autoscale installed, be sure to remove the rails_autoscale_url config var. dashboard & settings why don't i see any data in my dashboard? most likely, the agent is not running. the troubleshooting guide will help you resolve this. can i have different autoscaling behavior on the weekend (or some other schedule)? rails autoscale does not support this natively. this request is usually a desire to have a minimum number of dynos running during busy times and scale down further during quiet times. my recommendation here is to allow your app to scale down, even during busy times. if you scale down too far—or if traffic picks up—you'll immediately scale back up. trust the autoscaler and give it a try! if you really need different settings on some kind of schedule, you can create something yourself using the rails autoscale api alongside heroku scheduler or some other job scheduler. web autoscaling what is request queue time? put simply, request queue time is the time between heroku's router receiving a request and your app beginning to process the request. it includes network time between the router and application dyno, and it includes time waiting within the dyno for an available application process. the latter is what we care about—if requests are waiting for more than a few milliseconds, there's a capacity issue. this is why rails autoscale only scales based on queue time. web requests can be slow for lots of reasons, but queue time always reflects capacity. how quickly does rails autoscale respond to a capacity issue? when your request queue time breaches your upscale threshold, rails autoscale will send an upscale request to heroku within seconds. the agent reports metrics every seconds, and it can take up to more seconds for this data to be processed. after sending the request to heroku, it'll take between and seconds (depending on the startup time for your app) for your new dyno to begin receiving requests. what if my app struggles to recover from a capacity issue? apps that receive steep spikes in traffic should consider scaling up by multiple dynos at a time. this option is available in your advanced settings. why is the queue time in rails autoscale so much higher than what i see in new relic or scout? most apm tools like new relic and scout are showing you the average for a given metric. averages might provide smoother charts for overall trends, but they aren't useful for detecting a capacity issue. rails autoscale uses the th percentile, so it will always be higher. why does my queue time spike whenever i deploy my app? unless you're using heroku's preboot feature, your app will be temporarily unavailable while it boots, such as during deploys and daily restarts. during this time, requests are routed to your web dynos, where they wait. all this waiting is reflected in your request queue time, which will likely cause an autoscale for your app. this is not a bad thing! your app autoscaling during a deploy means it'll quickly recover from the temporary downtime during boot, and of course, it'll autoscale back down once it catches up. why isn't my app scaling down when i have almost no traffic? two possible reasons: the autoscale agent starts up when your app receives its first request. if you've recently deployed or restarted—and your app is not receiving traffic—then the agent is probably not running. even if the agent is running, rails autoscale has a safeguard in place that prevents downscaling an app that hasn't reported any data for five minutes. this prevents downscaling an app that might be having a problem with the reporting agent. both of these issues are most common on staging/demo apps. as long as your app is receiving some regular traffic (such as most production apps), then you shouldn't run into this. if you need a workaround, the best approach is to use an uptime monitor to regularly ping your app. worker autoscaling can i use rails autoscale for a worker-only app (no web process)? the rails autoscale agent only runs in a web process, so you must be running at least one web dyno. even worker metrics are collected from the agent running in your web process. also note that a web request is what initially starts the agent process. if your app receives little or no web traffic, this could result in the agent never starting and never reporting metrics to rails autoscale. to work around this limitation, use an uptime monitor (freshping is a free option) to continually ping your site. can i autoscale multiple worker processes independently? yes! if you have multiple worker process types defined in your procfile, they will all be available for autoscaling. configuration for each process type is completely independent. look for "autoscale additional worker dynos" in your settings page. which rails background worker libraries are supported? sidekiq, delayed job, que, and resque (beta) are currently supported. why are my long-running jobs being terminated when scaling down? anytime you restart or shut down a worker dyno (such as downscaling, deploying, or restarting), you risk killing long-running jobs. autoscaling often magnifies this issue because you're shutting down worker dynos much more frequently. your worker backend will typically re-enqueue these jobs after being terminated, so you must ensure that your jobs are reentrant—that they can successfully re-run after a previous, interrupted run. you can also configure rails autoscale to prevent downscaling during long-running jobs. i've enabled worker autoscaling, why don't i see any data? the agent takes a snapshot of job latency (queue time) every seconds. if your job latency frequently hovers at milliseconds, this might look like missing data in rails autoscale. if you do expect to see some worker queue time in rails autoscale, it's possible the agent is not running. do you see queue time for your web dynos? if not, you're probably running a worker-only app or an app that receives very little web traffic. if you do see web queue times but no data for your worker dynos, email help@railsautoscale.com. homebloghelp & docsupdatesherokutwitterprivacyterms information wants to be free | a librarian, writer and educator reflecting on the profession and the tools we use to serve our patrons home about speaking writing contact facebook twitter google+ linkedin skype rss see full post in all the bad… some good things hi, libraries, work, work-life balance by meredith farkas on / / with comments wow, this has been a hard year. no one’s life has been untouched by between the pandemic and unrelenting proof that the social safety net has been dismantled by late-stage capitalism, the state-sanctioned murders of black and brown people and ensuing protests, the horrendous wildfires that felt like horsemen of the coming climate apocalypse, and a stressful election. it’s horrifying. … continue reading ... see full post making customizable interactive tutorials with google forms free the information!, higher ed, instruction, librarianship, online education, reference, work by meredith farkas on / / with comments in september, i gave a talk at oregon state university’s instruction librarian get-together about the interactive tutorials i built at pcc last year that have been integral to our remote instructional strategy. i thought i’d share my slides and notes here in case others are inspired by what i did and to share the amazing … continue reading ... see full post the crushing expectations on working women and where’s my fucking village? career, libraries, management, work, work-life balance by meredith farkas on / / with comments on friday and saturday, my twitter feed was full of anger and frustration over a blog post on the alsc (association for library services to children) blog. entitled “how motherhood has influenced me as a children’s librarian,” the post was problematic because it suggested (probably unintentionally) that childless children’s librarians could not connect with patrons as much or have … continue reading ... see full post recognition doesn’t have to be a zero sum game librarianship, libraries, management, speaking, work by meredith farkas on / / with comment as usual, the week the library journal movers and shakers were announced, i saw plenty of complaints about the award and, in some cases, awardees. i’ve been reading this sort of hurtful negativity since when i was named a mover and shaker (and a friend of mine wrote a blog comment calling us “the … continue reading ... see full post thoughts on work, well-being, solidarity, and advocacy in our current… situation ala, hi, librarianship, libraries, management, work, work-life balance by meredith farkas on / / with comments i have been wanting to blog for weeks. i have several blog posts i started that i just couldn’t get through. my attention span reminds me of my son’s at age when his teacher delicately suggested we should have him assessed for adhd. it rapidly jumps between various tasks at hand, my family, my … continue reading ... see full post #lismentalhealth: that time my brain and job tried to kill me about me, classic blunders, librarianship, work, work-life balance by meredith farkas on / / with comments happy lis mental health week friends! i want to start this post by recognizing someone who has done a great deal to support library workers’ mental health in the face of toxic workplaces, kaetrena davis kendrick. kaetrena has done some incredibly valuable research on low morale and toxic workplaces in librarianship and has created an awesome … continue reading ... see full post my year in books (and podcasts) about me, hi by meredith farkas on / / with comments this was a pretty good year for me. nothing particularly amazing or wonderful or eventful happened to me, though my son has been such a source of pride and light for me that i sometimes can’t believe i’m his mom. i still live in the same messed up world we all do. my migraines have actually … continue reading ... see full post when libraries and librarians pretend to be neutral, they often cause harm intellectual freedom, librarianship, libraries by meredith farkas on / / with comments two recent events made me think (again) about the toxic nature of “library neutrality” and the fact that, more often than not, neutrality is whiteness/patriarchy/cis-heteronormativity/ableism/etc. parading around as neutrality and causing harm to folks from historically marginalized groups. the insidious thing about whiteness and these other dominant paradigms is that they are largely invisible to … continue reading ... see full post thoughts at mid-career part : where to from here? about me, librarianship, mid-career, social software, work, work-life balance by meredith farkas on / / with comments this is the fifth in a series of essays. you can access the rest here, though it’s not necessary to read them all or in order. “to me, the only habit worth ‘designing for’ is the habit of questioning one’s habitual ways of seeing” -jenny odell, how to do nothing “we have to fight for this world, but we … continue reading ... see full post thoughts at mid-career part – the cult of productivity: you’re never doing enough about me, career, librarianship, mid-career, social software, work, work-life balance by meredith farkas on / / with comments this is the fourth in a series of essays. you can access the rest here, though it’s not necessary to read them all or in order. “these days, i just want to slow down. i want to pull the shutters closed and block out the world… the more time i have, the more i realize that all that … continue reading ... see full post thoughts at mid-career part – our achievement culture: what you’re doing will never be enough career, librarianship, libraries, management, mid-career, mpow, work, work-life balance by meredith farkas on / / with comments this is the third in a series of essays. you can access the rest here, though it’s not necessary to read them all or in order. of all my annoying qualities, my most self-destructive may be that if you put a ladder in front of me, i’ll try to climb it. doesn’t matter if the entire premise … continue reading ... see full post thoughts at mid-career part – ambition: you are not enough about me, career, gender, management, mid-career, work, work-life balance, writing by meredith farkas on / / with comments this is the second in a series of essays. you can access the first here, though it’s not necessary to read them all or in order: “so maybe my great ambition, such as it is, is to refrain from engagement with systems that purport to tell me what i’m worth compared to anyone else. maybe … continue reading ... see full post thoughts at mid-career part – letting go, questioning, and pathfinding about me, librarianship, mid-career, work, work-life balance by meredith farkas on / / with comments this is the first in a (probably) five-part series of essays. for about two years, until january, i felt a disturbing lack of ambition. i felt directionless and passionless; devoid of my usual neverending energy and interest. i chalked it up to mid-career malaise, but it was more than that. having only in the past … continue reading ... see full post my year in books, hi by meredith farkas on / / with comments i had such good intentions to blog more this year, but the second half of has thrown me a lot of curveballs emotionally and it’s pulled me away from a lot of the things that keep me engaged with others (funny how that seems to happen when you need people the most).books are always a … continue reading ... see full post “devaluing” the mls vs. respect for all library workers ala, librarianship, libraries, library school, management, work by meredith farkas on / / with comments i’m sure some of you remember the big push last year and early this year to require the mls for the executive director of the american library association (ala) — if you don’t, here is an article, column, and blog post about it. one big argument i kept hearing was that we needed someone who understood and … continue reading ... see full post we are atomized. we are monetized. we are ephemera. do we deserve more online? our digital future, social software, tech trends by meredith farkas on / / with comments in march and april, i took about weeks off from social media. i didn’t post anything to or look at twitter, facebook, or instagram. i’d wondered if i’d feel disconnected or feel some irresistible pull like an addict to their drug of choice. to be honest, i didn’t really feel any of that. i didn’t … continue reading ... see full post anxiety, performative allyship, and stepping away from social media about me, librarianship, social software by meredith farkas on / / with comments anxiety: hello brain: oh no anxiety: something is wrong brain: what? anxiety: that person you said that one awkward thing to three years ago probably hates you brain: you woke me up at am for this? anxiety: let’s think about the void now — christine is on editing hiatus 🖤 (@christineexists) march , friends, … continue reading ... see full post wayfinding and balance at mid-career about me, gender, librarianship, management, tenure track, work, work-life balance by meredith farkas on / / with comments it’s lis mental health week; a week focused on raising awareness of mental health. this post isn’t about mental health per se, but something that i think, for me, is very much exacerbated by anxiety and the constant negative self-appraisal that comes with it. two blog posts really resonated with me recently. sarah houghton (who i believe … continue reading ... see full post my year in books about me, ebooks, work-life balance by meredith farkas on / / with comment reading this year has been so many things for me. an escape. a way to educate myself. a way to see my own struggles in a different way through another’s story. a way to understand the struggles of others. a way to better understand where i came from. this year i think i’ve read more than … continue reading ... see full post saying goodbye to the library success wiki wikis by meredith farkas on / / with comments in july , on the heels of the successful ala annual wiki, i developed the library success wiki. here’s what i said about it then: “i would like this wiki to be a one-stop-shop for inspiration. all over the country, librarians are developing successful programs and doing innovative things with technology that no one … continue reading ... » last meredith farkas, author, information wants to be free subscribe via rss subscribe to this blog from the archives from the archives select month december november august may april february december november september august january june march february december november october september july june march january december november october september august june march january december november october september june april march february january december november september august july june may march february january december november october september august july may april march january december november october september august july june may april february january december october september august july june may march february january december october september august july june may april march february january november october august july june april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november october september august july june may april march february january december november categories about me ala american libraries assessment blogging book career classic blunders comment community college libraries community colleges ebooks election farce free the information! gender general hi higher ed inspiring stuff instruction intellectual freedom job search knowledge management librarianship libraries library school librarydayinthelife management mid-career mpow online education open access open source our digital future random reference research rss and syndication screencasting search social bookmarking social software speaking tech trends tenure track vermont wikis work work-life balance writing recent comments meredith farkas on in all the bad… some good things linda on in all the bad… some good things michelle kidd tackabery on contact dana on making customizable interactive tutorials with google forms meredith farkas on making customizable interactive tutorials with google forms most popular posts the essence of library . ? ( ) skills for the st century librarian ( ) keeping it real ( ) libraries in social networking software ( ) ebooks and libraries: a stream of concerns ( ) disclaimer this blog contains the author’s personal thoughts, which do not necessarily reflect the views of her employer. comments the author reserves the right to delete any comments she deems offensive, irrelevant, fraudulent, or blatant advertisements. go to top new findings from our investigation of sunburst - orange matter solarwinds.com home categories techpod video solarfocus brains networks systems database security applications itsm search home categories networks systems database security applications itsm techpod video solarfocus brains home > new findings from our investigation of sunburst new findings from our investigation of sunburst by sudhakar ramakrishna january , | solarfocus since the cyberattack on our customers and solarwinds, we have been working around the clock to support our customers. as we shared in our recent update, we are partnering with multiple industry-leading cybersecurity experts to strengthen our systems, further enhance our product development processes, and adapt the ways that we deliver powerful, affordable, and secure solutions to our customers. we are working with our counsel, dla piper, crowdstrike, kpmg, and other industry experts to perform our root cause analysis of the attack. as part of that analysis, we are examining how the sunburst malicious code was inserted into our orion platform software and once inserted, how the code operated and remained undetected. today we are providing an update on the investigation thus far and an important development we believe brings us closer to understanding how this serious attack was carried out. we believe we have found a highly sophisticated and novel malicious code injection source the perpetrators used to insert the sunburst malicious code into builds of our orion platform software. we recognize the software development and build process used by solarwinds is common throughout the software industry, so we believe that sharing this information openly will help the industry guard against similar attacks in the future and create safer environments for customers. the security of our customers and our commitment to transparency continue to guide our work in these areas and going forward. highly sophisticated and complex malware designed to circumvent threat detection as we and industry experts have noted previously, the sunburst attack appears to be one of the most complex and sophisticated cyberattacks in history. the u.s. government and many private-sector experts have stated the belief that a foreign nation-state conducted this intrusive operation as part of a widespread attack against america’s cyberinfrastructure. to date, our investigations have not independently verified the identity of the perpetrators. analysis suggests that by managing the intrusion through multiple servers based in the united states and mimicking legitimate network traffic, the attackers were able to circumvent threat detection techniques employed by both solarwinds, other private companies, and the federal government. the sunburst malicious code itself appears to have been designed to provide the perpetrators a way to enter a customer’s it environment. if exploited, the perpetrators then had to avoid firewalls and other security controls within the customer’s environment. kpmg and crowdstrike, working together with the solarwinds team, have been able to locate the malicious code injection source. we have reverse-engineered the code responsible for the attack, enabling us to learn more about the tool that was developed and deployed into the build environment. this highly sophisticated and novel code was designed to inject the sunburst malicious code into the solarwinds orion platform without arousing the suspicion of our software development and build teams. we encourage everyone to visit this blog post, authored by the crowdstrike team, which provides additional details into these findings and other technical aspects of this attack, and contains valuable information intended to help the industry better understand attacks of this nature. as we discussed in our previous post, we hope that this event ushers in a new level of collaboration and information sharing within the technology industry to address and prevent similar attacks in the future. our concern is that right now similar processes may exist in software development environments at other companies throughout the world. the severity and complexity of this attack has taught us that more effectively combatting similar attacks in the future will require an industry-wide approach as well as public-private partnerships that leverage the skills, insight, knowledge, and resources of all constituents. we want to be a part of that solution, which is why we are sharing this information with the broader community, and we will continue to share progress as we assimilate this information into our go-forward practices. our investigations to date we are actively working with law enforcement, the intelligence community, governments, and industry colleagues in our and their investigations. as we recently disclosed, we even shared all of our proprietary code libraries that we believed to have been affected by sunburst to give security professionals the information they needed in their research. our current timeline for this incident begins in september , which is the earliest suspicious activity on our internal systems identified by our forensic teams in the course of their current investigations. the subsequent october version of the orion platform release appears to have contained modifications designed to test the perpetrators’ ability to insert code into our builds an updated version of the malicious code injection source that inserted the sunburst malicious code into orion platform releases starting on february , . the perpetrators remained undetected and removed the sunburst malicious code from our environment in june . during that time, through to today, solarwinds investigated various vulnerabilities in its orion platform. it remediated or initiated the process of remediating vulnerabilities, a regular process that continues today. however, until december , the company did not identify any vulnerabilities as what we now know as sunburst. on december , , we were informed of the cyberattack and moved swiftly to notify and protect our customers and to investigate the attack in collaboration law enforcement, intelligence and governments. as part of our ongoing efforts to protect our customers and investigate the sunburst attack, we are reviewing historical and current customer inquiries that might contribute to a better understanding of the attack. to date, we have identified two previous customer support incidents during the timeline referenced above that, with the benefit of hindsight, we believe may be related to sunburst. we investigated the first in conjunction with our customer and two third-party security companies. at that time, we did not determine the root cause of the suspicious activity or identify the presence of the sunburst malicious code within our orion platform software. the second incident occurred in november, and similarly, we did not identify the presence of the sunburst malicious code. we are still investigating these incidents and are sharing information related to them with law enforcement to support investigation efforts. we will continue our investigations to help ensure our products and internal systems are secure and to provide information that we hope leads to the identification of the perpetrators and the prevention of these types of attacks in the future. we also plan to continue to share our broader findings with the industry at large in the hope that everyone is better able protect themselves and deliver more secure solutions to their customers. ********************************************************************************* this blog post contains "forward-looking" statements, which are subject to the safe harbor provisions of the private securities litigation reform act of , including statements regarding solarwinds’ investigation into the recent sunburst attack, the high-level timeline provided above and the company’s findings to date, solarwinds’ understanding of the nature, source and duration of the attack and solarwinds’ plans to further investigate the attack, ensure our products and internal systems are secure and provide information regarding its findings. the information in this blog post is based on management's beliefs and assumptions and on information currently available to management, which may change as solarwinds continues to address the vulnerability in its products, investigate the sunburst attack and related matters and as new or different information is discovered about these matters or generally. forward-looking statements include all statements that are not historical facts and may be identified by terms such as "aim," "anticipate," "believe," "can," "could," "seek," "should," "feel," "expect," "will," "would," "plan," "intend," "estimate," "continue," "may," or similar expressions and the negatives of those terms. forward-looking statements involve known and unknown risks, uncertainties and other factors that may cause actual results, performance or achievements to be materially different from any future results, performance or achievements expressed or implied by the forward-looking statements. factors that could cause or contribute to such differences include, but are not limited to, (a) the discovery of new or different information regarding the sunburst attack and related security incidents or of additional vulnerabilities within, or attacks on, solarwinds’ products, services and systems, (b) the possibility that solarwinds’ mitigation and remediation efforts with respect to the sunburst attack and related security incidents may not be successful, (c) the possibility that customer, personnel or other data was exfiltrated as a result of the sunburst attack and related security incidents, (d) numerous financial, legal, reputational and other risks to solarwinds related to the sunburst attack and related security incidents, including risks that the incidents may result in the loss, compromise or corruption of data, loss of business, severe reputational damage adversely affecting customer or vendor relationships and investor confidence, u.s. or foreign regulatory investigations and enforcement actions, litigation, indemnity obligations, damages for contractual breach, penalties for violation of applicable laws or regulations, significant costs for remediation and the incurrence of other liabilities, (e) risks that solarwinds’ insurance coverage, including coverage relating to certain security and privacy damages and claim expenses, may not be available or sufficient to compensate for all liabilities solarwinds incurs related to these matters, (f) the possibility that solarwinds’ steps to secure its internal environment, improve its product development environment and ensure the security and integrity of the software that it delivers to customers may not be successful or sufficient to protect against threat actors or cyberattacks and (g) such other risks and uncertainties described more fully in documents filed with or furnished to the u.s. securities and exchange commission by solarwinds, including the risk factors discussed in solarwinds’ annual report on form -k for the period ended december , filed on february , , its quarterly report on form -q for the quarter ended march , filed on may , , its quarterly report on form -q for the quarter ended june , filed on august , and its quarterly report on form -q for the quarter ended september , filed on november , . all information provided in this blog post is as of the date hereof and solarwinds undertakes no duty to update this information except as required by law. share: tags sudhakar ramakrishna sudhakar ramakrishna joined solarwinds as president and chief executive officer in january . he is a global technology leader with nearly years of experience… read more tweets hours ago solarwinds @solarwinds watch now on-demand: solarwinds president and ceo sudhakar ramakrishna, cybersecurity expert, stanford internet obs… t.co/etavtz h n reply retweet like days ago solarwinds @solarwinds reporting for the sake of reporting is like buying food but not eating. here are tips to get the most out of… t.co/jyzwc gtqz reply retweet like days ago solarwinds @solarwinds as the role of the #itpro continues to evolve, what are the top skills needed? find out in this article. t.co/zd nfacrr reply retweet like previous our plan for a safer solarwinds and customer community next call them “soft” skills. one. more. time. you may also like committed to security: solarwinds database performance monitor and soc type solarwinds makes itsm debut with solarwinds service desk solarwinds update on security vulnerability solarwinds at microsoft ignite we’re geekbuilt ™. developed by network and systems engineers who know what it takes to manage today’s dynamic it environments, solarwinds has a deep connection to the it community. the result? it management products that are effective, accessible, and easy to use. company career center resource center email preference center for customers for government for partners gdpr resource center legal documents security information documentation & uninstall information © solarwinds worldwide, llc. all rights reserved. we're geekbuilt ™. developed by network and systems engineers who know what it takes to manage today's dynamic it environments, solarwinds has a deep connection to the it community. the result? it management products that are effective, accessible, and easy to use. company career center resource center email preference center for customers for government for partners gdpr resource center legal documents security information documentation & uninstall information © solarwinds worldwide, llc. all rights reserved. solarwinds uses cookies on its websites to make your online experience easier and better. by using our website, you consent to our use of cookies. for more information on cookies, see our cookie policy. the programming historian | programming historian donate to the programming historian today! the programming historian about about ph project team research privacy policy contribute overview feedback reviewer guidelines author guidelines translator guidelines editor guidelines lesson requests technical contributions lessons support us institutional partnership programme individual supporters our supporters blog en es fr pt the programming historian the programming historian enter the programming historian (the initial english version) lessons issn: - we publish novice-friendly, peer-reviewed tutorials that help humanists learn a wide range of digital tools, techniques, and workflows to facilitate research and teaching. entrar the programming historian en español lecciones issn: - publicamos tutoriales revisados por pares dirigidos a humanistas que quieran aprender una amplia gama de herramientas digitales, técnicas computacionales y flujos de trabajo útiles para investigar y enseñar. entrez the programming historian en français leçons issn: - nous publions des tutoriels évalués par des pairs qui permettent l'initiation à et l'apprentissage d'un large éventail d'outils numériques, de techniques et de flux de travail pour faciliter la recherche et l'enseignement en sciences humaines et sociales. entrar the programming historian em português lições issn: - publicamos tutoriais acessíveis, avaliados por pares, que ajudam os humanistas a aprender uma ampla gama de ferramentas digitais, técnicas computacionais e metodologias de trabalho que facilitam a pesquisa e o ensino. the programming historian (issn: - ) is released under a cc-by license. this project is administered by proghist limited, company number . issn - (english) issn - (spanish) issn - (french) issn - (portuguese) hosted on github site last updated february rss feed subscriptions see page history make a suggestion lesson retirement policy translation concordance pinboard bookmarks tagged code lib pinboard recent ‧ popular ‧ tour ‧ howto log in code lib « earlier listserv . - code lib archives rt @kiru: i forgot to post the call earlier: the code lib journal () is looking for volunteers to join its editorial committee. deadline: oct. #code lib code lib from twitter september by miaridge copy to mine - c l [ ] future role of libraries in researcher workflows - google slides research-lifecycle code lib publish scholarly-communication march by elibtronic copy to mine twitter new issue of the the #code lib journal published. some terrific looking papers, including a review of pids for heri… code lib from twitter_favs february by aarontay copy to mine ( ) https://journal.code lib.org/ rt @kiru: i am very happy to announce the publication of the @code lib journal issue # : webscraping… code lib from twitter february by miaridge copy to mine the code lib journal – column: we love open source software. no, you can’t have our code librarians are among the strongest proponents of open source software. paradoxically, libraries are also among the least likely to actively contribute their code to open source projects. this article identifies and discusses six main reasons this dichotomy exists and offers ways to get around them. code lib library libt opensource finalproject december by pfhyper copy to mine the code lib journal – barriers to initiation of open source software projects in libraries libraries share a number of core values with the open source software (oss) movement, suggesting there should be a natural tendency toward library participation in oss projects. however dale askey’s code lib column entitled “we love open source software. no, you can’t have our code,” claims that while libraries are strong proponents of oss, they are unlikely to actually contribute to oss projects. he identifies, but does not empirically substantiate, six barriers that he believes contribute to this apparent inconsistency. in this study we empirically investigate not only askey’s central claim but also the six barriers he proposes. in contrast to askey’s assertion, we find that initiation of and contribution to oss projects are, in fact, common practices in libraries. however, we also find that these practices are far from ubiquitous; as askey suggests, many libraries do have opportunities to initiate oss projects, but choose not to do so. further, we find support for only four of askey’s six oss barriers. thus, our results confirm many, but not all, of askey’s assertions. code lib library libt opensource finalproject december by pfhyper copy to mine twitter rt @kiru: the #code lib journal's issue ( / ) has been just published: . worldcat search api, go… code lib from twitter november by jbfink copy to mine twitter rt @mjingle: who's excited for the next #code lib conference?! it will be in pittsburgh, pa from march - . is your org interes… code lib from twitter november by jbfink copy to mine attempto project nlp basic cnl computationallinguistics controlledlanguage controlled_language code lib compsci english knowledgerepresentation september by blebo copy to mine twitter when our grandchildren ask about the great #code lib irc battle of the tisane, we will serve them both tea and coff… code lib from twitter_favs august by danbri copy to mine code lib recap – bloggers! code lib digitallibraries research saa archives july by geephroh copy to mine digital technologies development librarian | nc state university libraries we're hiring a digital technologies development librarian @ncsulibraries ! #job #libjobs #code lib #dlf #libtech dlf libtech code lib job libjobs from twitter_favs july by cdmorris copy to mine twitter ) all the men who want to preserve the idea of a #code lib discussion space as one that's free of such topics as s… code lib from twitter_favs july by jbfink copy to mine google refine cheat sheet (code lib) openrefine code lib how-to cheatsheet may by psammead copy to mine untitled (https://www.youtube.com/watch?v=icblvnchpnw) code lib southeast happening today! live stream starting at : am eastern. #code libse #code lib code libse code lib from twitter_favs may by cdmorris copy to mine twitter it occurs to me the #code lib statement of support for chris bourg, , offers a better model… code lib from twitter april by lbjay copy to mine github - code lib/c l -keynote-statement: code lib community statement in support of chris bourg it occurs to me the #code lib statement of support for chris bourg, , offers a better model… code lib from twitter april by lbjay copy to mine twitter now that the #code lib discord is up & running, i'm contemplating leaving slack overall, with exception for plannin… code lib from twitter_favs march by jbfink copy to mine ( ) https://twitter.com/palcilibraries/status/ /photo/ talking privacy and ra at #c l with dave lacy from @templelibraries #code lib c l code lib from twitter_favs february by cdmorris copy to mine scope: an access interface for dips from archivematica archives code lib february by sdellis copy to mine « earlier related tags accessibility algorithms analytics archives art article awesome basic bias blog blogs c l c l c l c l c ln c lse career center cheatsheet cnl code code lib- code libse code libse compsci computationallinguistics conference controlled_language controlledlanguage crowdsourcing culture dev dh dh dighum digitalhumanities digitallibraries diversity dlf docker dpla english excel failure finalproject floss github harlow history how-to inclusion ipfs ischoolui job journal keynote knowledgerepresentation libjobs libraries library-tech library library_technology libt libtech libtechwomen libux lightning_talks lita litaux lodlam lt mansplaining mashcat memorylab musetech news nlp open open_source openrefine opensource philadelphia philly presentation privacy programming publish python research-lifecycle research resources reuse saa scholarly-communication search security sharing software source spreadsheet spreadsheets tabularformats teamwork tech technology unread ux washingtondc webarchive webdev work-life xml xpath yay copy this bookmark: description: - tags: grab all tags - clear tags to read © nine fives software. problems or questions? contact . tos ‧ privacy ‧ about ‧ blog ‧ faq ‧ resources ‧ security github pages | websites for you and your projects, hosted directly from your github repository. just edit, push, and your changes are live. websites for you and your projects. hosted directly from your github repository. just edit, push, and your changes are live. pages help what is github pages? ready to get started? build your own site from scratch or generate one for your project. you get one site per github account and organization, and unlimited project sites. let‘s get started. user or organization site project site create a repository head over to github and create a new public repository named username.github.io, where username is your username (or organization name) on github. if the first part of the repository doesn’t exactly match your username, it won’t work, so make sure to get it right. what git client are you using? a terminal github desktop i don't know download github desktop github desktop is a great way to use git and github on macos and windows. download github desktop clone the repository go to the folder where you want to store your project, and clone the new repository: ~$git clone https://github.com/username/username.github.io clone the repository click the "set up in desktop" button. when the github desktop app opens, save the project. if the app doesn't open, launch it and clone the repository from the app. clone the repository after finishing the installation, head back to github.com and refresh the page. click the "set up in desktop" button. when the github desktop app opens, save the project. if the app doesn't open, launch it and clone the repository from the app. hello world enter the project folder and add an index.html file: ~$cd username.github.io ~$echo "hello world" > index.html create an index file grab your favorite text editor and add an index.html file to your project: index.html hello world

i'm hosted with github pages.

push it add, commit, and push your changes: ~$git add --all ~$git commit -m "initial commit" ~$git push -u origin main commit & publish enter the repository, commit your changes, and press the publish button. …and you're done! fire up a browser and go to https://username.github.io. use a theme, or start from scratch? you have the option to start with one of the pre-built themes, or to create a site from scratch. choose a theme start from scratch repository settings head over to github.com and create a new repository, or go to an existing one. click on the settings tab. theme chooser scroll down to the github pages section. press choose a theme. pick a theme choose one of the themes from the carousel at the top. when you're done, click select theme on the right. edit content use the editor to add content to your site. commit enter a commit comment and click on commit changes below the editor. create an index file head over to github.com and create a new repository, or go to an existing one. click on the create new file button. hello world name the file index.html and type some html content into the editor. commit the file scroll to the bottom of the page, write a commit message, and commit the new file. repository settings click on the settings tab and scroll down to the github pages section. then select the main branch source and click on the save button. …and you're done! fire up a browser and go to http://username.github.io/repository. now that you’re up and running, here are a few things you should know. blogging with jekyll using jekyll, you can blog using beautiful markdown syntax, and without having to deal with any databases. learn how to set up jekyll. custom urls want to use your own custom domain for a github pages site? just create a file named cname and include your url. read more. guides learn how to create custom pages, use submodules, and learn more about github pages. status api training shop blog about © github, inc. terms privacy security contact none none nick ruest search nick ruest home c.v. posts presentations publications projects visualizations music contact nick ruest associate librarian york university biography nick ruest is an associate librarian in the digital scholarship infrastructure department at york university, co-principal investigator of the andrew w. mellon foundation funded the archives unleashed project, co-principal investigator of the sshrc grant “a longitudinal analysis of the canadian world wide web as a historical resource, - ”, and co-principal investigator of the compute canada research platforms and portals web archives for longitudinal knowledge. at york university, he oversees the libraries’ preservation initiatives, along with creating and implementing systems that support the capture, description, delivery, and preservation of digital objects having significant content of enduring value. he was previously active in the islandora and fedora communities, serving as project director for the islandora claw project, member of the islandora foundation’s roadmap committee and board of directors, and contributed code to the project. he has also served as the release manager for islandora and fedora, the moderator for the ocul digital curation community, the president of the ontario library and technology association, and president of mcmaster university academic librarians’ association. interests web archives data analytics distributed systems information retrieval digital preservation education mlis, wayne state university bachelor of arts political science, minor in history, university of michigan-dearborn recent publications more publications from archive to analysis: accessing web archives at scale through a cloud-based interface nick ruest, samantha fritz, ryan deschamps, jimmy lin, ian milligan pdf building community at distance: a datathon during covid- samantha fritz, ian milligan, nick ruest, jimmy lin pdf project content-based exploration of archival images using neural networks tobi adewoye, xiao han, nick ruest, ian milligan, samantha fritz, jimmy lin pdf project video the archives unleashed project: technology, process, and community to improve scholarly access to web archives nick ruest, jimmy lin, ian milligan, samantha fritz pdf project video we could, but should we? ethical considerations for providing access to geocities and other historical digital collections jimmy lin, ian milligan, douglas w. oard, nick ruest, katie shilton pdf the archives unleashed project: technology, process, and community to improve scholarly access to web archives nick ruest, jimmy lin, ian milligan, samantha fritz pdf solr integration in the anserini information retrieval toolkit ryan clancy, toke eskildsen, nick ruest, jimmy lin pdf dataset project building community and tools for analyzing web archives through datathons ian milligan, nathalie casemajor, samantha fritz, jimmy lin, nick ruest, matthew s. weber, nicholas worby pdf project slides scalable content-based analysis of images in web archives with tensorflow and the archives unleashed toolkit hsiu-wei yang, linqing liu, ian milligan, nick ruest, jimmy lin pdf dataset project poster the archives unleashed notebook: madlibs for jumpstarting scholarly exploration ryan deschamps, nick ruest, jimmy lin, samantha fritz, ian milligan pdf project poster recent & upcoming talks more talks lowering the barrier to access: the archives unleashed cloud project jun , the web that was: archives, traces, reflections resaw slides project sustainability and research platforms: the archives unleashed cloud project jun , international internet preservation consortium web archiving conference slides see a little warclight: building an open-source web archive portal with project blacklight jun , international internet preservation consortium web archiving conference slides web archives analysis at scale with the archives unleashed cloud (with ian milligan) apr , cni spring membership meeting slides oh, i get by with a little help from my friends: interdisciplinary web archive collaboration. feb , workshop on quantitative analysis and the digital turn in historical studies slides make it walk! may , archives association of ontario slides hot tips to boost your interdisciplinary web archive collaboration! apr , lewis & ruth sherman centre for digital scholarship speak series slides the world is a beautiful and terrible place mar , national forum on ethics and archiving the web slides video boosting your interdisciplinary web archive collaboration feb , bc research libraries group lecture series slides twitter and web archive analysis at scale feb , data love-in : a day of data management planning and conversations slides recent posts more posts four fucking years of donald trump nearly four years ago i decided to start collecting tweets to donald trump out of morbid curiosity. if i was a real archivist, i would … jan , min read enhancing archives unleashed toolkit usability with spark-submit originally posted here. over the last month, we have put out several toolkit releases. the primary focus of the releases has been … may , min read cloud-hosted web archive data: the winding path to web archive collections as data originally posted here. web archives are hard to use, and while the past activities of archives unleashed has helped to lower these … feb , min read twut. wait, wut? twut? originally posted here. introduction a few of the archives unleashed team members have a pretty in-depth background of working with … dec , min read exploring #elxn twitter data introduction a few years ago library archives canada, ian milligan and i collected tweets from the nd canadian federal election. ian … nov , min read projects archives unleashed project archives unleashed aims to make petabytes of historical internet content accessible to scholars and others interested in researching the recent past. supported by a grant from the andrew w. mellon foundation, we will be developing web archive search and data analysis tools to enable scholars and librarians to access, share, and investigate recent history since the early days of the world wide web. web archives for historical research our research focuses on both web histories - writing about the recent past as reflected in web archives - as well as methodological approaches to understanding these repositories. islandora claw islandora claw is the next generation of islandora. fedora repository fedora is the flexible, modular, open source repository platform with native linked data support. visualizations , #elxn images dear donald; may - january . happy new year to everyone, including the haters and the fake news media! totally clears the president. thank you! , , audio cover images from the internet archive , , #womensmarch images a month of tweets at @realdonaldtrump islandora claw development , , #panamapapers images , #ymmfire images , #thehip, #hipinkingston images , , #elxn images , #makedonalddrumpfagain images #elxn wordclouds by day anon development visualization islandora .x- . development music unfavourable offerings sloppy a-sides, last of the worst strange delights the humans soundtrackpro unlimited the achievements the potions - regular release ep sloppy b-sides, first of the worst surnom de gorille the potions @ the lifton matterwave foci . audio wardrobe's jacuzzi contact ruestn@yorku.ca cc-by · powered by the academic theme for hugo. cite × copy download shelley gullikson shelley gullikson usability and user experience in academic libraries. mine mostly. web librarians who do ux: access presentation this is the text (approximately) of my presentation from the virtual access conference on oct. , , &# ;web librarians who do ux: we are so sad, we are so very very sad.&# ; last year, i was doing interviews with library people who do user experience work and noticed that people who were primarily focused on the &# ; continue reading web librarians who do ux: access&# ;presentation the tpl debacle: values vs people i can’t stop thinking about the situation at tpl. the short version is that the library has accepted a room rental from an anti-trans speaker, and despite outcry from trans people and their allies, despite a petition and a boycott by writers, despite their own policy on room rentals not allowing events that promote discrimination, &# ; continue reading the tpl debacle: values vs&# ;people library workers and resilience: more than self-care an article in the globe and mail this spring about resilience was a breath of fresh air—no talk about &# ;grit&# ; or bootstraps or changing your own response to a situation. it was written by michael ungar, the canada research chair in child, family, and community resilience at dalhousie university and leader of the resilience research &# ; continue reading library workers and resilience: more than&# ;self-care research projects: call for help i&# ;m on a year-long sabbatical as of july and excited to get started on a few different research projects. for two of the projects, i&# ;m going to need some help from the uxlibs/libux community. in one of them, i want to look at love letters that users have written to academic libraries so i &# ; continue reading research projects: call for&# ;help uxlibsv: notes five years of uxlibs &# ; hurrah! let&# ;s dive straight in. barriers to ux design: andy priestner andy kicked off the conference with his address about why he thinks not many of us are moving beyond research reports when it comes to doing ux work in our libraries: we see research as the finish line. ux &# ; continue reading uxlibsv: notes website refresh: first round of iterative testing as i mentioned in my last post, we&# ;re doing a design refresh of our library website, with a goal to make it &# ;beautiful.&# ; as such, we&# ;re not touching much of the organization. but of course we have to pay attention to not just how the information is categorized but also where it appears on the &# ; continue reading website refresh: first round of iterative&# ;testing user research: beautiful websites? my university librarian has asked for a refresh of the library website. he is primarily concerned with the visual design; although he thinks the site meets the practical needs of our users, he would like it to be &# ;beautiful&# ; as well. eep! i&# ;m not a visual designer. i was a little unsure how to even &# ; continue reading user research: beautiful&# ;websites? access : a ux perspective i started my access conference experience with a meetup of library people interested in ux. there were only five of us, but we had good conversations about research ethics boards and ux research, about being a ux team of one, and about some of the projects we&# ;ll be working on in the coming year. &# ; continue reading access : a ux&# ;perspective ux from a technical services point of view this the text of a presentation i did last year at the access conference in regina. emma and i had plans to write this up as a paper, but life intervened and that didn&# ;t happen. i wanted to keep some record beyond the video of the presentation, so here it is. this morning i’m going &# ; continue reading ux from a technical services point of&# ;view adding useful friction to library ux: ideas from uxlibs workshop participants at this years uxlibs conference, i led a workshop on adding useful friction to the library user experience. i&# ;ve already posted my text of the workshop, but i told the participants that i also wanted to share the ideas that they came up with during the course of the workshop. the ideas were generated around &# ; continue reading adding useful friction to library ux: ideas from uxlibs workshop&# ;participants software development at royal danish library software development at royal danish library a peekhole into the life of the software development department at the royal danish library which type bug? a light tale of bug hunting an out of memory problem with solrcloud. the setup and the problem at the royal danish library we provide full text search for the danish netarchive. the heavy lifting is done in a single &# ; continue reading &# ; touching encouraged (an ongoing story) ongoing experiments with a large touch screen providing access to cultural heritage material continue reading &# ; docvalues jump tables in lucene/solr lucene/solr is about to be released. among a lot of other things is brings lucene- , written by your truly with a heap of help from adrien grand. lucene- introduces jump-tables for docvalues, is all about performance and brings speed-ups &# ; continue reading &# ; faster docvalues in lucene/solr + this is a fairly technical post explaining lucene- and its implications on lucene, solr and (qualified guess) elasticsearch search and retrieval speed. it is primarily relevant for people with indexes of m+ documents. teaser we have a solr setup for &# ; continue reading &# ; prebuild big data word vec dictionaries prebuild and trained word vec dictionaries ready for use two different prebuild big data word vec dictionaries has been added to loar (library open access repository) for download. these dictionaries are build from the text of , e-books from project gutenberg &# ; continue reading &# ; solrwayback software bundle has been released the solrwayback software bundle can be used to search and playback archived webpages in warc format. it is an out of the box solution with index workflow, solr and tomcat webserver and a free text search interface with playback functionality. &# ; continue reading &# ; visualising netarchive harvests &# ; an overview of website harvest data is important for both research and development operations in the netarchive team at det kgl. bibliotek. in this post we present a recent frontend visualisation widget we have made. from the solrwayback machine &# ; continue reading &# ; solrwayback machine another &# ;google innovation week&# ; at work has produced the solrwayback machine. it works similar to the internet archive: wayback machine (https://archive.org/web/) and can be used to show harvested web content (warc files). the danish internet archive has over billion harvested &# ; continue reading &# ; juxta – image collage with metadata creating large collages of images to give a bird&# ;s eye view of a collection seems to be gaining traction. two recent initiatives: the new york public library has a very visually pleasing presentation of public domain digitizations, but with a &# ; continue reading &# ; automated improvement of search in low quality ocr using word vec this abstract has been accepted for digital humanities in the nordic countries nd conference, http://dhn .eu/ in the danish newspaper archive[ ] you can search and view million newspaper pages. the search engine[ ] uses ocr (optical character recognition) from scanned pages &# ; continue reading &# ; , , images tweeted at donald trump | nick ruest search nick ruest home c.v. posts presentations publications projects visualizations music contact , , images tweeted at donald trump feb , min read juxta a couple years ago i wrote about a method for creating a collage out of . m images collected from the canadian federal election twitter dataset. that method was very resource intensive in terms of the amount of temporary disk storage required to create the collage. as the number of images in a given collage increased, the amount of temporary disk space scaled exponentially; . t for . m #exln images, and ~ t for . m #womensmarch images. once i shared the collage there was some interesting discussion around it, and a feature that came up a few times that i found fascinating, but had no idea how to implement using the montage method was linking back to the tweet a given image came from. but, luckily a colleague in denmark had something in mind: juxta. why does the juxta method work better? instead of creating a massive single image from all the base images, then generating tiles to display with openseadragon, it skips the creating a giant image part, and just creates the tiles! even better, if you’re working with twitter data, you’re able to link back to the original tweet a given image came from. if you want learn more about juxta, toke has a wonderful readme, and a blog post. pic.twitter.com/nbljeiqyfs — todd ouellette (@tetshui) january , about the dataset i began collecting tweets directed at donald trump (@realdonaldtrump) in may of documenting the now’s twarc. tweets from may , - june , of the dataset used a combination of the filter (streaming) api and search api. collecting via the filter api failed on june , . from june , forward only the search api was used to collect. this is done via a cron job every five days. periodically the dataset is deduplicated, tweet ids are extracted, and the public dataset is updated. the collage was created with version of the dataset: , , tweets from may , - january . about the collage the collage took about days to complete with threads on an core machine. it consists of , , images tweeted at donald trump, , , tiles, and uses , , inodes. understanding the collage; you can follow along in the image chronologically. the top left corner of the image will be the earliest images in the dataset (may ), and the bottom right corner will be the latest images in the dataset (january ). zoom in and pan around! donald trump juxta twitter big data twarc visualization nick ruest associate librarian related twitter wordcloud pipeline twitter datasets and derivative data , , womensmarch tweets january - , , , elxn images a look at , , paris bataclan parisattacks porteouverte tweets cc-by · powered by the academic theme for hugo. cite × copy download welcome to your new hq | slack skip to main content product features channels integrations security slack connect solutions customers download slack enterprise resources pricing search sign intalk to salestry for free product features channels integrations security slack connect solutions customers download slack enterprise resources pricing sign in sign indownload slack slack makes it downright pleasant to work together try for free what is slack? video of zoe wait... what is slack? it’s a new way to communicate—faster than email and more focused than chat. it brings your whole team together see what slack looks like for: small companies medium companies large companies direct messagesvideo of emily emily video of will will video of toby toby arcadio buendía : am zoe maxwell : am arcadio buendía : am pdf uploaded this file: direct messagesvideo of emily emily arcadio video of mia mia jagdeep, lisa video of marcus marcus arcadio buendía : am zoe maxwell : am arcadio buendía : am pdf uploaded this file: direct messagesvideo of emily emily arcadio video of mia mia jagdeep, lisa financebot video of elena elena lisa video of marcus marcus arcadio buendía : am zoe maxwell : am arcadio buendía : am pdf uploaded this file: there’s a space for every project channels keep work organized. you don’t have to think twice about where to go to ask a question or make a decision. see how teams organize slack: molly moon’s homemade ice creamvalley behavioral healthfrontline foods cole haansplunklyft td ameritradetargetarizona state university see all customer stories even projects with other companies slack connect lets two companies move as quickly as one. hatch a partnership. close the deal. build something new. learn more about slack connect and you can chat face to face, with just a click and video calling, too in short: it’s a more human way to work try for freetalk to sales %of users feel more connected to their teams two teammates having a conversation in a channel %of users depend on slack to get work done globe with people messaging people around the world %of users feel their ability to work remotely has improved why slack? slack vs. email channels engagement scale watch the demo product features integrations enterprise solutions pricing plans paid vs. free resources partners developers apps blog help center events company about us leadership investor relations news media kit careers status privacy & terms contact us change region selecting a different region will change the language and content of slack.com. americas latinoamérica (español)brasil (português)united states (english) europe deutschland (deutsch)españa (español)france (français)italia (italiano)united kingdom (english) asia pacific india (english)日本 (日本語)한국어 change region download slack © copyright slack technologies, inc. all rights reserved. various trademarks held by their respective owners. planet code lib http://planet.code lib.org acrl techconnect acrl techconnect aaron schmidt influx library user experience alf eaton, alf hublog andrew pace hectic pace andromeda yelton andromeda yelton archival connections archival connections archives unleashed project archives unleashed - medium ariadne magazine table of contents: issue bethany nowviskie bethany nowviskie bohyun kim library hat brown university library digital technologies projects brown university library digital technologies casey bisson maisonbisson chris beer blog.cbeer.info code lib code lib journal the code lib journal conal tuohy conal tuohy's blog coral sheldon-hess coral sheldon-hess crossref coyle's information cynthia ng learning (lib)tech d-lib d-lib magazine dan cohen dan cohen dan scott coffee|code: dan scott's blog - coding david fiander rapid communications david rosenthal dshr's blog digital library federation dlf district dispatch district dispatch dominic bordelon dominic bordelon duraspace news news – duraspace.org ed summers inkdroid equinox software the open source experts eric hellman go to hellman eric lease morgan planet eric lease morgan erin white libraries – erin white europeana » technical evergreen ils evergreen ils foss lib recent releases releases foss lib updated packages updated packages | foss lib future archives (bodleian library) futurearch, or the future of archives... galen charlton meta interchange hangingtogether hanging together harvard library innovation lab the harvard library innovation lab hugh cayless scriptio continua hugh rundle information flaneur ian davis internet alchemy in the library, with the lead pipe in the library with the lead pipe information technology and libraries information technology and libraries islandora islandora jakob voss en – jakoblog jason ronallo preliminary inventory of digital collections by jason ronallo jeremy frumkin the digital librarian jez cope erambler jodi schneider jodischneider.com/blog john mark ockerbloom everybody's libraries jonathan brinley x + jonathan rochkind bibliographic wilderness journal of web librarianship tandf: journal of web librarianship: table of contents karen g. schneider free range librarian lita lita blog libux library user experience community - medium libx libx library tech talk (u of michigan) library tech talk - u-m library librarything (thingology) the thingology blog librecat/catmandu blog catmandu lorcan dempsey lorcan dempsey's weblog lucidworks lucidworks lukas koster commonplace.net m. ryan hess fail!lab manage metadata (diane hillmann and jon phipps) metadata matters mark e. phillips mark e. phillips journal mark matienzo posts on mark a. matienzo mashcat mashcat max planck digital library max planck vlib news meredith farkas information wants to be free mita williams librarian of things nypl special collection nypl blogs: specialcollections.txt nick ruest nick ruest nicole engard what i learned today… oclc dev network oclc developers network news open knowledge foundation open knowledge foundation blog open library the open library blog peter murray disruptive library technology jester peter sefton ptsefton.com raffaele messuti literary machines ranti junus ranti. centuries.org raymond yee data unbound richard wallis roy tennant roy tennant: digital libraries – the digital shift samvera samvera shelley gullikson shelley gullikson state library of denmark software development at royal danish library stuart yeates open source exile suzanne chapman userslib tara robertson tara robertson consulting ted lawless ted lawless terry reese terry's worklog thom hickey outgoing tim ribaric elibtronic.ca villanova library technology blog falvey memorial library blog william denton miskatonic university press zbw german national library of economics zbw labs zotero zotero flickr recent uploads tagged code lib pinboard pinboard (items tagged code lib) planet code lib planet code lib david rosenthal: talk at berkeley's information access seminar andromeda yelton: archival face recognition for fun and nonprofit mita williams: weeknote ( ) david rosenthal: chromebook linux update open knowledge foundation: what is a public impact algorithm? ed summers: outgoing david rosenthal: stablecoins jonathan rochkind: product management islandora: islandora open meeting: february , hangingtogether: emerging roles for libraries in bibliometric and research impact analysis: lessons learned from the university of waterloo jez cope: glam data science network fellow travellers digital library federation: metadata during covid lucidworks: accelerate time to value for information retrieval with ai oclc dev network: planned maintenance: classify api terry reese: marcedit . .x/ . .x (beta) updates cynthia ng: choosing not to go into management (again) harvard library innovation lab: new updates to search: advanced filters open knowledge foundation: announcing a new partner for open data day mini-grants cynthia ng: reflection part : my first year at gitlab and becoming senior cynthia ng: reflection part : my second year at gitlab and on becoming senior again jonathan rochkind: rails auto-scaling on heroku hugh rundle: automation workflows with github actions and webhooks - library map part mita williams: weeknote ( ) david rosenthal: effort balancing and rate limits open knowledge foundation: launching the net zero challenge: a global pitch competition about using open data for climate action peter sefton: research data management looking outward from it peter sefton: research data management looking outward from it hugh rundle: library map part - how jonathan rochkind: managed solr saas options digital library federation: three new ndsa members duraspace news: fedora migration paths and tools project update: january open knowledge foundation: open knowledge justice programme takes new step on its mission to ensure algorithms cause no harm david rosenthal: isp monopolies lucidworks: cast a smarter net with semantic vector search digital library federation: virtual ndsa digital preservation recordings available online! terry reese: marcedit . .x/macos . .x timelines jez cope: a new font for the blog jez cope: training a model to recognise my own handwriting lucidworks: learning from einstein’s brain meredith farkas: making customizable interactive tutorials with google forms journal of web librarianship: the culture of digital scholarship in academic libraries hugh rundle: library map part - why nick ruest: four fucking years of donald trump lucidworks: consider a new application for ai in retail mita williams: weeknote ( ) jez cope: blogging by hand tara robertson: diversity, equity and inclusion core competencies: get cross functional projects done (part of ) open knowledge foundation: how to run your open data day event online in ed summers: trump's tweets lucidworks: lucidworks announces updated integration with drupal the weirdest people in the world? - pubmed this site needs javascript to work properly. please enable it to take advantage of the complete set of features! clipboard, search history, and several other advanced features are temporarily unavailable. skip to main page content covid- is an emerging, rapidly evolving situation. get the latest public health information from cdc: https://www.coronavirus.gov get the latest research information from nih: https://www.nih.gov/coronavirus find ncbi sars-cov- literature, sequence, and clinical content: https://www.ncbi.nlm.nih.gov/sars-cov- / national institutes of health national library of medicine national center for biotechnology information ncbi homepage log in show account info close account logged in as: username dashboard (my ncbi) publications (my bibliography) account settings log out access keys ncbi homepage myncbi homepage main content main navigation search: search advanced clipboard user guide save email send to clipboard my bibliography collections citation manager display options display options format abstract pubmed pmid save citation to file format: summary (text) pubmed pmid abstract (text) csv create file cancel email citation subject: to: format: summary summary (text) abstract abstract (text) mesh and other data send email cancel add to collections create a new collection add to an existing collection name your collection: name must be less than characters choose a collection: unable to load your collection due to an error please try again add cancel add to my bibliography my bibliography unable to load your delegates due to an error please try again add cancel your saved search name of saved search: search terms: test search terms would you like email updates of new search results? saved search alert radio buttons yes no email: (change) frequency: monthly weekly daily which day? the first sunday the first monday the first tuesday the first wednesday the first thursday the first friday the first saturday the first day the first weekday which day? sunday monday tuesday wednesday thursday friday saturday report format: summary summary (text) abstract abstract (text) pubmed send at most: item items items items items items items send even when there aren't any new results optional text in email: save cancel create a file for external citation management software create file cancel your rss feed name of rss feed: number of items displayed: create rss cancel rss link copy full text links cambridge university press core full text links actions cite favorites display options display options format abstract pubmed pmid share permalink copy page navigation title & authors abstract similar articles cited by mesh terms related information linkout - more resources title & authors abstract similar articles cited by mesh terms related information linkout - more resources behav brain sci actions search in pubmed search in nlm catalog add to search . jun; ( - ): - ; discussion - . doi: . /s x x. epub jun . the weirdest people in the world? joseph henrich , steven j heine, ara norenzayan affiliations expand affiliation department of psychology and department of economics, university of british columbia, vancouver v t z , canada. joseph.henrich@gmail.com pmid: doi: . /s x x free article item in clipboard the weirdest people in the world? joseph henrich et al. behav brain sci. jun. free article show details display options display options format abstract pubmed pmid behav brain sci actions search in pubmed search in nlm catalog add to search . jun; ( - ): - ; discussion - . doi: . /s x x. epub jun . authors joseph henrich , steven j heine, ara norenzayan affiliation department of psychology and department of economics, university of british columbia, vancouver v t z , canada. joseph.henrich@gmail.com pmid: doi: . /s x x item in clipboard full text links citedisplay options display options format abstract pubmed pmid abstract behavioral scientists routinely publish broad claims about human psychology and behavior in the world's top journals based on samples drawn entirely from western, educated, industrialized, rich, and democratic (weird) societies. researchers - often implicitly - assume that either there is little variation across human populations, or that these "standard subjects" are as representative of the species as any other population. are these assumptions justified? here, our review of the comparative database from across the behavioral sciences suggests both that there is substantial variability in experimental results across populations and that weird subjects are particularly unusual compared with the rest of the species - frequent outliers. the domains reviewed include visual perception, fairness, cooperation, spatial reasoning, categorization and inferential induction, moral reasoning, reasoning styles, self-concepts and related motivations, and the heritability of iq. the findings suggest that members of weird societies, including young children, are among the least representative populations one could find for generalizing about humans. many of these findings involve domains that are associated with fundamental aspects of psychology, motivation, and behavior - hence, there are no obvious a priori grounds for claiming that a particular behavioral phenomenon is universal based on sampling from a single subpopulation. overall, these empirical patterns suggests that we need to be less cavalier in addressing questions of human nature on the basis of data drawn from this particularly thin, and rather unusual, slice of humanity. we close by proposing ways to structurally re-organize the behavioral sciences to best tackle these challenges. similar articles weird people, yes, but also weird experiments. baumard n, sperber d. baumard n, et al. behav brain sci. jun; ( - ): - . doi: . /s x . epub jun . behav brain sci. . pmid: the weirdest brains in the world. chiao jy, cheon bk. chiao jy, et al. behav brain sci. jun; ( - ): - . doi: . /s x . epub jun . behav brain sci. . pmid: culture and the quest for universal principles in moral reasoning. sachdeva s, singh p, medin d. sachdeva s, et al. int j psychol. jun ; ( ): - . doi: . / . . . int j psychol. . pmid: review. responsible behavioral science generalizations and applications require much more than non-weird samples. konecni vj. konecni vj. behav brain sci. jun; ( - ): - . doi: . /s x . epub jun . behav brain sci. . pmid: a framework for the unification of the behavioral sciences. gintis h. gintis h. behav brain sci. feb; ( ): - ; discussion - . doi: . /s x . behav brain sci. . pmid: review. see all similar articles cited by articles neural signatures of syntactic variation in speech planning. sauppe s, choudhary kk, giroud n, blasi de, norcliffe e, bhattamishra s, gulati m, egurtzegi a, bornkessel-schlesewsky i, meyer m, bickel b. sauppe s, et al. plos biol. jan ; ( ):e . doi: . /journal.pbio. . ecollection jan. plos biol. . pmid: free pmc article. the social function of the feeling and expression of guilt. julle-danière e, whitehouse j, vrij a, gustafsson e, waller bm. julle-danière e, et al. r soc open sci. dec ; ( ): . doi: . /rsos. . ecollection dec. r soc open sci. . pmid: free pmc article. the perception of interpersonal distance is distorted by the müller-lyer illusion. bunce c, gray klh, cook r. bunce c, et al. sci rep. jan ; ( ): . doi: . /s - - -y. sci rep. . pmid: free pmc article. canadian undergraduate men's visual attention to cisgender women, cisgender men, and feminine trans individuals. petterson lj, vasey pl. petterson lj, et al. sci rep. jan ; ( ): . doi: . /s - - - . sci rep. . pmid: free pmc article. to which world regions does the valence-dominance model of social perception apply? jones bc, debruine lm, flake jk, liuzza mt, antfolk j, arinze nc, ndukaihe ilg, bloxsom ng, lewis sc, foroni f, willis ml, cubillas cp, vadillo ma, turiegano e, gilead m, simchon a, saribay sa, owsley nc, jang c, mburu g, calvillo dp, wlodarczyk a, qi y, ariyabuddhiphongs k, jarukasemthawee s, manley h, suavansri p, taephant n, stolier rm, evans tr, bonick j, lindemans jw, ashworth lf, hahn ac, chevallier c, kapucu a, karaaslan a, leongómez jd, sánchez or, valderrama e, vásquez-amézquita m, hajdu n, aczel b, szecsi p, andreychik m, musser ed, batres c, hu cp, liu ql, legate n, vaughn la, barzykowski k, golik k, schmid i, stieger s, artner r, mues c, vanpaemel w, jiang z, wu q, marcu gm, stephen id, lu jg, philipp mc, arnal jd, hehman e, xie sy, chopik wj, seehuus m, azouaghe s, belhaj a, elouafa j, wilson jp, kruse e, papadatou-pastou m, de la rosa-gómez a, barba-sánchez ae, gonzález-santoyo i, hsu t, kung cc, wang hh, freeman jb, oh dw, schei v, sverdrup te, levitan ca, cook cl, chandel p, kujur p, parganiha a, parveen n, pati ak, pradhan s, singh mm, pande b, bavolar j, kačmár p, zakharov i, Álvarez-solas s, baskin e, thirkettle m, schmidt k, christopherson cd, leonis t, suchow jw, olofsson jk, jernsäther t, lee as, beaudry jl, gogan td, oldmeadow ja, balas b, stevens lm, colloff mf, flowe hd, gülgöz s, brandt mj, hoyer k, jaeger b, ren d, sleegers wwa, wissink j, kaminski g, floerke va, urry hl, chen sc, pfuhl g, vally z, basnight-brown dm, jzerman hi, sarda e, neyroud l, badidi t, van der linden n, tan cby, kovic v, sampaio w, ferreira p, santos d, burin di, gardiner g, protzko j, schild c, Ścigała ka, zettler i, o'mara kunz em, storage d, wagemans fma, saunders b, sirota m, sloane gv, lima tjs, uittenhove k, vergauwe e, jaworska k, stern j, ask k, van zyl cjj, körner a, weissgerber sc, boudesseul j, ruiz-dodobara f, ritchie kl, michalak nm, blake kr, white d, gordon-finlayson ar, anne m, janssen smj, lee km, nielsen tk, tamnes ck, zickfeld jh, rosa ad, vianello m, kocsor f, kozma l, putz Á, tressoldi p, irrazabal n, chatard a, lins s, pinto ir, lutz j, adamkovic m, babincak p, baník g, ropovik i, coetzee v, dixson bjw, ribeiro g, peters k, steffens nk, tan kw, thorstenson ca, fernandez am, hsu rmcs, valentova jv, varella mac, corral-frías ns, frías-armenta m, hatami j, monajem a, sharifian m, frohlich b, lin h, inzlicht m, alaei r, rule no, lamm c, pronizius e, voracek m, olsen j, giolla em, akgoz a, Özdoğru aa, crawford mt, bennett-day b, koehn ma, okan c, gill t, miller jk, dunham y, yang x, alper s, borras-guevara ml, cai sj, tiantian d, danvers af, feinberg dr, armstrong mm, gilboa-schechtman e, mccarthy rj, muñoz-reyes ja, polo p, shiramazu vkm, yan wj, carvalho l, forscher ps, chartier cr, coles na. jones bc, et al. nat hum behav. jan; ( ): - . doi: . /s - - - . epub jan . nat hum behav. . pmid: see all "cited by" articles mesh terms behavioral sciences / methods* actions search in pubmed search in mesh add to search cognition actions search in pubmed search in mesh add to search cross-cultural comparison* actions search in pubmed search in mesh add to search decision making actions search in pubmed search in mesh add to search humans actions search in pubmed search in mesh add to search morals actions search in pubmed search in mesh add to search population groups* actions search in pubmed search in mesh add to search visual perception actions search in pubmed search in mesh add to search related information cited in books linkout - more resources full text sources core cambridge university press full text links [x] cambridge university press core [x] cite copy download .nbib format: ama apa mla nlm send to clipboard email save my bibliography collections citation manager [x] connect twitter facebook youtube linkedin github blog support center national center for biotechnology information rockville pike bethesda, md usa about us contact us policies foia popular pubmed pubmed central bookshelf pubchem gene blast nucleotide protein geo resources literature health genomes genes proteins chemicals actions submit download learn develop analyze research nlm | nih | hhs | usa.gov software carpentry software carpentry navigation contact search home about about us code of conduct software carpentry foundation our members become a member organization our history community service awards privacy policy assessment workshops our workshops code of conduct request a workshop pitching a workshop our audience testimonials workshop operations past workshops lessons our lessons license development dashboard previous material curriculum advisors get involved join us donate our instructors related projects help wanted reading material blog recent posts blog archive blog posts by category blog posts by date commenting policy teaching basic lab skills for research computing our workshops › find or host a workshop. our lessons › have a look at what we teach. get involved › help us help researchers. upcoming workshops click on an individual event to learn more about that event, including contact information and registration instructions. west virginia university instructors: asher hudson, vivian guetler feb - feb , carpentries @ mit: intro to unixshell/python/git instructors: ye li, ece turnator, madeline wrable, daniel sheehan, christine malinowski helpers: phoebe ayers, ye li, ece turnator, madeline wrable, daniel sheehan, christine malinowski feb - feb , george mason university (online) instructors: annajiat alim rasel feb - feb , newcastle university instructors: jannetta steyn feb - feb , kaust visualization core lab instructors: david pugh feb - feb , discord instructors: tim ribaric, daniel brett, thomas guignard helpers: john fink feb - feb , noaa/nos (online) instructors: mike trizna, ian carroll, camilla bressan feb - feb , united states department of agriculture (usda) (online) instructors: annajiat alim rasel, manisha munasinghe, sichong peng mar - mar , ** workshops marked with asterisks are based on curriculum from the carpentries lesson programs but may not follow our standard workshop format. workshops with a globe icon are being held online. the corresponding flag notes the country where the host organization is based. see our past workshops recent blog posts in addition of the posts below, find out what's happening in our community through the carpentries blog, a great resource that collates posts from data carpentry, library carpentry, and software carpentry, and publishes updates of general interest to the community. git lesson using worksheetspariksheet nanda / - - meet the members of the software carpentry cacerin becker, belinda weaver / - - launching the carpentries websitetracy teal, belinda weaver / - - running workshops on limited budgetsbelinda weaver, toby hodges, anelda van der walt / - - launching our new handbooktracy teal, maneesha sane, belinda weaver / - - building library carpentry community and developmenttracy teal, john chodacki, chris erdmann / - - developing github labels for the carpentries lessonsfrançois michonneau / - - mentoring groups open for multiple timezoneskari l. jordan / - - what can i do during the bug bbq? · how to be involved in the bug bbq?françois michonneau / - - who belongs at carpentrycon ? you do!belinda weaver / - - more › dialogue & discussion you can review our commenting policy here. please enable javascript to view the comments powered by disqus.  about this site software carpentry is a volunteer project dedicated to teaching basic computing skills to researchers. more › services contact rss sitemap.xml site design: feeling responsive. dear real donald trump a month of tweets at @realdonaldtrump may , may , may , may , may , may , may , may , may , may , may , may , may , may , may , june , june , june , june , june , june , june , june , june , june , june , june , june , june , june , june , june , june , june , this work is licensed under a creative commons attribution . international license. nick ruest, web archives for historical research api codex | dpla skip to main content digital public library of americashow menu home hubs ebooks projects about events dpla main site news home hubs ebooks projects about events dpla main site news developers api codex projects hub network prospective hubs developers api codex api basics requests responses object structure field reference glossary policies philosophy technologies troubleshooting & faq contact bulk download sample code & libraries technical documentation education ebooks community reps about dpla pro events contact us api codex welcome to the dpla api codex! this is where you can find authoritative documentation for the dpla api and resources you can use to make the most of it. we’re happy to have you. (wondering why we built an api in the first place? check out our philosophy.) what now? if you’ve never played (or worked) with an api before, head on over to api basics. we’ll get you started accessing some of our data. if you’re already familiar with apis in general and you just want to know how to get at our data, skip the basics and dive right into requests (make sure you get an api key first.) once you’ve got some shiny new json-ld, how can you tell what it all means? in responses, we explore the structure of objects and define the fields within those objects that the api will return. ready to start building something against our api, but want to know how we operate? we’ve got you. check out our policies. uh oh. you’re lost. we’re lost. everything’s broken. i’m confused. time to check out our troubleshooting and faq page! confused by the terminology we’re using? check out the glossary of terms. did we miss/break/overlook/forget something? tell us. the essentials need to start right now? here’s a cheat sheet. https://api.dp.la/v is the base url of the dpla api. items and collections are the two resource types you can request. in order to pass requests to the api, you’ll need to request an api key first. you also need to append &api_key=$your_key (where $your_key is your -character key) to any requests you make. all records are returned as structured, wonderful, json-ld objects. and we have reasons for that. we’ve got a boatload of fields you can query, and a whole bunch of ways you can query them. dive head first into a bucket full of kitten metadata: https://api.dp.la/v /items?q=kittens&api_key=. (copy and paste the url into your browser and add your api key on the end.) about us about dpla pro contact us feedback projects events communities hub network prospective hubs community reps developers education ebooks dpla dpla home primary source sets browse by partner browse by topic exhibitions search donate rails autoscale - rails autoscale pricingbloghelp & docsupdates homepricingbloghelp & docsupdates autoscale your web and worker dynos save money, handle uneven traffic, and spend less time on dev ops. built specifically for heroku and rails. try rails autoscale for free or watch a demo “ i needed to save on dyno hours at slow times of day/week, but handle the busier times without doing anything manually. rails autoscale has given me this peace of mind. ” – björn forsberg sleep well, without stress never worry about slow-downs or traffic spikes again. stop guessing about the "right" number of dynos for your app. let rails autoscale do the dirty work so you can focus on building, not managing. “ rails autoscale has allowed us to reduce our dyno usage for our primary application down to less than half of what we were using before, which is amazing (and allows us to not have to upgrade our heroku plan). thanks for making this, it really is a great little service. ” – mark urich cut your heroku bill in half most apps are overscaled so they can handle peak traffic. rails autoscale easily pays for itself by scaling down your app during quieter periods, some apps earn back half of what they were previously paying for heroku. try it and be amazed at how many dynos you don't need. “ if i was the king of the world, i would make it illegal to horizontally scale based on execution time. scale based on queue depths, people! ” – nate berkopec, author of the complete guide to rails performance a smarter way to autoscale rails autoscale is the only autoscaler based on request queue time, not total response time. some endpoints naturally take longer than others, and that shouldn't trigger scaling. rails autoscale watches for real capacity issues to scale your app reliably and consistently. “ we have a very uneven sidekiq workflow with a large number of jobs that get enqueued overnight. rails autoscale lets us get those jobs done quickly without paying for idle workers most of the day. ” – geoff harcourt worker dynos included if you use a background worker like sidekiq, you can autoscale those dynos, too! even if you split your worker queues across multiple sidekiq processes, it's no problem. configure and autoscale each of them independently. worker autoscaling is part of the package, no extra charge. “ rails autoscale saved me big time yesterday!! i'm new to this and i'm learning on the fly, but you made it very easy. ” – sam wood impossibly simple rails autoscale embraces the / rule and provides sensible defaults that work for % of apps. no sweat if you're in the remaining %. just a few tweaks is all it takes to get your app scaling smoothly. “i've used most of them. adam's is the best. i highly recommend it.” – andrew culver, creator of bullet train a partner to support you i'm adam, the founder of rails autoscale and a rails developer like you. i personally answer every support request, and it makes my day to help developers run their heroku apps with confidence. got a question? send me an email! “ you’ve built a fantastic product which my team at neos love, it’s really nice to find something that solves what appears to be a hard problem in a simple way. in the last month we’ve already made hundreds of dollars of savings from being able to cleanly scale down during quiet periods. ” – jon wood pricing that makes sense the more dynos you use, the more you save by autoscaling them. rails autoscale is priced accordingly. each plan below includes all rails autoscale features. choose your plan based on how how many web or workers dynos you use at peak traffic. for more info, check out the plans & pricing guide. standard dynos - dynos $ - dynos $ - dynos $ - dynos $ performance dynos - dynos $ - dynos $ - dynos $ - dynos $ dip your toes in with a free -day trial. start your trial now homebloghelp & docsupdatesherokutwitterprivacyterms twitter bots | nick ruest search nick ruest home c.v. posts presentations publications projects visualizations music contact twitter bots last updated on apr , min read introduction list of bots i run, divided up by type. anon @gccaedits ip address ranges periodic twitter archive requests diffengine @canadaland_diff account suspended @cbc_diff account suspended @cpc_diff account suspended @fairpressdiff account suspended @globemail_diff account suspended @greenparty_diff account suspended @lapress_diff account suspended @liberalca_diff account suspended @millennial_diff account suspended @natpost_diff account suspended @ndpca_diff account suspended @onn_diff account suspended @ontario_diff account suspended @pmgcca_diff account suspended @therebel_diff account suspended @torontosun_diff account suspended @torstar_diff account suspended @yyc_herald_diff account suspended yudlbots @yudlbot @yudldog @yudlcat dpla bots @dplafy account suspended @dpl_eh deactivated other @notflix_n_chill (code) twitter bots nick ruest associate librarian related twitter datasets and derivative data , , womensmarch tweets january - , , , elxn images a look at , , paris bataclan parisattacks porteouverte tweets an exploratory look at , , jesuischarlie, jesuisahmed, jesuisjuif, and charliehebdo tweets cc-by · powered by the academic theme for hugo. cite × copy download four fucking years of donald trump | nick ruest search nick ruest home c.v. posts presentations publications projects visualizations music contact four fucking years of donald trump jan , min read nearly four years ago i decided to start collecting tweets to donald trump out of morbid curiosity. if i was a real archivist, i would have planned this out a little bit better, and started collecting on election night in , or inaguration day . i didn’t. using twarc, i started collecting with the filter (streaming) api on may , . that process failed, and i pivoted to using the search api. i dropped that process into a simple bash script, and pointed cron at it to run every days. here’s what the bash script looked like: #!/bin/bash date=`date +"%y_%m_%d"` cd /mnt/vol /data_sets/to_trump/raw /usr/local/bin/twarc search 'to:realdonaldtrump' --log donald_search_$date.log > donald_search_$date.json it’s not beautiful. it’s not perfect. but, it did the job for the most part for almost four years save and except a couple twitter suspensions on accounts that i used for collection, and an absolutely embarassing situtation where i forgot to setup cron correctly on a machine i moved the collecting to for a couple weeks while i was on family leave this past summer. in the end, the collection ran from may , - january , , and collected , , unique tweets; . t of line-delminted json! the final created_at timestamp was wed jan : : + , and the text of that tweet very fittingly reads, “@realdonaldtrump you’re fired!“ the “dehydrated” tweets can be found here. in that dataset i decided to include in a number of derivatives created with twut which, i hope rounds out the dataset. this update is the final update on the dataset. i also started working on some notebooks here where i’ve been trying to explore the dataset a bit more in my limited spare time. i’m hoping to have the time and energy to really dig into this dataset sometime in the future. i’m especially curious at what the leadup to the storming of the united states capitol looks like in the dataset, as well as the sockpuppet frequency. i’m also hopeful that others will explore the dataset and that it’ll be useful in their research. i have a suspicion folks can do a lot smarter, innovative, and creative things with the dataset than i did here, here, here, here, or here. for those who are curious what the tweet volume for the last few months looked like (please note that the dates are utc), check out these bar charts. january is especially fun. - - twitter apache spark twut twarc donald trump nick ruest associate librarian related , , images tweeted at donald trump twut. wait, wut? twut? twitter wordcloud pipeline twitter datasets and derivative data , , womensmarch tweets january - , cc-by · powered by the academic theme for hugo. cite × copy download totally clears the president. thank you! totally clears the president. thank you! , images tweeted at @realdonaldtrump | full dataset is available here. | created with juxta. generated - - : this work is licensed under a creative commons attribution . international license. web archives for historical research weeknote ( ) – librarian of things skip to content librarian of things weeknote ( ) hey. i missed last week’s weeknote. but we are here now. § this week i gave a class on searching scientific literature to a group of biology masters students. while i was making my slides comparing the advanced search capabilities of web of science and scopus, i discovered this weird behaviour of google scholar: a phrase search generated more hits than not. i understand that google scholar performs ‘stemming’ instead of truncation in generating search results but this still makes no sense to me. § new to me: if you belong to an organization that is already a member of crossref, you are eligible to use a similarity check of documents for an additional fee. perhaps this is a service we could provide to our ojs editors. § i’m still working through the canadian journal of academic librarianship special issue on academic libraries and the irrational. long time readers know that i have a fondness for the study of organizational culture and so it should not be too surprising that the first piece i wanted to read was the digital disease in academic libraries. it begins…. though several recent books and articles have been written about change and adaptation in contemporary academic libraries (mossop ; eden ; lewis ), there are few critical examinations of change practices at the organizational level. one example, from which this paper draws its title, is braden cannon’s ( ) the canadian disease, where the term disease is used to explore the trend of amalgamating libraries, archives, and museums into monolithic organizations. though it is centered on the impact of institutional convergence, cannon’s analysis uses an ethical lens to critique the bureaucratic absurdity of combined library-archive-museum structures. this article follows in cannon’s steps, using observations from organizational de-sign and management literature to critique a current trend in the strategic planning processes and structures of contemporary academic libraries. my target is our field’s ongoing obsession with digital transformation beyond the shift from paper-based to electronic resources, examined in a north american context and framed here as the digital disease. i don’t want to spoil the article but i do want to include this zinger of a symptom which is the first of several: if your library’s organizational chart highlights digital forms of existing functions, you might have the digital disease. kris joseph, the digital disease in academic libraries, canadian journal of academic librarianship, vol ( ) ouch. that truth hurts almost as much as this tweet did: share this: twitter facebook author mita williamsposted on january , january , categories weeknotes leave a reply cancel reply your email address will not be published. required fields are marked * comment name * email * website post navigation previous previous post: weeknote ( ) next next post: weeknote ( ) about me librarian of things is a blog by me, mita williams, who used to blog at new jack librarian until blogger.com finally gave up the ghost. if you don’t have an rss reader, you can subscribe for email delivery through mailchimp. you can learn more about my work at aedileworks.com as well as my other blogs and my weekly newsletter. if you are an editor of a scholarly journal and think that a post could be expanded into a more academic form, please let me know. search for: search recent posts weeknote ( ) weeknote ( ) weeknote ( ) weeknote ( ) weeknote ( ) archives february january december november october september june may april october june may april march january july june may april december june may april november august july meta log in entries feed comments feed wordpress.org librarian of things proudly powered by wordpress planet code lib http://planet.code lib.org planet code lib - http://planet.code lib.org david rosenthal: talk at berkeley's information access seminar https://blog.dshr.org/ / /talk-at-berkeleys-information-access.html once again cliff lynch invited me to give a talk to the information access seminar at uc berkeley's ischool. preparation time was limited because these days i'm a full-time grandparent so the talk, entitled securing the digital supply chain summarizes and updates two long posts from two years ago:

the abstract was:

the internet is suffering an epidemic of supply chain attacks, in which a trusted supplier of content is compromised and delivers malware to some or all of their clients. the recent solarwinds compromise is just one glaring example. this talk reviews efforts to defend digital supply chains.

below the fold, the text of the talk with links to the sources.

solarwinds, and many other recent system and network compromises have been supply chain attacks. these are extremely efficient, because unlike one-at-a-time attacks such as phishing, they provide a built-in mass deployment mechanism. a single compromise of solarwinds infected at least , networks. clearly, the vendors' security practices, and their vendors' security practices, and so on ad infinitum are important, but the sad truth is that current digital supply chain technologies are incapable of mitigating the inevitable security lapses along the chain.

this talk reviews the efforts to defend supply chains that deliver digital content, such as software. but lets start with a simpler case, web pages.
web page supply chainhow do i know that i'm talking to the right web site? because there's a closed padlock icon in the url bar, right?

[slide ]

mozilla says:

a green padlock (with or without an organization name) indicates that:
you are definitely connected to the website whose address is shown in the address bar; the connection has not been intercepted.
the connection between firefox and the website is encrypted to prevent eavesdropping.

nb - this is misleading!

the padlock icon appears when the browser has validated that the connection to the url in the url bar supplied a certificate for the site in question carrying a signature chain ending in one of the root certificates the browser trusts. browsers come with a default list of root certificates from certificate authorities (cas). my firefox trusts certificates from different organizations including, for example, amazon and google, but also chunghwa telecom co., ltd. and the dutch government. why is this list a problem?

the browser trusts all of them equally.
the browser trusts cas that the cas on the list delegate trust to. back in , the eff found more than organizations that internet explorer and firefox trusted.
commercial cas on the list, and cas they delegate to, have regularly been found to be issuing false or insecure certificates.

[slide ]

one of these trusted organizations is the internet security research group, a not-for-profit organization hosted by the linux foundation and sponsored by many organizations including mozilla and the eff, which has greatly improved the information hygiene of the web through a program called let's encrypt. this has provided over million web sites with free certificates carrying a signature chain rooted in a certificate that almost all browsers trust. my blog's certificate is one of them, as you can see by clicking on the padlock icon.

[slide ]

barysevich identified four such sellers of counterfeit certificates since . two of them remain in business today. the sellers offered a variety of options. in , one provider calling himself c@t advertised certificates that used a microsoft technology known as authenticode for signing executable files and programming scripts that can install software. c@t offered code-signing certificates for macos apps as well. ... "in his advertisement, c@t explained that the certificates are registered under legitimate corporations and issued by comodo, thawte, and symantec—the largest and most respected issuers,"

dan goodin one-stop counterfeit certificate shops for all your malware-signing needs

abuse of the trust users place in cas is routine:

in one case, a prominent dutch ca (diginotar) was compromised and the hackers were able to use the ca’s system to issue fake ssl certificates. the certificates were used to impersonate numerous sites in iran, such as gmail and facebook, which enabled the operators of the fake sites to spy on unsuspecting site users. ... more recently, a large u.s.-based ca (trustwave) admitted that it issued subordinate root certificates to one of its customers so the customer could monitor traffic on their internal network. subordinate root certificates can be used to create ssl certificates for nearly any domain on the internet. although trustwave has revoked the certificate and stated that it will no longer issue subordinate root certificates to customers, it illustrates just how easy it is for cas to make missteps and just how severe the consequences of those missteps might be.

in sennheiser provided another example:

the issue with the two headsetup apps came to light earlier this year when german cyber-security firm secorvo found that versions . , . , and . installed two root certification authority (ca) certificates into the windows trusted root certificate store of users' computers but also included the private keys for all in the senncomcckey.pem file.

certificates depend on public-key cryptography, which splits keys into public/private key pairs. private keys can decrypt text encrypted by the public key, and vice versa. the security of the system depends upon private keys being kept secret. this poses two problems:

as the sennheiser example shows, it is easy for the private keys to leak. another common way for them to leak is for a server to be compromised. for the server to be able to verify its identity, and thus unlock the padlock, the private key needs to be stored on the server in cleartext. so an intruder can steal it to impersonate the server.
there is no alarm bell or notification to the owner or affected users when a private key leaks. so, as in the sennheiser case, the attacker may be able to use it unimpeded for a long time, until some security researcher notices some anomaly.

catalin cimpanu continues:

in a report published today, secorvo researchers published proof-of-concept code showing how trivial would be for an attacker to analyze the installers for both apps and extract the private keys.

making matters worse, the certificates are also installed for mac users, via headsetup macos app versions, and they aren't removed from the operating system's trusted root certificate store during current headsetup updates or uninstall operations.
...
sennheiser's snafu ... is not the first of its kind. in , lenovo shipped laptops with a certificate that exposed its private key in a scandal that became known as superfish. dell did the exact same thing in in a similarly bad security incident that became known as edellroot.

cimpanu also reports on a more recent case:

under the guise of a "cybersecurity exercise," the kazakhstan government is forcing citizens in its capital of nur-sultan (formerly astana) to install a digital certificate on their devices if they want to access foreign internet services.

once installed, the certificate would allow the government to intercept all https traffic made from users' devices via a technique called mitm (man-in-the-middle).

this type of “mistake” allows attackers to impersonate any web site to affected devices.

cas are supposed to issue three grades of certificate based on increasingly rigorous validation:

domain validated (dv) certificates verify control over the dns entries, email and web content of the specified domain. they can be issued via automated processes, as with let's encrypt.
organization validated (ov) certificates are supposed to verify the legal entity behind the dv-level control of the domain, but in practice are treated the same as dv certificates.
extended validation (ev) certificates require "verification of the requesting entity's identity by a certificate authority (ca)". verification is supposed to be an intrusive, human process.

[slide ]

source

but, as can be seen from the advert, the extended verification process is far from fool-proof. this lack of trustworthiness of cas should not be a surprise. six years ago security collapse in the https market, a fascinating analysis of the (lack of) security on the web from an economic rather than a technical perspective by axel arnbak et al from amsterdam and delft universities showed that cas lack incentives to be trustworthy. they write:

information asymmetry prevents buyers from knowing what cas are really doing. buyers are paying for the perception of security, a liability shield, and trust signals to third parties. none of these correlates verifiably with actual security. given that ca security is largely unobservable, buyers’ demands for security do not necessarily translate into strong security incentives for cas.
negative externalities of the weakest-link security of the system exacerbate these incentive problems. the failure of a single ca impacts the whole ecosystem, not just that ca’s customers. all other things being equal, these interdependencies undermine the incentives of cas to invest, as the security of their customers depends on the efforts of all other cas.

the reason for the weakest-link is:

a crucial technical property of the https authentication model is that any ca can sign certificates for any domain name. in other words, literally anyone can request a certificate for a google domain at any ca anywhere in the world, even when google itself has contracted one particular ca to sign its certificate.

this "technical property" is actually important, it is what enables a competitive market of cas. symantec in particular has exploited it wholesale:

google's investigation revealed that over a span of years, symantec cas have improperly issued more than , certificates. ... they are a major violation of the so-called baseline requirements that major browser makers impose of cas as a condition of being trusted by major browsers.

but symantec has suffered no effective sanctions because they are too big to fail:

symantec's repeated violations underscore one of the problems google and others have in enforcing terms of the baseline requirements. when violations are carried out by issuers with a big enough market share they're considered too big to fail. if google were to nullify all of the symantec-issued certificates overnight, it might cause widespread outages.

my firefox still trusts symantec root certificates. because google, mozilla and others prioritize keeping the web working over keeping it secure, deleting misbehaving big cas from trust lists won't happen. when mozilla writes:

you are definitely connected to the website whose address is shown in the address bar; the connection has not been intercepted.

they are assuming a world of honest cas that isn't this world. if you have the locked padlock icon in your url bar, you are probably talking to the right web site, but there is a chance you aren't.

[slide ]

recent data from anti-phishing company phishlabs shows that percent of all phishing sites in the third quarter of bore the padlock security icon next to the phishing site domain name as displayed in a browser address bar. that’s up from percent just one year ago, and from percent in the second quarter of .

brian krebs half of all phishing sites now have the padlock

building on earlier work by wendlandt et al, moxie marlinspike, the eff and others, in google started work on an approach specified in rfc , and called certificate transparency (ct). the big difference from earlier efforts, which didn't require cooperation from website owners and cas, was that google's did require cooperation and they had enough leverage to obtain it:

[slide ]

google's certificate transparency project fixes several structural flaws in the ssl certificate system, which is the main cryptographic system that underlies all https connections. these flaws weaken the reliability and effectiveness of encrypted internet connections and can compromise critical tls/ssl mechanisms, including domain validation, end-to-end encryption, and the chains of trust set up by certificate authorities. if left unchecked, these flaws can facilitate a wide range of security attacks, such as website spoofing, server impersonation, and man-in-the-middle attacks.

certificate transparency helps eliminate these flaws by providing an open framework for monitoring and auditing ssl certificates in nearly real time. specifically, certificate transparency makes it possible to detect ssl certificates that have been mistakenly issued by a certificate authority or maliciously acquired from an otherwise unimpeachable certificate authority. it also makes it possible to identify certificate authorities that have gone rogue and are maliciously issuing certificates.

certificate transparency

the basic idea is to accompany the certificate with a hash of the certificate signed by a trusted third party, attesting that the certificate holder told the third party that the certificate with that hash was current. thus in order to spoof a service, an attacker would have to both obtain a fraudulent certificate from a ca, and somehow persuade the third party to sign a statement that the service had told them the fraudulent certificate was current. clearly this is:

more secure than the current situation, which requires only compromising a ca, and:
more effective than client-only approaches, which can detect that a certificate has changed but not whether the change was authorized.

ct also requires participation from browser manufacturers:

in order to improve the security of extended validation (ev) certificates, google chrome requires certificate transparency (ct) compliance for all ev certificates issued after jan .

clients now need two lists of trusted third parties, the cas and the sources of ct attestations. the need for these trusted third parties is where the blockchain enthusiasts would jump in and claim (falsely) that using a blockchain would eliminate the need for trust. but ct has a much more sophisticated approach, ronald reagan's "trust, but verify". in the real world it isn't feasible to solve the problem of untrustworthy cas by eliminating the need for trust. ct's approach instead is to provide a mechanism by which breaches of trust, both by the cas and by the attestors, can be rapidly and unambiguously detected.

[slide ]

source

here is a brief overview of how ct works to detect breaches of trust. the system has the following components:

logs, to which cas report their current certificates, and from which they obtain attestations, called signed certificate timestamps (scts), that owners can attach to their certificates. clients can verify the signature on the sct, then verify that the hash it contains matches the certificate. if it does, the certificate was the one that the ca reported to the log, and the owner validated. it is envisaged that there will be tens but not thousands of logs; chrome currently trusts logs operated by organizations. each log maintains a merkle tree data structure of the certificates for which it has issued scts.
monitors, which periodically download all newly added entries from the logs that they monitor, verify that they have in fact been added to the log, and perform a series of validity checks on them. they also thus act as backups for the logs they monitor.
auditors, which use the merkle tree of the logs they audit to verify that certificates have been correctly appended to the log, and that no retroactive insertions, deletions or modifications of the certificates in the log have taken place. clients can use auditors to determine whether a certificate appears in a log. if it doesn't, they can use the sct to prove that the log misbehaved.

in this way, auditors, monitors and clients cooperate to verify the correct operation of logs, which in turn provides clients with confidence in the [certificate,attestation] pairs they use to secure their communications. although the process works if certificate owners each obtain their scts from only one log, they should get them from multiple logs and send a random selection of their scts to each client to improve robustness. note the key architectural features of ct:

[slide ]

certificate transparency architecture:

the certificate data is held by multiple independent services.
they get the data directly from the source, not via replication from other services.
clients access the data from a random selection of the services.
there is an audit process continually monitoring the services looking for inconsistencies.

these are all also features of the protocol underlying the lockss digital preservation system, published in . in both cases, the random choice among a population of independent services makes life hard for attackers. if they are to avoid detection, they must compromise the majority of the services, and provide correct information to auditors while providing false information to victims.

looking at the list of logs chrome currently trusts, it is clear that almost all are operated by cas themselves. assuming that each monitor at each ca is monitoring some of the other logs as well as the one it operates, this does not represent a threat, because misbehavior by that ca would be detected by other cas. a ca's monitor that was tempted to cover up misbehavior by a different ca's log it was monitoring would risk being "named and shamed" by some other ca monitoring the same log, just as the misbehaving ca would be "named and shamed".

it is important to observe that, despite the fact that cas operate the majority of the ct infrastructure, its effectiveness in disciplining cas is not impaired. all three major cas have suffered reputational damage from recent security failures, although because they are "too big to fail" this hasn't impacted their business much. however, as whales in a large school of minnows it is in their interest to impose costs (for implementing ct) and penalties (for security lapses) on the minnows. note that google was sufficiently annoyed with symantec's persistent lack of security that it set up its own ca. the threat that their business could be taken away by the tech oligopoly is real, and cooperating with google may have been the least bad choice. because these major corporations have an incentive to pay for the ct infrastructure, it is sustainable in a way that a market of separate businesses, or a permissionless blockchain supported by speculation in a cryptocurrency would not be.

fundamentally, if applications such as ct attempt to provide absolute security they are doomed to fail, and their failures will be abrupt and complete. it is more important to provide the highest level of security compatible with resilience, so that the inevitable failures are contained and manageable. this is one of the reasons why permissionless blockchains, subject to % attacks, and permissioned blockchains, with a single, central point of failure, are not suitable.
software supply chain[slide ]

when the mass compromise came to light last month, microsoft said the hackers also stole signing certificates that allowed them to impersonate any of a target’s existing users and accounts through the security assertion markup language. typically abbreviated as saml, the xml-based language provides a way for identity providers to exchange authentication and authorization data with service providers.

the full impact of the recent compromise of solarwind's orion network management software will likely never be known, it affected at least , networks, including microsoft's and:

the treasury department, the state department, the commerce department, the energy department and parts of the pentagon

it was not detected by any of the us government's network monitoring systems, but by fireeye, a computer security company that was also a victim. but for a mistake by the attackers at fireeye it would still be undetected. it was an extremely sophisticated attack, which has rightfully gained a lot of attention.

to understand how defenses against attacks like this might work, it is first necessary to understand how the supply chain that installs and updates the software on your computer works. i'll use apt, the system used by debian linux and its derivatives, as the example.

a system running debian or another apt-based linux distribution runs software it received in "packages" that contain the software files, and metadata that includes dependencies. their hashes can be verified against those in a release file, signed by the distribution publisher. packages come in two forms, source and compiled. the source of a package is signed by the official package maintainer and submitted to the distribution publisher. the publisher verifies the signature and builds the source to form the compiled package, whose hashes are then included in the release file.

the signature on the source package verifies that the package maintainer approves this combination of files for the distributor to build. the signature on the release file verifies that the distributor built the corresponding set of packages from approved sources and that the combination is approved for users to install.

[slide ]

there are thus two possible points of entry for an attacker:

they could compromise the developer, so that the signed source code files received by the distributor contained malware (type a),
or they could compromise the distributor, so that the package whose hash was in the signed release file did not reflect the signed source code, but contained malware (type b).

an example of a type a attack occurred in november . dan goodin reported that:

the malicious code was inserted in two stages into event-stream, a code library with million downloads that's used by fortune companies and small startups alike. in stage one, version . . , published on september , included a benign module known as flatmap-stream. stage two was implemented on october when flatmap-stream was updated to include malicious code that attempted to steal bitcoin wallets and transfer their balances to a server located in kuala lumpur.

how were the attackers able to do this? goodin explains:

according to the github discussion that exposed the backdoor, the longtime event-stream developer no longer had time to provide updates. so several months ago, he accepted the help of an unknown developer. the new developer took care to keep the backdoor from being discovered. besides being gradually implemented in stages, it also narrowly targeted only the copay wallet app. the malicious code was also hard to spot because the flatmap-stream module was encrypted.

all that was needed to implement this type a attack was e-mail and github accounts, and some social engineering.

dan goodin describes a simple type b attack in new supply chain attack uses poisoned updates to infect gamers’ computers:

in a nutshell, the attack works this way: on launch, nox.exe sends a request to a programming interface to query update information. the bignox api server responds with update information that includes a url where the legitimate update is supposed to be available. eset speculates that the legitimate update may have been replaced with malware or, alternatively, a new filename or url was introduced.

malware is then installed on the target’s machine. the malicious files aren’t digitally signed the way legitimate updates are. that suggests the bignox software build system isn’t compromised; only the systems for delivering updates are. the malware performs limited reconnaissance on the targeted computer. the attackers further tailor the malicious updates to specific targets of interest.

[slide ]

source

the solarwinds attackers tried but failed to penetrate the network of crowdstrike, another computer security company. sunspot: an implant in the build process, crowdstrike's analysis of the attack, reveals the much greater sophistication of this type b attack. once implanted in solarwinds' build system:

sunspot runs once a second scanning for instances of msbuild.exe, the tool used to build the target software.
if sunspot finds an msbuild.exe, it next locates the directory in which the build is running.
then sunspot checks whether what is being built is the target software.
if it is, sunspot checks whether the target source file has changed.
if it hasn't, sunspot carefully substitutes the modified source file for the target source file.
sunspot waits until the build completes, then carefully restores the target source file and erases the traces of its work.

solarwinds forensic timeline shows that the attackers penetrated their network in september , and a month later tested sunspot by injecting test code into the next release of orion. an improved sunspot was deployed from february to june, when it was removed having successfully compromised the orion release with the production malware. no-one noticed until december, when fireeye spotted suspicious activity on their internal network and traced it to orion.

microsoft's analysis reveals a lot more sophistication of the attacker's operations once they had penetrated the network:

[slide ]

each cobalt strike dll implant was prepared to be unique per machine and avoided at any cost overlap and reuse of folder name, file name, export function names, c domain/ip, http requests, timestamp, file metadata, config, and child process launched. this extreme level of variance was also applied to non-executable entities, such as wmi persistence filter name, wmi filter query, passwords used for -zip archives, and names of output log files.

how could software supply chains be enhanced to resist these attacks? in an important paper entitled software distribution transparency and auditability, benjamin hof and georg carle from tu munich:

describe how apt works to maintain up-to-date software on clients by distributing signed packages.
review previous efforts to improve the security of this process.
propose to enhance apt's security by layering a system similar to certificate transparency (ct) on top.
detail the operation of their systems' logs, auditors and monitors, which are similar to ct's in principle but different in detail.
describe and measure the performance of an implementation of their layer on top of apt using the trillian software underlying some ct implementations.

their system's ct-like logs contain the hashes of both the source and the binaries of each version of each package, and ensure that attackers would be detected if they, for example, create a short-lived version containing malware for a specific victim. it certainly defeats a significant class of attacks but, alas, does not address either the solarwinds or the event-stream attacks.

as regards the solarwinds attack, there are two important "missing pieces" in their system, and all the predecessors. each is the subject of a separate effort:
[slide ]

reproducible builds.
bootstrappable compilers.

suppose solarwinds had been working in hof and carle's system. they would have signed their source code, built it, and signed the resulting binaries. the attackers would have arranged that the source that was built was not the source that solarwinds signed, but solarwinds would not have known that. so the signatures on both the unmodified source and the modified binaries would appear valid in the logs, but the binaries would be malign.

the problem is that the connection between the source and the binaries rests on an assumption that the distributor's build environment has not been compromised - i.e. no type b attack. as with the multiple logs of ct, what is needed is multiple independent builds of the signed source. unless all of the independent build environments are compromised, a compromised build will differ from the others because it contains malware.

this is a great idea, but in practice it is very hard to achieve for both technical and organizational reasons:

the first technical reason is that in general, building the same source twice results in different binaries. compiler and linker output typically contains timestamps, temporary file names, and other sources of randomness. the build system needs to be reproducible.
the second technical reason is that, in order to be reproducible, the multiple independent builds have to use the same build environment. so each of the independent build environments will have the same vulnerabilities, allowing for the possibility that the attacker could compromise them all.
the organizational reason is that truly independent builds can only be done in an open source environment in which anyone, and in particular each of the independent builders, can access the source code.

to enable binaries to be securely connected to their source, a reproducible builds effort has been under way for more than years. debian project lead chris lamb's -minute talk think you're not a target? a tale of developers ... provides an overview of the problem and the work to solve it using three example compromises:

alice, a package developer who is blackmailed to distribute binaries that don't match the public source (a type a attack).
bob, a build farm sysadmin whose personal computer has been compromised, leading to a compromised build toolchain in the build farm that inserts backdoors into the binaries (a type b attack).
carol, a free software enthusiast who distributes binaries to friends. an evil maid attack has compromised her laptop.

as lamb describes, eliminating all sources of irreproducibility from a package is a painstaking process because there are so many possibilities. they include non-deterministic behaviors such as iterating over hashmaps, parallel builds, timestamps, build paths, file system directory name order, and so on. the work started in with % of debian packages building reproducibly. currently, around % of the debian packages for the amd and arm architectures are now reproducible. that is good, but % coverage is really necessary to provide security.

[slide ]

way back in , paul karger and roger schell discovered a devastating attack against computer systems. ken thompson described it in his classic speech, "reflections on trusting trust." basically, an attacker changes a compiler binary to produce malicious versions of some programs, including itself. once this is done, the attack perpetuates, essentially undetectably. thompson demonstrated the attack in a devastating way: he subverted a compiler of an experimental victim, allowing thompson to log in as root without using a password. the victim never noticed the attack, even when they disassembled the binaries -- the compiler rigged the disassembler, too.

in , bruce schneier summarized the message of perhaps the most famous of acm's annual turing award lectures. in this attack, the compromised build environment inserts malware even though it is building the unmodified source code. unlike the solarwinds attack, the signatures testifying that the binaries are the output of building the signed source code are correct.

[slide ]

source

this is the motivation for the bootstrappable builds project, whose goal is to create a process for building a complete toolchain starting from a "seed" binary that is simple enough to be certified "by inspection". recently, they achieved a major milestone. starting from a tiny "seed" binary, they were able to create a working tinycc compiler for the arm architecture. starting from tinycc, it is possible to build the entire gnucc toolchain and thus, in principle, a working linux. there is clearly a long way still to go to a bootstrapped full toolchain proof against type b attacks.

the event-stream attack can be thought of as the organization-level analog of a sybil attack on a peer-to-peer system. creating an e-mail identity is almost free. the defense against sybil attacks is to make maintaining and using an identity in the system expensive. as with proof-of-work in bitcoin, the idea is that the white hats will spend more (compute more useless hashes) than the black hats. even this has limits. eric budish's analysis shows that, if the potential gain from an attack on a blockchain is to be outweighed by its cost, the value of transactions in a block must be less than the block reward.

would a similar defense against "sybil" type a attacks on the software supply chain be possible? there are a number of issues:

the potential gains from such attacks are large, both because they can compromise very large numbers of systems quickly (event-stream had m downloads), and because the banking credentials, cryptocurrency wallets, and other data these systems contain can quickly be converted into large amounts of cash.
thus the penalty for mounting an attack would have to be an even larger amount of cash. package maintainers would need to be bonded or insured for large sums, which implies that distributions and package libraries would need organizational structures capable of enforcing these requirements.
bonding and insurance would be expensive for package maintainers, who are mostly unpaid volunteers. there would have to be a way of paying them for their efforts, at least enough to cover the costs of bonding and insurance.
thus users of the packages would need to pay for their use, which means the packages could neither be free, nor open source.
which would make implementing the reproducible builds and bootstrapped compilers needed to defend against type b attacks extremely difficult.

the foss (free open source software) movement will need to find other ways to combat sybil attacks, which will be hard if the reward for a successful attack greatly exceeds the cost of mounting it. adequately rewarding maintainers for their essential but under-appreciated efforts is a fundamental problem for foss.

it turns out that this talk is timely. two days ago, eric brewer, rob pike et al from google posted know, prevent, fix: a framework for shifting the discussion around vulnerabilities in open source, an important and detailed look at the problem of vulnerabilities in open source and what can be done to reduce them. their summary is:

it is common for a program to depend, directly or indirectly, on thousands of packages and libraries. for example, kubernetes now depends on about , packages. open source likely makes more use of dependencies than closed source, and from a wider range of suppliers; the number of distinct entities that need to be trusted can be very high. this makes it extremely difficult to understand how open source is used in products and what vulnerabilities might be relevant. there is also no assurance that what is built matches the source code.

taking a step back, although supply-chain attacks are a risk, the vast majority of vulnerabilities are mundane and unintentional—honest errors made by well-intentioned developers. furthermore, bad actors are more likely to exploit known vulnerabilities than to find their own: it’s just easier. as such, we must focus on making fundamental changes to address the majority of vulnerabilities, as doing so will move the entire industry far along in addressing the complex cases as well, including supply-chain attacks.

the bulk of their post addresses improvements to the quality of the development process, with three goals:

know about the vulnerabilities in your software
prevent the addition of new vulnerabilities, and
fix or remove vulnerabilities.

then, in a section entitled prevention for critical software they specifially address the security of the development process and thus the two types of supply chain attacks we have been discussing. they write:

this is a big task, and currently unrealistic for the majority of open source. part of the beauty of open source is its lack of constraints on the process, which encourages a wide range of contributors. however, that flexibility can hinder security considerations. we want contributors, but we cannot expect everyone to be equally focused on security. instead, we must identify critical packages and protect them. such critical packages must be held to a range of higher development standards, even though that might add developer friction.

[slide ]

define criteria for “critical” open source projects that merit higher standards
no unilateral changes to critical software
- require code review for critical software
- changes to critical software require approval by two independent parties
authentication for participants in critical software
- for critical software, owners and maintainers cannot be anonymous
- strong authentication for contributors of critical software
- a federated model for identities
notification for changes in risk
transparency for artifacts
trust the build process

their goals for the "higher development standards" include identifying the important packages that require higher standards, implementing review and signoff of changes by at least two independent developers, "transparency for artifacts", by which they mean reproducible builds, and "trust the build process" which implies a bootstrappable toolchain.

they acknowledge that these are very aggressive goals, because in many ways they cut against the free-wheeling development culture of open source that has sparked its remarkable productivity. if google were to persuade other major corporations to put significant additional resources of money and manpower into implementing them they would likely succeed. absent this, the additional load on developers will likely cause resistance.

- - t : : + : david. (noreply@blogger.com) andromeda yelton: archival face recognition for fun and nonprofit https://andromedayelton.com/ / / /archival-face-recognition-for-fun-and-nonprofit/

in , dominique luster gave a super good code lib talk about applying ai to metadata for the charles “teenie” harris collection at the carnegie museum of art — more than , photographs of black life in pittsburgh. they experimented with solutions to various metadata problems, but the one that’s stuck in my head since is the face recognition one. it sure would be cool if you could throw ai at your digitized archival photos to find all the instances of the same person, right? or automatically label them, given that any of them are labeled correctly?

sadly, because we cannot have nice things, the data sets used for pretrained face recognition embeddings are things like lots of modern photos of celebrities, a corpus which wildly underrepresents ) archival photos and ) black people. so the results of the face recognition process are not all that great.

i have some extremely technical ideas for how to improve this — ideas which, weirdly, some computer science phds i’ve spoken with haven’t seen in the field. so i would like to experiment with them. but i must first ~~invent the universe~~ set up a data processing pipeline.

three steps here:

fetch archival photographs;
do face detection (draw bounding boxes around faces and crop them out for use in the next step);
do face recognition.

for step , i’m using dpla, which has a super straightforward and well-documented api and an easy-to-use python wrapper (which, despite not having been updated in a while, works just fine with python . , the latest version compatible with some of my dependencies).

for step , i’m using mtcnn, because i’ve been following this tutorial.

for step , face recognition, i’m using the steps in the same tutorial, but purely for proof-of-concept — the results are garbage because archival photos from mid-century don’t actually look anything like modern-day celebrities. (neural net: “i have % confidence this is stevie wonder!” how nice for you.) clearly i’m going to need to build my own corpus of people, which i have a plan for (i.e. i spent some quality time thinking about numpy) but haven’t yet implemented.

so far the gotchas have been:

gotcha : if you fetch a page from the api and assume you can treat its contents as an image, you will be sad. you have to treat them as a raw data stream and interpret that as an image, thusly:

from pil import image import requests response = requests.get(url, stream=true) response.raw.decode_content = true data = requests.get(url).content image.open(io.bytesio(data))

this code is, of course, hilariously lacking in error handling, despite fetching content from a cesspool of untrustworthiness, aka the internet. it’s a first draft.

gotcha : you see code snippets to convert images to pixel arrays (suitable for ai ingestion) that look kinda like this: np.array(image).astype('uint '). except they say astype('float ') instead of astype('uint '). i got a creepy photonegative effect when i used floats.

gotcha : although pil was happy to manipulate the .pngs fetched from the api, it was not happy to write them to disk; i needed to convert formats first (image.convert('rgb')).

gotcha : the suggested keras_vggface library doesn’t have a pipfile or requirements.txt, so i had to manually install keras and tensorflow. luckily the setup.py documented the correct versions. sadly the tensorflow version is only compatible with python up to . (hence the comment about dpyla compatibility above). i don’t love this, but it got me up and running, and it seems like an easy enough part of the pipeline to rip out and replace if it’s bugging me too much.

the plan from here, not entirely in order, subject to change as i don’t entirely know what i’m doing until after i’ve done it:

build my own corpus of identified people
- this means the numpy thoughts, above
- it also means spending more quality time with the api to see if i can automatically apply names from photo metadata rather than having to spend too much of my own time manually labeling the corpus
decide how much metadata i need to pull down in my data pipeline and how to store it
figure out some kind of benchmark and measure it
try out my idea for improving recognition accuracy
benchmark again
hopefully celebrate awesomeness

- - t : : + : andromeda mita williams: weeknote ( ) https://librarian.aedileworks.com/ / / /weeknote- - / §

last friday i was interviewed for the podcast the grasscast — a game-themed podcast named after the book, the grasshopper: games, life, and utopia. i ramble a little bit in the episode as i tried to be more open and conversational than concise and correct. but i also spoke that way because for some of the questions, no pat answer came immediately to mind.

there was one question that stumped me but in my trying to answer, i think i found something i had not considered before. the question was, what is one bad thing about games? and i tried to convey that, unlike video games where you can play with strangers, most tabletop games are generally constrained by the preferences of your social circles. in order to convince others to spend time on a game that might think is too complicated for them or not for them, you need to have be a successful evangelist.

also the episode drifts into chatter about libraries, copyright and ebooks.

this week, i reviewed and published another batch of works for our institutional repository from our department of history that was prepared by our library assistants at leddy at this point, we have reviewed and uploaded the works of half the faculty from this department. i’m hoping to finish the rest this month but i think i have some outstanding h p work that might push the end of this project til march.

this morning i assisted with an online workshop called data analysis and visualization in r for ecologists that was being lead by a colleague of mine.

r version . . (“bunny-wunnies freak out”) was released on - - .

the release of r . . (“lost library book”) is scheduled for monday - - .

on sunday, i published a short response to “windsor works – an economic development strategy” which is going to city council on monday.

why am i writing about this document here?

i am mention this here because the proposed strategy (l.i.f.t.) lists the following as potential metric for measuring the strategy’s success…

take it from me, someone who knows a quite a bit about citations — the city should use another metric — perhaps one pertaining to local unemployment levels instead.

a viral post from resurfaced on my fb feed this week and unlike most of the posts i read there, this one did spark joy:

and it struck me how much i loved that the anti-prom was being at the library.

so i started doing some research!

it appears to me that some anti-proms are technically better described as alternative proms. these proms have been established as an explicitly safe place where lgbtq young people can enjoy prom. other anti-proms are true morps.

i now wonder what other anti-traditions should find a home at the public library.

- - t : : + : mita williams david rosenthal: chromebook linux update https://blog.dshr.org/ / /chromebook-linux-update.html my three acer c chromebooks running linux are still giving yeoman service, although for obvious reasons i'm not travelling these days. but it is time for an update to 's travels with a chromebook. below the fold, an account of some adventures in sysadmin.

battery replacementthe battery in c # , which is over six years old, would no longer hold a charge. i purchased a dentsing ap j k replacement battery from amazon. i opened the c , removed the old battery, inserted the new one, closed up the case, and all was well. it was an impressively easy fix. sleeping & wakingsometime around last october linux mint switched from kernels in the . series to kernels in the . series. mint uses . series kernels. the . kernels on the c went to sleep properly when the lid closed, and woke properly when the lid opened. the . kernels appeared to go to sleep correctly, but when the lid opened did a cold boot. because this problem happens immediately on wake, and because sleep appears to work correctly, there is no useful information in the logs; this appears to be a very hard problem to diagnose.

here is my work-around to use the . . - kernel (the last i installed via updates) on a vanilla linux mint installation:

install linux mint . mate edition.
add repositories using administration/software sources:
- deb http://archive.ubuntu.com/ubuntu bionic main restricted universe multiverse
- deb http://archive.ubuntu.com/ubuntu bionic-updates main restricted universe multiverse
- deb http://security.ubuntu.com/ubuntu/ bionic-security main restricted universe multiverse
install kernel . . - -generic:

sudo apt-get install linux-headers- . . - linux-headers- . . - -generic linux-image- . . - -generic linux-modules- . . - -generic linux-modules-extra- . . - -generic
edit /etc/default/grub to show the menu of kernels:

grub_timeout_style=menu grub_timeout=
edit /etc/default/grub so that your most recent choice of kernel becomes the default:

grub_savedefault=true grub_default=saved
run update-grub

after you choose the . . - kernel the first time, it should boot by default, and sleep and wake should work properly. the problem with ehci-pci on wakeup has gone away, there is no need to install the userland files from galliumos.
disk & home directory encryptionnote that you should ideally install linux mint . with full-disk encryption. the release notes explain:

the move to systemd caused a regression in ecrypts which is responsible for mounting/unmounting encrypted home directories when you login and logout. because of this issue, please be aware that in mint and newer releases, your encrypted home directory is no longer unmounted on logout: https://bugs.launchpad.net/ubuntu/+source/gnome-session/+bug/ .

mint with a full-disk encryption had this problem but i haven't been able to reproduce it with mint and the . . - kernel. home directory encryption works, but will leave its contents decrypted after you log out, rather spoiling the point. touchpadas i described here, the touchpad isn't one of the c 's best features, and it is necessary to disable it while typing, or while using a mouse such as the excellent tracpoint. i use ataraeo's touchpad-indicator, but this doesn't seem to work on mint . out of the box, using x.org's libinput driver. the release notes discuss using the synaptics driver instead. i installed it and, after creating the directory ~/.config/autostart the touchpad-indicator starts on login and works fine.
- - t : : + : david. (noreply@blogger.com) open knowledge foundation: what is a public impact algorithm? https://blog.okfn.org/ / / /what-is-a-public-impact-algorithm/

meg foulkes discusses public impact algorithms and why they matter.

“when i look at the picture of the guy, i just see a big black guy. i don’t see a resemblance. i don’t think he looks like me at all.”

this is what robert williams said to police when he was presented with the evidence upon which he was arrested for stealing watches in june . williams had been identified by an algorithm, when detroit police ran grainy security footage from the theft through a facial recognition system. before questioning williams, or checking for any alibi, he was arrested. it was not until the matter came to trial that detroit police admitted that he had been falsely, and solely, charged on the output of an algorithm.

it’s correct to say that in many cases, when ai and algorithms go wrong, the impact is pretty innocuous – like when a music streaming service recommends music you don’t like. but often, ai and algorithms go wrong in ways that cause serious harm, as in the case of robert williams. although he had done absolutely nothing wrong, he was deprived of a fundamental right on the basis of a computer output: his liberty.

it’s not just on an individual scale that these harms are felt. algorithms are written by humans, so they can reflect human biases. what algorithms can do is amplify, through automatedly entrenching the bias, this prejudice over a massive scale.

the bias isn’t exclusively racialised; last year, an algorithm used to determine exam grades disproportionately downgraded disadvantaged students. throughout the pandemic, universities have been turning to remote proctoring software that falsely identifies students with disabilities as cheats. for example, those who practice self-stimulatory behaviour or ‘stimming’ may get algorithmically flagged again and again for suspicious behaviour, or have to disclose sensitive medical information to avoid this.

we identify these types of algorithms as ‘public impact algorithms’ to clearly name the intended target of our concern. there is a big difference between the harm caused by inaccurate music suggestions and algorithms that have the potential to deprive us of our fundamental rights. to call out these harms, we have to precisely define the problem. only then can we hold the deployers of public impact algorithms to account, and ultimately to achieve our mission of ensuring public impact algorithms do no harm.

sign up to our mailing list or follow the open knowledge justice programme on twitter to receive updates.

- - t : : + : meg foulkes ed summers: outgoing https://inkdroid.org/ / / /legislators-social/

mark graham posed a question to me earlier today asking what we know about the twitter accounts of the members of congress, specifically whether they have been removed after they left office. the hypothesis was that some members of the house and senate may decide to delete their account on leaving dc.

i was immediately reminded of the excellent congress-legislators project which collects all kinds of information about house and senate members including their social media accounts into yaml files that are versioned in a github repository. github is a great place to curate a dataset like this because it allows anyone with a github account to contribute to editing the data, and to share utilities to automate checks and modifications.

unfortunately the file that tracks social media accounts is only for current members. once they leave office they are removed from the file. the project does track other historical information for legislators. but the social media data isn’t pulled in when this transition happens, or so it seems.

luckily git doesn’t forget. since the project is using a version control system all of the previously known social media links are in the history of the repository! so i wrote a small program that uses gitpython to walk the legislators-social-media.yaml file backwards in time through each commit, parse the yaml at that previous state, and merge that information into a union of all the current and past legislator information. you can see the resulting program and output in us-legislators-social.

there’s a little bit of a wrinkle in that not everything in the version history should be carried forward because errors were corrected and bugs were fixed. without digging into the diffs and analyzing them more it’s hard to say whether a commit was a bug fix or if it was simply adding new or deleting old information. if the yaml doesn’t parse at a particular state that’s easy to ignore.

it also looks like the maintainers split out account ids from account usernames at one point. derek willis helpfully pointed out to me that twitter don’t care about the capitalization of usernames in urls, so these needed to be normalized when merging the data. the same is true of facebook, instagram and youtube. i guarded against these cases but if you notice other problems let me know.

with the resulting merged historical data it’s not too hard to write a program to read in the data, identify the politicians who left office after the th congress, and examine their twitter accounts to see that they are live. it turned out to be a little bit harder than i expected because it’s not as easy as you might think to check if a twitter account is live or not.

twitter’s web servers return a http ok message even when responding to requests for urls of non-existent accounts. to complicate things further the error message that displays indicating it is not an account only displays when the page is rendered in a browser. so a simple web scraping job that looks at the html is not sufficient.

and finally just because a twitter username no longer seems to work, it’s possible that the user has changed it to a new screen_name. fortunately the unitedstates project also tracks the twitter user id (sometimes). if the user account is still there you can use the twitter api to look up their current screen_name and see if it is different.

after putting all this together it’s possible to generate a simple table of legislators who left office at the end of the th congress, and their twitter account information.

name	url	url_ok	new_url
lamar alexander	https://twitter.com/senalexander	true
michael b. enzi	https://twitter.com/senatorenzi	true
pat roberts	https://twitter.com/senpatroberts	true
tom udall	https://twitter.com/senatortomudall	true
justin amash	https://twitter.com/justinamash	true
rob bishop	https://twitter.com/reprobbishop	true
k. michael conaway	https://twitter.com/conawaytx	true
susan a. davis	https://twitter.com/repsusandavis	false
eliot l. engel	https://twitter.com/repeliotengel	true
bill flores	https://twitter.com/repbillflores	false
cory gardner	https://twitter.com/sencorygardner	true
peter t. king	https://twitter.com/reppeteking	true
steve king	https://twitter.com/stevekingia	true
daniel lipinski	https://twitter.com/replipinski	true
david loebsack	https://twitter.com/daveloebsack	true
nita m. lowey	https://twitter.com/nitalowey	true
kenny marchant	https://twitter.com/repkenmarchant	true
pete olson	https://twitter.com/reppeteolson	true
martha roby	https://twitter.com/repmartharoby	false	https://twitter.com/martharobyal
david p. roe	https://twitter.com/drphilroe	true
f. james sensenbrenner, jr.	https://twitter.com/jimpressoffice	false
josé e. serrano	https://twitter.com/repjoseserrano	true
john shimkus	https://twitter.com/repshimkus	true
mac thornberry	https://twitter.com/mactxpress	true
scott r. tipton	https://twitter.com/reptipton	true
peter j. visclosky	https://twitter.com/repvisclosky	true
greg walden	https://twitter.com/repgregwalden	true
rob woodall	https://twitter.com/reprobwoodall	true
ted s. yoho	https://twitter.com/reptedyoho	true
doug collins	https://twitter.com/repdougcollins	true
tulsi gabbard	https://twitter.com/tulsipress	true
susan w. brooks	https://twitter.com/susanwbrooks	true
joseph p. kennedy iii	https://twitter.com/repjoekennedy	false	https://twitter.com/joekennedy
george holding	https://twitter.com/repholding	true
denny heck	https://twitter.com/repdennyheck	false	https://twitter.com/ltgovdennyheck
bradley byrne	https://twitter.com/repbyrne	true
ralph lee abraham	https://twitter.com/repabraham	true
will hurd	https://twitter.com/hurdonthehill	true
david perdue	https://twitter.com/sendavidperdue	true
mark walker	https://twitter.com/repmarkwalker	true
francis rooney	https://twitter.com/reprooney	true
paul mitchell	https://twitter.com/reppaulmitchell	true
doug jones	https://twitter.com/sendougjones	true
tj cox	https://twitter.com/reptjcox	true
gilbert ray cisneros, jr.	https://twitter.com/repgilcisneros	true
harley rouda	https://twitter.com/repharley	true
ross spano	https://twitter.com/reprossspano	true
debbie mucarsel-powell	https://twitter.com/repdmp	true
donna e. shalala	https://twitter.com/repshalala	false
abby finkenauer	https://twitter.com/repfinkenauer	true
steve watkins	https://twitter.com/rep_watkins	false
xochitl torres small	https://twitter.com/reptorressmall	true
max rose	https://twitter.com/repmaxrose	true
anthony brindisi	https://twitter.com/repbrindisi	true
kendra s. horn	https://twitter.com/repkendrahorn	false	https://twitter.com/kendrashorn
joe cunningham	https://twitter.com/repcunningham	true
ben mcadams	https://twitter.com/repbenmcadams	false	https://twitter.com/benmcadamsut
denver riggleman	https://twitter.com/repriggleman	true

in most cases where the account has been updated the individual simply changed their twitter username, sometimes remove “rep” from it–like repjoekennedy to joekennedy. as an aside i’m kind of surprised that twitter username wasn’t taken to be honest. maybe that’s a perk of having a verified account or of being a politician? but if you look closely you can see there were a few that seemed to have deleted their account altogether:

name	url	url_ok
susan a. davis	https://twitter.com/repsusandavis	false
bill flores	https://twitter.com/repbillflores	false
f. james sensenbrenner, jr.	https://twitter.com/jimpressoffice	false
donna e. shalala	https://twitter.com/repshalala	false
steve watkins	https://twitter.com/rep_watkins	false

there are two notable exceptions to this. the first is vice president kamala harris. my logic for determining if a person was leaving congress was to see if they served in a term ending on - - , and weren’t serving in a term starting then. but harris is different because her term as a senator is listed as ending on - - . her old account (???) is no longer available, but her twitter user id is still active and is now attached to the account at (???). the other of course is joe biden, who stopped being a senator in order to become the president. his twitter account remains the same at (???).

it’s worth highlighting here how there seems to be no uniform approach to handling this process. in one case (???) is temporarily blessed as the vp, with a unified account history underneath. in the other there is a separation between (???) and (???). it seems like twitter has some work to do on managing identities, or maybe the congress needs to prescribe a set of procedures? or maybe i’m missing part of the picture, and that just as (???) somehow changed back to (???) there is some namespace management going on behind the scenes?

if you are interested in other social media platforms like facebook, instagram and youtube the unitedstates project tracks information for those platforms too. i merged that information into the legislators.yaml file i discussed here if you want to try to check them. i think that one thing this experiment shows is that if the platform allows for usernames to be changed it is critical to track the user id as well. i didn’t do the work to check that those accounts exist. but that’s a project for another day.

i’m not sure this list of five deleted accounts is terribly interesting at the end of all this. possibly? but on the plus side i did learn how to interact with git better from python, which is something i can imagine returning to in the future. it’s not every day that you have to think of the versions of a dataset as an important feature of the data, outside of serving as a backup that can be reverted to if necessary. but of course data changes in time, and if seeing that data over time is useful, then the revision history takes on a new significance. it’s nothing new to see version control systems as critical data provenance technologies, but it felt new to actually use one that way to answer a question. thanks mark!

- - t : : + : david rosenthal: stablecoins https://blog.dshr.org/ / /stablecoins.html i have long been skeptical of bitcoin's "price" and, despite its recent massive surge, i'm still skeptical. but it turns out i was wrong two years ago when i wrote in blockchain: what's not to like?:

permissionless blockchains require an inflow of speculative funds at an average rate greater than the current rate of mining rewards if the "price" is not to collapse. to maintain bitcoin's price at $ k requires an inflow of $ k/hour.

i found it hard to believe that this much actual money would flow in, but since then bitcoin's "price" hasn't dropped below $ k, so i was wrong. caution — i am only an amateur economist, and what follows below the fold is my attempt to make sense of what is going on.

first, why did i write that? the economic argument is that, because there is a low barrier to entry for new competitors, margins for cryptocurrency miners are low. so the bulk of their income in terms of mining rewards has to flow out of the system in "fiat" currency to pay for their expenses such as power and hardware. these cannot be paid in cryptocurrencies. at the time, the bitcoin block reward was . btc/block, or btc/hour. at $ k/btc this was $ k/hour, so on average k usd/hour had to flow in from speculators if the system was not to run out of usd.

source

what has happened since then? miners' income comes in two parts, transaction fees (currently averaging around btc/day) and mining rewards ( btc/day) for a total around k btc/day. at $ k/btc, that is $ k/hour. the combination of halving of the block reward, increasing transaction fees, and quintupling the "price" has roughly tripled the required inflow.

second, lets set the context for what has happened in cryptocurrencies in the last year.

source

in the last year bitcoin's "market cap" went from around $ b to around $ b ( . x) and its "price" went from about $ k to about $ k.

source

in the last year ethereum's "market cap" went from around $ b to around $ b ( . x) and its "price went from around $ to around $ .

source

the key observation that explains why i write "price" in quotes is shown in this graph. very little of the trading in btc is in terms of usd, most of it is in terms of tether (usdt). the "price" is set by how many usdt people are prepared to pay for btc, not by how many usd. the usd "price" follows because people believe that usdt ≅ usd.

source

in the past year, tether's "market cap" has gone from about b usdt to about b usdt ( x).

tether (usdt) is a "stablecoin", intended to maintain a stable price of usd = usdt. initially, tether claimed that it maintained a stable "price" because every usdt was backed by an actual usd in a bank account. does that mean that investors transferred around sixteen billion us dollars into tether's bank account in the past year? no-one believes that. there has never been an audit of tether to confirm what is backing usdt. tether themselves admitted to the new york attorney general in october that:

the $ . billion worth of tethers are only % backed:
tether has cash and cash equivalents (short term securities) on hand totaling approximately $ . billion, representing approximately percent of the current outstanding tethers.

if usdt isn't backed by usd, what is backing it, and is usdt really worth usd?

source

just in october, tether minted around b usdt. the graph tracks the "price" of bitcoin against the "market cap" of usdt. does it look like they're correlated? amy castor thinks so.

tether transfers newly created usdt to an exchange, where one of two things can happen to it:

it can be used to buy usd or an equivalent "fiat" currency. but only a few exchanges allow this. for example, coinbase, the leading regulated exchange, will not provide this "fiat off-ramp":

please note that coinbase does not support usdt — do not send it to your bitcoin account on coinbase.
because of usdt's history and reputation, exchanges that do offer a "fiat off-ramp" are taking a significant risk, so they will impose a spread; the holder will get less than $ . why would you send $ to tether to get less than $ back?
it can be used to buy another cryptocurrency, such as bitcoin (btc) or ethereum (eth), increasing demand for that cryptocurrency and thus increasing its price.

since newly created usdt won't be immediately sold for "fiat", they will pump the "price" of cryptocurrencies.

for simplicity of explanation, lets imagine a world in which there are only usd, usdt and btc. in this world some proportion of the backing for usdt is usd and some is btc.

someone sends usd to tether. why would they do that? they don't want usdt as a store of value, because they already have usd, which is obviously a better store of value than usdt. they want usdt in order to buy btc. tether adds the usd to the backing for usdt, and issues the corresponding number of usdt, which are used to buy btc. this pushes the "price" of btc up, which increases the "value" of the part of the backing for usdt that is btc. so tether issues the corresponding amount of usdt, which is used to buy btc. this pushes the "price" of btc up, which increases the "value" of the part of the backing for usdt that is btc. ...

tether has a magic "money" pump, creating usdt out of thin air. but there is a risk. suppose for some reason the "price" of btc goes down, which reduces the "value" of the backing for usdt. now there are more usdt in circulation than are backed. so tether must buy some usdt back. they don't want to spend usd for this, because they know that usd are a better store of value than usdt created out of thin air. so they need to sell btc to get usdt. this pushes the "price" of btc down, which reduces the "value" of the part of the backing for usdt that is btc. so tether needs to buy more usdt for btc, which pushes the "price" of btc down. ...

the magic "money" pump has gone into reverse, destroying the usdt that were created out of thin air. tether obviously wants to prevent this happening, so in our imaginary world what we would expect to see is that whenever the "price" of btc goes down, tether supplies the market with usdt, which are used to buy btc, pushing the price back up. over time, the btc "price" would generally go up, keeping everybody happy. but there is a second-order effect. over time, the proportion of the backing for usdt that is btc would go up too, because each usd that enters the backing creates r> usd worth of "value" of the btc part of the backing. and, over time, this effect grows because the greater the proportion of btc in the backing, the greater r becomes.

source

in our imaginary world we would expect to see:

the "price" of btc correlated with the number of usdt in circulation. the graph shows this in the real world.
both the "price" of btc and the number of usdt in circulation growing exponentially. the graph shows this in the real world.
spikes in the number of usdt in circulation following falls in the "price" of btc. is bitcoin really untethered? by john griffin and amit shams shows that:
rather than demand from cash investors, these patterns are most consistent with the supply‐based hypothesis of unbacked digital money inflating cryptocurrency prices.
their paper was originally published in and updated in and .
tether being extremely reluctant to be audited because that would reveal how little money and how much "money" was supporting the btc "price".

our imaginary world replicates key features of the real world. of course, since tether has never been audited, we don't know the size or composition of usdt's backing. so we don't know whether tether has implemented a magic "money" pump. but the temptation to get rich quick by doing so clearly exists, and tether's history isn't reassuring about their willingness to skirt the law. because of the feedback loops i described, if they ever dipped a toe in the flow from a magic "money" pump, they would have to keep doubling down.

apart from the work of griffin and shams, there is a whole literature pointing out the implausibility of tether's story. here are a few highlights:

jp konig's things about tether stablecoins
social capital's series explaining tether and the "stablecoin" scam:
price manipulation in the bitcoin ecosystem by neil gandal et al
cryptocurrency pump-and-dump schemes by tao li et al
patrick mckenzie's tether: the story so far:
a friend of mine, who works in finance, asked me to explain what tether was.

short version: tether is the internal accounting system for the largest fraud since madoff.

bernie madoff's $ . b ponzi scheme was terminated in but credible suspicions had been raised nine years earlier, not least by the indefatigable harry markopolos. credible suspicions were raised against wirecard shortly after it was incorporated in , but even after the financial times published a richly documented series based on whistleblower accounts it took almost a year before wirecard declared bankruptcy owing € . b.

massive frauds suffer from a "wile e. coyote" effect. because they are "too big to fail" there is a long time between the revelation that they are frauds, and the final collapse. it is hard for people to believe that, despite numbers in the billions, there is no there there. both investors and regulators get caught up in the excitement and become invested in keeping the bubble inflated by either attacking or ignoring negative news. for example, we saw this in the wirecard scandal:

bafin conducted multiple investigations against journalists and short sellers because of alleged market manipulation, in response to negative media reporting of wirecard. ... critics cite the german regulator, press and investor community's tendency to rally around wirecard against what they perceive as unfair attack. ... after initially defending bafin's actions, its president felix hufeld later admitted the wirecard scandal is a "complete disaster".

similarly, the cryptocurrency world has a long history of both attacking and ignoring realistic critiques. an example of ignoring is the dao:

the decentralized autonomous organization (the dao) was released on ^th april , but on ^th may dino mark, vlad zamfir, and emin gün sirer posted a call for a temporary moratorium on the dao, pointing out some of its vulnerabilities; it was ignored. three weeks later, when the dao contained about % of all the ether in circulation, a combination of these vulnerabilities was used to steal its contents.

source

the graph shows how little of the trading in btc is in terms of actual money, usd. on coinmarketcap.com as i write, usdt has a "market cap" of nearly $ b and the next largest "stablecoin" is usdc, at just over $ . b. usdc ~~is audited and~~[ ] complies with banking regulations, which explains why it is used so much less. the supply of usdc can't expand enough to meet demand. the total "market cap" of all the cryptocurrencies the site tracks is $ b, an increase of more than % in the last day! so just one day is around the same as bernie madoff's ponzi scheme. the top cryptocurrencies (btc, eth, xrp, usdt) account for $ b ( %) of the total "market cap"; the others are pretty insignificant.

david gerard points out the obvious in tether is “too big to fail” — the entire crypto industry utterly depends on it:

the purpose of the crypto industry, and all its little service sub-industries, is to generate a narrative that will maintain and enhance the flow of actual dollars from suckers, and keep the party going.

increasing quantities of tethers are required to make this happen. we just topped twenty billion alleged dollars’ worth of tethers, sixteen billion of those just since march . if you think this is sustainable, you’re a fool.

gerard links to bryce weiner's hopes, expectations, black holes, and revelations — or how i learned to stop worrying and love tether which starts from the incident in april of when bitfinex, the cryptocurrency exchange behind tether, encountered a serious problem:

the wildcat bank backing tether was raided by interpol for laundering of criminally obtained assets to the tune of about $ , , . the percentage of that sum which was actually bitfinex is a matter of some debate but there’s no sufficient reason not to think it was all theirs.
...
the nature of the problem also presented a solution: instead of backing tether in actual dollars, stuff a bunch of cryptocurrency in a basket to the valuation of the cash that got seized and viola! a black hole is successfully filled with a black hole, creating a stable asset.

at the time, usdt's "market cap" was around $ . b, so assuming tether was actually backed by usd at that point, it lost % of its backing. this was a significant problem, more than enough to motivate shenanigans.

weiner goes on to provide a detailed explanation, and argue that tether is impossible to shut down. he may be right, but it may be possible to effectively eliminate the "fiat off-ramp", thus completely detaching usdt and usd. this would make it clear that "prices" expressed in usdt are imaginary, not the same as prices expressed in usd.

source

postscript: david gerard recounts the pump that pushed btc over $ k:

we saw about million tethers being lined up on binance and huobi in the week previously. these were then deployed en masse.

you can see the pump starting at : utc on december. btc was $ , . on coinbase at : utc. notice the very long candles, as bots set to sell at $ , sell directly into the pump.

see cryptocurrency pump-and-dump schemes by tao li, donghwa shin and baolian wang.

source

ki joung yu watched the pump in real time:

lots of people deposited stablecoins to exchanges mins before breaking $ k.

price is all about consensus. i guess the sentiment turned around to buy $btc at that time.
...
eth block interval is - seconds.

this chart means exchange users worldwide were trying to deposit #stablecoins in a single block — seconds.

note that " mins" is about one bitcoin block time, and by "exchange users" he means "addresses — it could have been a pre-programmed "smart contract".

[ ] david gerard points out that:

usdc loudly touts claims that it’s well-regulated, and implies that it’s audited. but usdc is not audited — accountants grant thornton sign a monthly attestation that centre have told them particular things, and that the paperwork shows the right numbers.

- - t : : + : david. (noreply@blogger.com) jonathan rochkind: product management https://bibwild.wordpress.com/ / / /product-management/

in my career working in the academic sector, i have realized that one thing that is often missing from in-house software development is “product management.”

but what does that mean exactly? you don’t know it’s missing if you don’t even realize it’s a thing and people can use different terms to mean different roles/responsibilities.

basically, deciding what the software should do. this is not about colors on screen or margins (what our stakeholderes often enjoy micro-managing) — i’d consider those still the how of doing it, rather than the what to do. the what is often at a much higher level, about what features or components to develop at all.

when done right, it is going to be based on both knowledge of the end-user’s needs and preferences (user research); but also knowledge of internal stakeholder’s desires and preferences (overall organiational strategy, but also just practically what is going to make the right people happy to keep us resourced). also knowledge of the local capacity, what pieces do we need to put in place to get these things developed. when done seriously, it will necessarily involve prioritization — there are many things we could possibly done, some subset of them we very well may do eventually, but which ones should we do now?

my experience tells me it is a very big mistake to try to have a developer doing this kind of product management. not because a developer can’t have the right skillset to do them. but because having the same person leading development and product management is a mistake. the developer is too close to the development lense, and there’s just a clarification that happens when these roles are separate.

my experience also tells me that it’s a mistake to have a committee doing these things, much as that is popular in the academic sector. because, well, just of course it is.

but okay this is all still pretty abstract. things might become more clear if we get more specific about the actual tasks and work of this kind of product management role.

i found damilola ajiboye blog post on “product manager vs product marketing manager vs product owner” very clear and helpful here. while it is written so as to distinguish between three different product management related roles, but ajiboye also acknowledges that in a smaller organization “a product manager is often tasked with the duty of these roles.

regardless of if the responsibilities are to be done by one or two or three person, ajiboye’s post serves as a concise listing of the work to be done in managing a product — deciding the what of the product, in an ongoing iterative and collaborative manner, so that developers and designers can get to the how and to implementation.

i recommend reading the whole article, and i’ll excerpt much of it here, slightly rearranged.

the product manager
these individuals are often referred to as mini ceos of a product. they conduct customer surveys to figure out the customer’s pain and build solutions to address it. the pm also prioritizes what features are to be built next and prepares and manages a cohesive and digital product roadmap and strategy.
the product manager will interface with the users through user interviews/feedback surveys or other means to hear directly from the users. they will come up with hypotheses alongside the team and validate them through prototyping and user testing. they will then create a strategy on the feature and align the team and stakeholders around it. the pm who is also the chief custodian of the entire product roadmap will, therefore, be tasked with the duty of prioritization. before going ahead to carry out research and strategy, they will have to convince the stakeholders if it is a good choice to build the feature in context at that particular time or wait a bit longer based on the content of the roadmap.
the product marketing manager
the pmm communicates vital product value — the “why”, “what” and “when” of a product to intending buyers. he manages the go-to-market strategy/roadmap and also oversees the pricing model of the product. the primary goal of a pmm is to create demand for the products through effective messaging and marketing programs so that the product has a shorter sales cycle and higher revenue.
the product marketing manager is tasked with market feasibility and discovering if the features being built align with the company’s sales and revenue plan for the period. they also make research on how sought-after the feature is being anticipated and how it will impact the budget. they communicate the values of the feature; the why, what, and when to potential buyers — in this case users in countries with poor internet connection.
[while expressed in terms of a for-profit enterprise selling something, i think it’s not hard to translate this to a non-profit or academic environment. you still have an audience whose uptake you need to be succesful, whether internal or external. — jrochkind ]
the product owner
a product owner (po) maximizes the value of a product through the creation and management of the product backlog, creation of user stories for the development team. the product owner is the customer’s representative to the development team. he addresses customer’s pain points by managing and prioritizing a visible product backlog. the po is the first point of call when the development team needs clarity about interpreting a product feature to be implemented.
the product owner will first have to prioritize the backlog to see if there are no important tasks to be executed and if this new feature is worth leaving whatever is being built currently. they will also consider the development effort required to build the feature i.e the time, tools, and skill set that will be required. they will be the one to tell if the expertise of the current developers is enough or if more engineers or designers are needed to be able to deliver at the scheduled time. the product owner is also armed with the task of interpreting the product/feature requirements for the development team. they serve as the interface between the stakeholders and the development team.

when you have someone(s) doing these roles well, it ensures that the development team is actually spending time on things that meet user and business needs. i have found that it makes things so much less stressful and more rewarding for everyone involved.

when you have nobody doing these roles, or someone doing it in a cursory or un-intentional way not recognized as part of their core job responsibilities, or have a lead developer trying to do it on top of develvopment, i find it leads to feelings of: spinning wheels, everything-is-an-emergency, lack of appreciation, miscommunication and lack of shared understanding between stakeholders and developers, general burnout and dissatisfaction — and at the root, a product that is not meeting user or business needs well, leading to these inter-personal and personal problems.

- - t : : + : jrochkind islandora: islandora open meeting: february , https://islandora.ca/content/islandora-open-meeting-february- - islandora open meeting: february , manez wed, / / - :

body

we will be holding another open drop-in session on tuesday, february from : am to : pm eastern. full details, and the zoom link to join, are in this google doc. the meeting is free form, with experienced islandora users on hand to answer questions or give demos on request. please drop in at any time during the four-hour window.

registration is not required. if you would like a calendar invite as a reminder, please let us know at community@islandora.ca.

- - t : : + : manez hangingtogether: emerging roles for libraries in bibliometric and research impact analysis: lessons learned from the university of waterloo http://feedproxy.google.com/~r/hangingtogetherorg/~ /qetext_pdsk/

library support for bibliometrics and research impact (bri) analysis is a growing area of library investment and service. not just in the provision of services to researchers, but for the institutions themselves, which increasingly need to quantify research impact for a spectrum of internally and externally motivated purposes, such as strategic decision support, benchmarking, reputation analysis, support for funding requests, and to better understand research performance.

research libraries are adopting new roles to support bibliometrics and research impact analysis, and the university of waterloo library’s efforts have caught my attention for some time, and for two specific reasons:

they are leaders in the area of bibliometrics and research impact in north america
they have exemplified exceptional cross-institutional collaboration—what i might call “social interoperability”–in developing services, staffing, and programs.

alison hitchens, associate university librarian for collections, technology, and scholarly communication, and laura bredahl, bibliometrics and research impact librarian, recently shared about their activities in an oclc research library partnership (rlp) works in progress webinar presentation entitled case study—supporting bibliometric and research impact analysis at the university of waterloo. their efforts also will be described in a forthcoming arl library practice brief on supporting bibliometric data needs at academic institutions.

rlp works in progress webinar: case study—supporting bibliometric and research impact analysis at the university of waterloo the waterloo bri story

like many institutions, the library at waterloo has been supporting individual researchers with bibliometrics information for reputation management for over a decade. however, around the university recognized that it needed an institutional understanding of bibliometrics because important external stakeholders like funders, governments, and other accountability organizations were using them to evaluate their organization. additionally, as the campus developed a new strategic plan emphasizing transformational research, it also needed indicators to help chart progress. as a result, the provost established a working group on bibliometrics that included cross-institutional representation from the office of research, office of institutional analysis, library, and faculties, with the goal to provide guidance to the university on the effective and appropriate use of bibliometrics.

this working group led to a few significant outcomes:

first, it led to the realization and recommendation that a campus expert was needed, which led to the creation of a new bibliometrics and research impact librarian role in . this library professional works at the institutional level, providing expertise, leadership, and support for institutional needs.
secondly, it led to the publication in of a white paper on measuring research output through bibliometrics. this type of white paper is critical for any institution utilizing bibliometrics for decision making, as it requires institutional investment to understand and document the opportunities and limitations of bibliometric analysis. it also offers transparent guidance for how researchers and administrators can use bibliometric analysis responsibly. another good example of this type of white paper comes from virginia tech, developed as a companion to that institution’s strategic planning efforts.

the white paper was followed by the development of a research metrics framework, intended to provide detailed bibliometric indicators related to the work of research institutes supporting key research areas identified in the - strategic plan. and this in turn was followed in by the development of an internal bibliometric assessment tools report, offering an extremely detailed review of existing bibliometrics assessment tools, known use cases, and an overview of other prominent bibliometrics tools. the working group on bibliometrics continues its work today, supporting the current strategic plan, particularly by advising on the definitions of research areas and the responsible use of indicators at the institutional level.

so what does a bibliometrics and research impact librarian do?

laura described several examples of her work at waterloo:

validating university rankings. a variety of rankings, such as cwts leiden, shanghai’s arwu, macleans (canada), qs, and times higher education, are all closely watched by academic institutions and each has their own ranking methodologies. in general, laura works to understand and verify the data from the rankings, and her efforts also serve to monitor institutional performance, and to provide local level analysis to better understand the rankings.
support and training to others. laura is not the only person on campus conducting bibliometric analysis. for example, the faculty of engineering has a dedicated research analyst, and laura provides consultation and expert guidance on the use of tools like scival. laura has also developed an informal community of practice, open to anyone on campus, which is intended to support knowledge sharing and discussion.
strategic planning. laura supports efforts to identify appropriate bibliometrics indicators that can be used to understand progress on the current strategic plan.

importance to the library

libraries seem to me to be a natural place for bibliometrics and research impact leadership. librarians have expertise across the research and scholarly communications life cycle, understand disciplinary differences—and how these impact bibliometrics—and also have extensive knowledge with bibliographic data and tools.

in general, this type of engagement can also positively impact the library by “raising the profile of the library on campus.” for example, in the webinar alison commented,

“it was clear to me that being connected to and known by high level administration in the office of research really had an impact on building partnerships in other areas such as research data management. it was a lot easier to send an email or pick up the phone and call an avp of research because they knew me through the working group on bibliometrics.”

overall, this type of activity may result in greater stakeholder appreciation for the value proposition of the library, an improved understanding of the scope of library expertise, and more invitations for the library to participate on campus committees and enterprise-wide projects. at waterloo, for example, this included opportunities for the aul to join the office of research systems advisory group and for the library to contribute to the institutional rim project. as the new strategic planning effort has launched, and seven working groups were formed to develop background papers, the library was in a position to successfully advocate for a librarian on each committee.

learn more! continue the conversation with the oclc research library partnership

of course there’s much more to discuss, so we are offering affiliates with the research library partnership an opportunity to continue the conversation through informal small group discussions with alison and laura. please join us on the following dates:

europe and east coast-friendly time: wednesday, march at am edt (utc- ). this will be pm in the uk.
anz and american west coast-friendly time: wednesday, march at pm edt (utc- )/ pm pst (this should be am in melbourne/sydney on thursday the ^th)

we are interested in exploring some of these questions:

does your campus have an internal, cross-unit community of practice around bri? what is the role of your library?
what products/tools are being used at your institution?
how is your rim/cris system also being incorporated into these efforts?
what’s the status of institutional data governance and data sharing at your institution?

if you send me an email message, i will send you a calendar invitation (which can help to make sure all the time zones align).

the post emerging roles for libraries in bibliometric and research impact analysis: lessons learned from the university of waterloo appeared first on hanging together.

- - t : : + : rebecca bryant jez cope: glam data science network fellow travellers https://erambler.co.uk/blog/glam-data-science-network-fellow-travellers/

updates

- - thanks to gene @dzshuniper@ausglam.space for suggesting adho and a better attribution for the opening quote (see comments below for details)

see comments & webmentions for details.

“if you want to go fast, go alone. if you want to go far, go together.” — african proverb, probbly popularised in english by kenyan church leader rev. samuel kobia (original)

this quote is a popular one in the carpentries community, and i interpret it in this context to mean that a group of people working together is more sustainable than individuals pursuing the same goal independently. that’s something that speaks to me, and that i want to make sure is reflected in nurturing this new community for data science in galleries, archives, libraries & museums (glam). to succeed, this work needs to be complementary and collaborative, rather than competitive, so i want to acknowledge a range of other networks & organisations whose activities complement this.

the rest of this article is an unavoidably incomplete list of other relevant organisations whose efforts should be acknowledged and potentially built on. and it should go without saying, but just in case: if the work i’m planning fits right into an existing initiative, then i’m happy to direct my resources there rather than duplicate effort.

inspirations & collaborators

groups with similar goals or undertaking similar activities, but focused on a different sector, geographic area or topic. i think we should make as much use of and contribution to these existing communities as possible since there will be significant overlap.

code lib

probably the closest existing community to what i want to build, but primarily based in the us, so timezones (and physical distance for in-person events) make it difficult to participate fully. this is a well-established community though, with regular events including an annual conference so there’s a lot to learn here.

newcardigan

similar to code lib but an australian focus, so the timezone problem is even bigger!

glam labs

focused on supporting the people experimenting with and developing the infrastructure to enable scholars to access glam materials in new ways. in some ways, a glam data science network would be complementary to their work, by providing people not directly involved with building glam labs with the skills to make best use of glam labs infrastructure.

uk government data science community

another existing community with very similar intentions, but focused on uk government sector. clearly the british library and a few national & regional museums & archives fall into this, but much of the rest of the glam sector does not.

artifical intelligence for libraries, archives & museums (ai lam)

a multinational collaboration between several large libraries, archives and museums with a specific focus on the artificial intelligence (ai) subset of data science

uk reproducibility network

a network of researchers, primarily in heis, with an interest in improving the transparency and reliability of academic research. mostly science-focused but with some overlap of goals around ethical and robust use of data.

museums computer group

i’m less familiar with this than the others, but it seems to have a wider focus on technology generally, within the slightly narrower scope of museums specifically. again, a lot of potential for collaboration.

training

several organisations and looser groups exist specifically to develop and deliver training that will be relevant to members of this network. the network also presents an opportunity for those who have done a workshop with one of these and want to know what the “next steps” are to continue their data science journey.

supporters

these misson-driven organisations have goals that align well with what i imagine for the glam dsn, but operate at a more strategic level. they work by providing expert guidance and policy advice, lobbying and supporting specific projects with funding and/or effort. in particular, the ssi runs a fellowship programme which is currently providing a small amount of funding to this project.

digital preservation coalition (dpc)
software sustainability institute (ssi)
research data alliance (rda)
alliance of digital humanities organizations (adho)
- … and its libraries and digital humanities special interest group (lib&dh sig)

professional bodies

these organisations exist to promote the interests of professionals in particular fields, including supporting professional development. i hope they will provide communication channels to their various members at the least, and may be interested in supporting more directly, depending on their mission and goals.

conclusion

as i mentioned at the top of the page, this list cannot possibly be complete. this is a growing area and i’m not the only or first person to have this idea. if you can think of anything glaring that i’ve missed and you think should be on this list, leave a comment or tweet/toot at me!

- - t : : + : jez cope digital library federation: metadata during covid https://www.diglib.org/metadata-during-covid/

this post was written by members of the metadata working group, a subgroup of dlf’s assessment interest group.

digital collections work has changed in a number of ways during the covid- pandemic. for many libraries and archives, this has meant working remotely and shifting toward tasks that can be done online. within the dlf aig metadata working group, members have discussed a number of ways that organizations have chosen to increase capacity for metadata, transcription, and other tasks related to digital collections as a way of providing work for employees who would normally work in public-serving positions. this post documents some of those projects and activities.

university of north texas

at the university of north texas, our digital collections use a web-based metadata editing interface and we can add as many users as needed. when the stay-at-home order went into effect right after spring break, many of our library staff members (including full-time librarians/staff and part-time student workers) were no longer able to do their regular jobs and we offered metadata as an alternative. we added about new editors to our system in march . additionally, we added some quickly-drafted documentation to steer people toward easy metadata projects and known issues that require clean-up (like fixing name formatting). to keep oversight manageable, new editors were still attached to their own departments (or assigned to one that needed help), with a central contact person for each department and a specific sub-set of projects. our team of developers rushed an overhaul of the event tracking system that documents who is editing and what records they are changing so that managers could more easily verify if workers were editing when they said they were working. tracking edits has also let us measure how significantly overall editing has increased. multiple times since this started, we have had at least one editor working during every hour of the day. having so many relatively-untrained editors has resulted in a large number of issues that will need to be reviewed, but we have tools built into our system to help identify those issues and have added them to our ongoing list of things to fix. overall, this was still an extremely positive experience since the increase in editors allowed significant progress or completion of work that would not have been done otherwise. – hannah tarver

university of utah marriott library

at the university of utah, the covid- pandemic pivot to remote work prompted the launch of transcription projects, both with handwritten materials from special collections and newspaper ocr correction. this includes the transcription of , employee records by our digital operations student employees which resulted in the complete transcription of the kennecott miner records collection. we are also using omeka classic with the scripto plug-in as the platform for manuscript transcription projects and are able to find valuable work for people to engage in when they couldn’t physically be at the library.

in addition, we developed a born-digital crowdsourced digital collection, the utah covid- digital collection designed to capture what is currently happening during this unusual time. we’ve gotten a great response from the university and larger utah communities, with over contributions so far available in the digital library. the covid- digital collection has enabled us to build new partnerships and provided the library with outreach opportunities. an article detailing the project is forthcoming in a special issue of the journal digital library perspectives. – anna neatrour

utah state archives

after starting with from the page a few months earlier, moving staff and volunteers to transcription and indexing projects proved to be successful. contributors finished a historical court case (and now working on a second one) and a year’s worth of birth certificates in only a few months using the web-based interface that integrates with contentdm digital collections. with a built-in notes feature, questions can be asked and answered directly on a document’s page, which will then be exported along with the rest of the metadata. we are now preparing to open up the birth certificate indexing to the general public with additional training materials. in addition, new digital collections have been published, even with metadata developed remotely, using tools like google sheets for input and then converting to delimited text files for import. – gina strack

university of texas at austin

at the start of march, the university of texas libraries collections portal, the public-facing search and discovery interface for our digital asset management system (dams), included approximately , items. shortly after, the ut-austin campus closed and many staff members from the libraries’ first-line customer service, acquisitions and cataloging units found their roles pivoting to create metadata remotely for our dams system. collection curators within ut libraries created large-scale digital projects to help ensure continued remote work and to utilize this unusual time to turn their focus to projects that had been placed on the back burner due to more pressing obligations. our digital asset management system coordinator and staff from our preservation and digital stewardship unit created flexible pathways to support these projects and to ensure successful ingests into the dams. staff at the architecture & planning library and the alexander architectural archives, the nettie lee benson latin american collection, and the perry-castañeda library map collection dedicated themselves to ingesting and describing large amounts of digital items, increasing our total number of items available online to over , by september. digital objects newly available online as a result of this unprecedented, organization-wide collaborative effort include over , digitized architectural drawings and images, historic books from the benson rare book collection and primeros libros de las américas, and , scanned maps. the university of texas libraries documented the experience and provided a more detailed explanation of our dams in texlibris. – mandy ryan

colgate university

colgate university’s special collections and university archives (scua) is documenting the colgate community’s experiences and stories of covid- . digital contributions can be submitted at any time via a google form and may be added to colgate’s digital collections portal. there have been direct submissions as of october . physical donations of covid- related materials will be accepted once staff return to the library building. colgate’s metadata and cataloging (m&c) staff have been working with scua’s digital collections at home for the first time, describing the work of the university’s longest-serving official photographer, edward h. stone. stone documented life at colgate from the s to the s, and also photographed the people, places, businesses, and industry of the village of hamilton and madison county, new york. m&c staff are creating and editing metadata for more than glass plate negatives scanned by scua staff and students. we anticipate this will be a successful collaboration between the two departments that will serve as a model for other metadata-based remote work projects on campus. m&c staff have also worked with a born-digital lgbtq oral history project curated by students in the explorations in lgbtq studies class. oral history interviews with colgate graduates active in the struggle for lgbtq rights on campus from the s to the s is now available on the digital collections site – rachel white

digital library of georgia

most of our staff were able to continue doing most of our work from home, though some imaging projects shifted from actively imaging work (which would have had to be done in the office with our cameras) to working on image editing and curation work. we also had to postpone a meeting for our digitization partners. some metadata projects that were waiting on new imaging work were shifted to complete later; metadata staff worked on metadata remediation and metadata harvesting projects. one colleague who works on newspaper imaging was shifted over to a project describing moving image footage for the parade of quartets collection. we set up a student transcription project to keep students teleworking while they had to remain off-campus due to covid- . their transcription work was incorporated into our full-text accessibility feature for some smaller collections. students are now working in the office and from home on newspaper collation and metadata work, and our imaging staff have worked out a schedule to work while social distancing. our full staff meetings shifted from weekly meetings (in person) to daily meetings (via zoom). unit and supervisor meetings continue with the same frequency as they were held pre-covid. our quarter - newsletter and our quarter newsletter both provide more details of what we have worked on throughout the year. – mandy mastrovita

university of florida

since the middle of march , the digital support services (dss) at the libraries has shifted the focus of its imaging assistant crew. collaborating with the metadata staff, this crew has carried out site-wide metadata cleanup projects for the university of florida digital collection (ufdc) using ufdc’s online metadata edit form. these tasks can all be done at home using a computer connected to the internet with minimum instructions. the projects include adding missing system id numbers, unifying the spelling of language terms, correcting diacritic displays, updating rights statements, transcribing hand-written content, merging genre terms of different spelling variations to selected ones. so far, dss staff has modified over , rights statements and transcribed over , words. these projects improve the overall metadata quality dramatically. for instance, the genre terms in use will then be cut down to about from the original terms gathered from all data contributors over the years. to maintain this smaller selection of genre terms, the dss will also implement steps to assure all incoming content uses terms from the controlled genre list. – xiaoli ma

the ohio state university libraries

the onset of the covid- pandemic necessitated a shift to telework for university libraries’ employees. in collaboration with metadata initiatives and preservation & digitization, staff and student employees in other units and needing remote work to do were given the opportunity to do metadata telework. these entailed review and description of content for digital collections, a digital repository for digitized and born-digital special collections and archival materials. catalogers worked on remediation of legacy metadata records, particularly audio and image resources. staff and student employees with no prior metadata experience assisted with review and description of digitized audio and video content in the backlog. this group also contributed to metadata gathering and quality review for a large migration of digitized student newspapers. virtual collaboration was conducted with zoom, e-mail, and the university’s instance of box, a cloud-based content management system. this work has made a significant impact on the backlog for dc. however, metadata initiatives and applicable stakeholders are still reviewing the work that was done before making updates to records and ingesting the newly processed content. – annamarie klose

the post metadata during covid appeared first on dlf.

- - t : : + : gayle lucidworks: accelerate time to value for information retrieval with ai https://lucidworks.com/post/information-retrieval-with-ai/

we’ve organized the virtuous cycle of our ai and machine learning discipline to make it clear how customers can make the most of the data science innovation at their disposal.

the post accelerate time to value for information retrieval with ai appeared first on lucidworks.

- - t : : + : radu miclaus oclc dev network: planned maintenance: classify api http://www.oclc.org/content/developer/worldwide/en_us/news/ /classify-feb- .html

oclc will be performing quarterly maintenance on the experimental classify api on february from : am – : am eastern us (utc - ).

- - t : : + : karen coombs terry reese: marcedit . .x/ . .x (beta) updates https://blog.reeset.net/archives/

versions are available at: https://marcedit.reeset.net/downloads

information about the changes:

. . change log: https://marcedit.reeset.net/software/update .txt
. . change log: https://marcedit.reeset.net/software/update .txt

if you are using .x – this will prompt as normal for update. . .x is the beta build, please be aware i expect to be releasing updates to this build weekly and also expect to find some issues.

questions, let me know.

–tr

- - t : : + : reeset cynthia ng: choosing not to go into management (again) https://cynthiang.ca/ / / /choosing-not-to-go-into-management-again/ often, to move up and get a higher pay, you have to become a manager, but not everyone is suited to become a manager, and sometimes given the preference, it’s not what someone wants to do. thankfully at gitlab, in every engineering team including support, we have two tracks: technical (individual contributor), and management. progression … continue reading - - t : : + : cynthia harvard library innovation lab: new updates to search: advanced filters https://lil.law.harvard.edu/blog/ / / /new-updates-to-search-advanced-filters/

the caselaw access project offers free, public access to over . million decisions published by state and federal courts throughout american history. because our mission is providing access to legal information, we make these decisions available in a variety of formats through a variety of different access methods.

one type of access we've been working hard on recently is our search interface, which you can get to at case.law/search. we've had basic search working for a while, and we're pleased to share our new advanced search filters.

advanced filters work exactly as you'd expect. start your search with keywords or phrases, and then use the filters to narrow down jurisdictions, courts, and dates. say you're looking for massachusetts cases from to that contain the word "whaling."

search for cases that include the word "whaling" decided from to in massachusetts, showing advanced filters for full-text search, date from, date to, case name abbreviation, docket number, reporter, jurisdiction, citation, and court.

you can also access the advanced filters from the search results screen, so that you can fine-tune your search if you're not happy with the initial results. delete or modify any of the filters as you go, and sort the results chronologically or by relevance.

search results for cases that include the word "whaling" decided from to in massachusetts, showing filters on left.

there is a lot more we hope to do with search, but we hope you enjoy this improvement. if you have ideas of your own, please share them with us at info@case.law.

cap is a project of the library innovation lab at harvard law school library. we make open source software that helps people access legal information, preserve web sources with perma.cc, and create open educational resources with h o.

- - t : : + : open knowledge foundation: announcing a new partner for open data day mini-grants https://blog.okfn.org/ / / /announcing-a-new-partner-for-open-data-day- -mini-grants/

gfdrr and opendri

for open data day on saturday th march, the open knowledge foundation is offering support and funding for in-person and online events anywhere in the world via our mini-grant scheme.

today we are pleased to announce an additional partner for the open data day mini-grant scheme: the global facility for disaster reduction and recovery (gfdrr) through the gfdrr labs and its open data for resilience initiative (opendri).

gfdrr will be supporting mini-grants in the environmental data track, with a particular focus on ‘data for resilience’.

if you need inspiration for your event using data for resilience, some useful resources to check out include: gfdrr labs, opendri, open cities project, thinkhazard, open data for resilience index and the risk data library.

we are extremely grateful to gfdrr and all our partners who have provided funding for this year’s mini-grant scheme. these include microsoft, uk foreign, commonwealth and development office, mapbox, latin american open data initiative (ilda), open contracting partnership and datopian.

open data day partners

- - t : : + : james hamilton cynthia ng: reflection part : my first year at gitlab and becoming senior https://cynthiang.ca/ / / /reflection-part- -my-first-year-at-gitlab-and-on-becoming-senior/ about a year ago, i wrote a reflection on summit and contribute, our all staff events, and later that year, wrote a series of posts on the gitlab values and culture from my own perspective. there is a lot that i mention in the blog post series and i’ll try not to repeat myself (too … continue reading - - t : : + : cynthia cynthia ng: reflection part : my second year at gitlab and on becoming senior again https://cynthiang.ca/ / / /reflection-part- -my-second-year-at-gitlab-and-on-becoming-senior-again/ this reflection is a direct continuation of part of my time at gitlab so far. if you haven’t, please read the first part before beginning this one. becoming an engineer ( months) the more time i spent working in support, the more i realized that the job was much more technical than i originally … continue reading - - t : : + : cynthia jonathan rochkind: rails auto-scaling on heroku https://bibwild.wordpress.com/ / / /rails-auto-scaling-on-heroku/

we are investigating moving our medium-small-ish rails app to heroku.

we looked at both the rails autoscale add-on available on heroku marketplace, and the hirefire.io service which is not listed on heroku marketplace and i almost didn’t realize it existed.

i guess hirefire.io doesn’t have any kind of a partnership with heroku, but still uses the heroku api to provide an autoscale service. hirefire.io ended up looking more fully-featured and lesser priced than rails autoscale; so the main service of this post is just trying to increase visibility of hirefire.io and therefore competition in the field, which benefits us consumers.

background: interest in auto-scaling rails background jobs

at first i didn’t realize there was such a thing as “auto-scaling” on heroku, but once i did, i realized it could indeed save us lots of money.

i am more interested in scaling rails background workers than i a web workers though — our background workers are busiest when we are doing “ingests” into our digital collections/digital asset management system, so the work is highly variable. auto-scaling up to more when there is ingest work piling up can give us really nice inget throughput while keeping costs low.

on the other hand, our web traffic is fairly low and probably isn’t going to go up by an order of magnitude (non-profit cultural institution here). and after discovering that a “standard” dyno is just too slow, we will likely be running a performance-m or performance-l anyway — which likely can handle all anticipated traffic on it’s own. if we have an auto-scaling solution, we might configure it for web dynos, but we are especially interested in good features for background scaling.

there is a heroku built-in autoscale feature, but it only works for performance dynos, and won’t do anything for rails background job dynos, so that was right out.

that could work for rails bg jobs, the rails autoscale add-on on the heroku marketplace; and then we found hirefire.io.

pricing: pretty different hirefire

as of now january , hirefire.io has pretty simple and affordable pricing. $ /month/heroku application. auto-scaling as many dynos and process types as you like.

hirefire.io by default can only check into your apps metrics to decide if a scaling event can occur once per minute. if you want more frequent than that (up to once every seconds), you have to pay an additional $ /month, for $ /month/heroku application.

even though it is not a heroku add-on, hirefire does advertise that they bill pro-rated to the second, just like heroku and heroku add-ons.

rails autoscale

rails autoscale has a more tiered approach to pricing that is based on number and type of dynos you are scaling. starting at $ /month for - standard dynos, the next tier up is $ for up to standard dynos, all the way up to $ (!) for to dynos. if you have performance dynos involved, from $ /month for - performance dynos, up to $ /month for up to performance dynos.

for our anticipated uses… if we only scale bg dynos, i might want to scale from (low) or to (high) or standard dynos, so we’d be at $ /month. our web dynos are likely to be performance and i wouldn’t want/need to scale more than probably , but that puts us into performance dyno tier, so we’re looking at $ /month.

this is of course significantly more expensive than hirefire.io’s flat rate.

metric resolution

since hirefire had an additional charge for finer than -minute resolution on checks for autoscaling, we’ll discuss resolution here in this section too. rails autoscale has same resolution for all tiers, and i think it’s generally seconds, so approximately the same as hirefire if you pay the extra $ for increased resolution.

configuration

let’s look at configuration screens to get a sense of feature-sets.

rails autoscale web dynos

to configure web dynos, here’s what you get, with default values:

the metric rails autoscale uses for scaling web dynos is time in heroku routing queue, which seems right to me — when things are spending longer in heroku routing queue before getting to a dyno, it means scale up.

worker dynos

for scaling worker dynos, rails autoscale can scale dyno type named “worker” — it can understand ruby queuing libraries sidekiq, resque, delayed job, or que. i’m not certain if there are options for writing custom adapter code for other backends.

here’s what the configuration options are — sorry these aren’t the defaults, i’ve already customized them and lost track of what defaults are.

you can see that worker dynos are scaled based on the metric “number of jobs queued”, and you can tell it to only pay attention to certain queues if you want.

hirefire

hirefire has far more options for customization than rails autoscale, which can make it a bit overwhelming, but also potentially more powerful.

web dynos

you can actually configure as many heroku process types as you have for autoscale, not just ones named “web” and “worker”. and for each, you have your choice of several metrics to be used as scaling triggers.

for web, i think queue time (percentile, average) matches what rails autoscale does, configured to percentile, , and is probably the best to use unless you have a reason to use another. (“rails autoscale tracks the th percentile queue time, which for most applications will hover well below the default threshold of ms.“)

here’s what configuration hirefire makes available if you are scaling on “queue time” like rails autoscale, configuration may vary for other metrics.

i think if you fill in the right numbers, you can configure to work equivalently to rails autoscale.

worker dynos

if you have more than one heroku process type for workers — say, working on different queues — hirefire can scale the independently, with entirely separate configuration. this is pretty handy, and i don’t think rails autoscale offers this. (update i may be wrong, rails autoscale says they do support this, so check on it yourself if it matters to you).

for worker dynos, you could choose to scale based on actual “dyno load”, but i think this is probably mostly for types of processes where there isn’t the ability to look at “number of jobs”. a “number of jobs in queue” like rails autoscale does makes a lot more sense to me as an effective metric for scaling queue-based bg workers.

hirefire’s metric is slightly difererent than rails autoscale’s “jobs in queue”. for recognized ruby queue systems (a larger list than rails autoscale’s; and you can write your own custom adapter for whatever you like), it actually measures jobs in queue plus workers currently busy. so queued+in-progress, rather than rails autoscale’s just queued. i actually have a bit of trouble wrapping my head around the implications of this, but basically, it means that hirefire’s “jobs in queue” metric strategy is intended to try to scale all the way to emptying your queue, or reaching your max scale limit, whichever comes first. i think this may make sense and work out at least as well or perhaps better than rails autoscale’s approach?

here’s what configuration hirefire makes available for worker dynos scaling on “job queue” metric.

since the metric isn’t the same as rails autosale, we can’t configure this to work identically. but there are a whole bunch of configuration options, some similar to rails autoscale’s.

the most important thing here is that “ratio” configuration. it may not be obvious, but with the way the hirefire metric works, you are basically meant to configure this to equal the number of workers/threads you have on each dyno. i have it configured to because my heroku worker processes use resque, with resque_pool, configured to run resque workers on each dyno. if you use sidekiq, set ratio to your configured concurrency — or if you are running more than one sidekiq process, processes*concurrency. basically how many jobs your dyno can be concurrently working is what you should normally set for ‘ratio’.

hirefire not a heroku plugin

hirefire isn’t actually a heroku plugin. in addition to that meaning separate invoicing, there can be some other inconveniences.

since hirefire only can interact with heroku api, for some metrics (including the “queue time” metric that is probably optimal for web dyno scaling) you have to configure your app to log regular statistics to heroku’s “logplex” system. this can add a lot of noise to your log, and for heroku logging add-ons that are tired based on number of log lines or bytes, can push you up to higher pricing tiers.

if you use paperclip, i think you should be able to use the log filtering feature to solve this, keep that noise out of your logs and avoid impacting data log transfer limits. however, if you ever have cause to look at heroku’s raw logs, that noise will still be there.

support and docs

i asked a couple questions of both hirefire and rails autoscale as part of my evaluation, and got back well-informed and easy-to-understand answers quickly from both. support for both seems to be great.

i would say the documentation is decent-but-not-exhaustive for both products. hirefire may have slightly more complete documentation.

other features?

there are other things you might want to compare, various kinds of observability (bar chart or graph of dynos or observed metrics) and notification. i don’t have time to get into the details (and didn’t actually spend much time exploring them to evaluate), but they seem to offer roughly similar features.

conclusion

rails autoscale is quite a bit more expensive than hirefire.io’s flat rate, once you get past rails autoscale’s most basic tier (scaling no more than standard dynos).

it’s true that autoscaling saves you money over not, so even an expensive price could be considered a ‘cut’ of that, and possibly for many ecommerce sites even $ a month might a drop in the bucket (!)…. but this price difference is so significant with hirefire (which has flat rate regardless of dynos), that it seems to me it would take a lot of additional features/value to justify.

and it’s not clear that rails autoscale has any feature advantage. in general, hirefire.io seems to have more features and flexibility.

until , hirefire.io could only analyze metrics with -minute resolution, so perhaps that was a “killer feature”?

honestly i wonder if this price difference is sustained by rails autoscale only because most customers aren’t aware of hirefire.io, it not being listed on the heroku marketplace? single-invoice billing is handy, but probably not worth $ + a month. i guess hirefire’s logplex noise is a bit inconvenient?

or is there something else i’m missing? pricing competition is good for the consumer.

and are there any other heroku autoscale solutions, that can handle rails bg job dynos, that i still don’t know about?

update a day after writing djcp on a reddit thread writes:

i used to be a principal engineer for the heroku add-ons program.
one issue with hirefire is they request account level oauth tokens that essentially give them ability to do anything with your apps, where rails autoscaling worked with us to create a partnership and integrate with our “official” add-on apis that limits security concerns and are scoped to the application that’s being scaled.
part of the reason for hirefire working the way it does is historical, but we’ve supported the endpoints they need to scale for “official” partners for years now.
a lot of heroku customers use hirefire so please don’t think i’m spreading fud, but you should be aware you’re giving a third party very broad rights to do things to your apps. they probably won’t, of course, but what if there’s a compromise?
“official” add-on providers are given limited scoped tokens to (mostly) only the actions / endpoints they need, minimizing blast radius if they do get compromised.

you can read some more discussion at that thread.

- - t : : + : jrochkind hugh rundle: automation workflows with github actions and webhooks - library map part https://www.hughrundle.net/automation-workflows-with-github-actions-and-webhooks/

this is the third in my series on the library map. part one dealt with why i made the map. part explained how i made it. this post is about strategies i've used to automate some things to keep it up to date.

what is a github action?

a github action is an automated script that runs on a virtual machine when triggered by some kind of event. triggers for actions are defined in a "workflow" configuration file at .github/workflows in your github repository. the terminology can be a bit confusing, because "github actions" is what github calls the whole system, but an "action" within that system is actually the smallest part in the series:

workflow
- job 
 - step 
 - action 
 - action 
 - step 
 - action 
- job 
 - step 
 - action 
 - action

github actions are really just github's version of a continuous integration / continuous deployment (ci/cd) tool. i say "just", but it's extremely powerful. unfortunately that does mean that even though github actions are quite extensively documented, the docs aren't necessarily all that clear if you're starting from scratch, and the process is quite confusing for the uninitiated. i spent a couple of days failing to make it work the way i wanted, so that you don't have to.

github actions ...in action

there are a zillion things you can use github actions for — auto-closing "stale" issues, adding labels automatically, running code linters on pull requests, and so on. if you've read my previous posts, you might remember that i wrote a little python script to merge the data from library_services_information.csv into boundaries.topo.json. but doing that manually every time the csv file is updated is a tedious manual task. wouldn't it be better if we could automate it? well, we can automate it with github actions!

what we want to do here is set up a trigger that runs the script whenever the csv file is changed. i originally tried doing this on a push event (every time code is pushed to the default branch), and it worked, but ultimately i decided it would be better to run it whenever someone (including me) makes a pull request. i'm in a reasonably consistent habit of always creating a new git branch rather than committing directly to the default branch, and there's less chance of something going wrong and the topojson file being corrupted if the merge is done at the pull request stage and then manually pulled in — if there can't be a clean merge, github will tell me before i break everything.

to set this up, we need to write a workflow configuration file, listing the jobs we want done, and the actions within each job. jobs within each workflow are run concurrently unless the workflow configuration tells them to wait for the previous job, though in our case that doesn't matter, because there is only a single job. the structure is:

workflow ('topo auto updater (pr)')
 - job ('auto-topo-updater')
 - step : git checkout code
 - step : add labels
 - step : merge files
 - step : git commit updated code

the first step uses an action provided by github itself. it runs a git checkout on the repository before anything else happens. this means nothing will happen in the actual repository if anything in the workflow fails, because the virtual machine that checked out your code just gets destroyed without checking the code back in.

step will use an action created by christian vuerings, and automatically adds labels to an issue or pull request, based on whatever criteria triggered the workflow.

step runs the python script to merge the csv data into the topojson.

step (care of stefan zweifel) commits and pushes the updated changes into the pull request that triggered the workflow. this is where the real magic happens, because it simply adds a second commit to the pull request as soon as it is received and before the pr is merged. i initially had set this up to create a second pull request with just the merged topojson changes and then tried to work out how to auto-merge that new pull request, but someone on mastodon helpfully asked me why i would bother creating a pull request if i wanted to auto-merge it anyway. the thought of auto-committing terrified me initially because i had no idea what i was doing, but on reflection a second pr was indeed a bit silly.

writing the config file

to get all this to happen, we need to write a configuration file. this is written in yaml, and saved in a special directory at the top of the repository, called .github/workflows. you can name this file whatever you want, but it has to end in .yml.

first we provide some kind of trigger, and include any conditions we might want to apply. i want this workflow to happen whenever someone creates a pull request that includes changes to the website/data/library_services_information.csv file:

name: topo auto updater (pr)

on:
 pull_request:
 paths:
 - 'website/data/library_services_information.csv'

 workflow_dispatch:

the on directive lists the different 'events' that can trigger the workflow. the first one is clear enough, but what about workflow_dispatch? this event simply means "when triggered manually by pressing a button". i don't know why it has such an obscure name.

once we've told github when we want the workflow to run, we can tell it what we want it to do. first we list our jobs:

jobs:
 auto-topo-updater:
 runs-on: ubuntu-latest
 steps:
			# steps go here

the first line under 'jobs' is the name of our job (this can be anything, but without spaces). runs on tells github which runner to use. a 'runner' is a special environment that runs automated continuous integration tools. in this case we're using github actions runners, but runners are also commonly used in other automated testing tools. here we are using the "latest" ubuntu linux runner, which is currently using ubuntu . even though ubuntu . is actually the latest ubuntu lts release. now we've outlined the trigger and where we want to run our steps, it's time to say what those steps are:


steps:
- uses: actions/checkout@v 
 with:
 ref: ${{ github.head_ref }}

- uses: christianvuerings/add-labels@v 
 with:
 labels: |
 auto update
 data
 env:
 github_token: $

- name: merge csv to topojson
 run: |
 python ./.github/scripts/merge_csv_to_topojson.py

- uses: stefanzweifel/git-auto-commit-action@v 
 with:
 commit_message: merge csv data to topo

whoah, that's a lot! you can see there are two ways we describe how to perform an action: uses, or name + run. the uses directive points to an action that someone has publicly shared on github. so uses: actions/checkout@v means "use version of the action at the repository address https://github.com/actions/checkout". this is an official github action. if we want to simply run some commands, we can just give our action a name and use the run directive:

- name: merge csv to topojson
	run: |
	python ./.github/scripts/merge_csv_to_topojson.py

in this example, we use a pipe (|) to indicate that the next lines should be read one after another in the default shell (basically, a tiny shell script). the first step checked out out our code, so we can now use any script that is in the repository. i moved the python merging script into .github/scripts/ to make it clearer how this script is used, and now we're calling it with the python command.

to pass data to an action, we use with. the step below passes a list of label names to add to the pull request ('auto update' and 'data'):

- uses: christianvuerings/add-labels@v 
 with:
 labels: |
 auto update
 data

finally, for the labels step we need to provide an environment variable. for certain activities, github requires actions to use a github_token so that you can't just run an action against any repository without permission. this is automatically stored in the "secret store", to which you can also add other secrets like api keys and so on. the env directive passes this through to the action:


env:
github_token: ${{ secrets.github_token }}

putting the robots to work

now when a pull request is sent, it gets tagged auto update and data, and a commit updating the topo.json file is automatically added to it:

screenshot of an updated pull request

you can see the full config file in the library map repository.

i've also worked out how to reduce the filesize of my geojson file, so i was able to check it in to the repository. this allowed me to automate the transformation from geojson to topojson whenever the geojson file is updated, with a workflow that runs some commands over the geojson and creates a new pull request. one little gotcha with this is that the action i used to process the geojson file into topojson also cleans up the geojson, which means triggering the action on any change to the geojson file creates a recursive loop whereby every time the new pull request is merged, it creates a new one. to get around this, i probably should just make it auto-commit rather than create a pull request, but for now i added an if statement:

jobs:
 processjson:
 if: "!contains(github.event.head_commit.message, 'from hughrun/geo-to-topo')"
...

 - name: create pull request
 uses: peter-evans/create-pull-request@v 
 with:
 commit-message: update topojson boundaries
 title: update topojson boundaries
 body: 'clean & minify geojson'
 branch: geo-to-topo
 labels: auto update,data

the last action creates a pull request on a new geo-to-topo branch, so if the commit message includes "from hughrun/geo-to-topo" the job won't run. recursive pull request problem solved!

what is a webhook?

i really like cherries, but they're not always season. imagine me sending a text message to the local greengrocer every day in early summer, to ask whether they have any cherries yet. they text me back: usually the answer is "no", but eventually it's a "yes". then i hit on an idea: i call them and ask them to just text me when cherries are in stock.

the first approach is how an api call works: you send a request, and the server sends a response. the second is how a webhook works — you get the response without having to even send the request, when a certain criteria is met. i've been playing around with apis and webhooks at work, because we want to connect eventbrite event information to a calendar on our own website. but github also offers webhooks (which actually pre-dates github actions), and this is the final piece of the library map automation pipeline.

the big difference of course is that sending an http request and receiving an http request are quite different things. you can send an http request in many different ways: including by just typing a url into a browser. but to receive a request you need some kind of server. especially if you don't know when it will be sent. conveniently i already have a vps that i use for a few things, including hosting this blog. so we have something to receive the webhook (a server), and something to send the webhook (github). now we need to tell those two things how to talk to each other.

what we want to do here is automatically update the data on the library map whenever there is an update in the repository. i could make this easier by just publishing the map with github pages, but i don't want to completely rely on github for everything.

sending the webhook

first of all we need to set up the webhook. in the repository we go to settings - webhooks and then click on add webhook. here we enter the payload url (the url we will set up on our server, to receive the webhook: https://example.com/gh-library-map), the content type (application/json), and a secret. the secret is just a password that can be any text string, but i recommend using something long and hard to guess. you could try one of my favourite urls to create it. we want the trigger to be "just the push event" because we don't want to trigger the webhook every time anything at all happens in the repository. unfortunately there doesn't seem to be a way to trigger it only on a push to the primary branch, but in future we could probably put some logic in at the receiving end to filter for that. make sure the webhook is set to "active", and click "add webhook".

screenshot of github webhooks page

receiving the webhook

so setting up the webhook to be sent is reasonably straightforward. receiving it is a bit more complicated. we need to set up a little application to hang around waiting to receive http requests.

first of all, we set up nginx to serve our domain — in this post i'll refer to that as 'example.com'. then we secure it using certbot so github can send the webhook to https://example.com.

because we might want to use other webhooks on other systems for different tasks, we're going to go with a slightly over-powered option and use express. this gives us a bit of control over routing different requests to different functions. express is a nodejs framework for building web apps, so first we need to make sure we have a recent version of nodejs installed. then we create a new package metadata file, and a javascript file:

npm init
touch webhooks.js

in our empty webhooks.js file we set up some basic routing rules with express:

npm install express --save

// webhook.js
const express = require('express')
const port =  

const app = express()
app.use(express.json())

app.post('/gh-library-map', (req, res, next) => {
	// do stuff
})

// everything else should 
app.use(function (req, res) {
 res.status( ).send("there's nothing here")
})

app.listen(port, () => {
 console.log(`webhooks app listening on port ${port}`)
})

this will do something when a post request is received at https://example.com/gh-library-map. all other requests will receive a response. you can test that now.

returning to the delicious cherries: what happens if someone else finds out about my arrangement with the greengrocer? maybe a nefarious strawberry farmer wants to entice me to go to the greengrocer and, upon discovering there are no cherries, buy strawberries instead. they could just send a text message to me saying "hey it's your friendly greengrocer, i totes have cherries in stock". this is the problem with our webhook endpoint as currently set up. anyone could send a post request to https://example.com/gh-library-map and trigger an action. luckily github has thought of that, and has a solution.

remember the "secret" we set when we set up the webhook? this is where we use it. but not directly. github instead creates a sha hash of the entire payload using your secret and includes the resulting hash in the payload itself. the hash is sent in a header called x-hub-signature- . we know what our secret is, and we can therefore check the hash by running the same process over the payload at the receiving end as github did at the sending end. as long as we use a strong secret and the hashes match, we can be confident the request did indeed come from github, and not a nefarious strawberry farmer. the crypto library is included in nodejs automatically, so we can use that check:

// webhook.js
const crypto = require('crypto')

app.post('/gh-library-map', (req, res, next) => {
 const hmac = crypto.createhmac('sha ', process.env.library_map_gh_secret)
 hmac.update(json.stringify(req.body))

 // check has signature header and the decrypted signature matches
 if (req.get('x-hub-signature- ')) {
 if ( `sha =${hmac.digest('hex').tostring()}` === req.get('x-hub-signature- ') ){
 // do something
 } else {
 console.error("signature header received but hash did not match")
 res.status( ).send('signature is missing or does not match')
 }
 } else {
 console.error('signature missing')
 res.status( ).send('signature is missing or does not match')
 }
})

now we just need to "do something" when the hash matches 😆.

push and pull

so what is the something we're going to do? the library map server simply contains a copy of the repository, sitting behind an nginx web proxy server. what we need to do to update it is run git pull inside that directory, and it will pull in the latest updates from the repository. our webhook will end up calling this action more often than is strictly useful, because a "push" action happens every time someone creates a pull request, for example, but it's pretty harmless to git pull more often than necessary.

first we create a new function:

// webhook.js
const util = require('util')
const exec = util.promisify(require('child_process').exec) // run child_process.exec as a promise/async

async function gitpull(local_repo, res) {
 try {
 const { stdout, stderr } = await exec(`cd ${local_repo} && git pull`);
 let msg = stderr ? stderr : stdout // message is the error message if there is one, else the stdout
 // do something with message
 res.status( ).send('ok')
 } catch (err) {
 console.error(err)
 res.status( ).send('server error sorry about that')
 }
}

this function is async because we need to await the git pull before we can do something with the output. to make it "awaitable" we use util.promisify() which is another built-in function in nodejs. we call this function back in our express route, where we said we would "do something":

// webhook.js
const local_repo = "/path/to/website/directory"
if (req.get('x-hub-signature- ')) {
	if ( `sha =${hmac.digest('hex').tostring()}` === req.get('x-hub-signature- ') ){
 gitpull(local_repo, res)
	} else { ...
	}
...
}

sweet! now every time someone does a git push we can do a git pull to add the change to the website! maybe we want to be sure that happened though, so we can add a final piece to this, by sending ourselves an email using emailjs every time the webhook is successfully received:

npm install emailjs

// webhook.js
const { smtpclient } = require('emailjs')
function sendemail(msg, trigger) {

 const client = new smtpclient({
 user: process.env.email_user,
 password: process.env.email_password,
 host: process.env.smtp_domain,
 ssl: true,
 });
 
 // send the message and get a callback with an error or details of the message that was sent
 client.send(
 {
 text: `github webhook for ${trigger} has triggered a "git pull" event with the following result:\n\n${msg}`,
 from: `webhook alerts<${process.env.email_send_address}>`,
 to: process.env.email_receive_address,
 subject: `github triggered a pull for ${trigger}`,
 },
 (err, message) => {
 console.log(err || message);
 }
 );
}

async function gitpull(local_repo, res) {
 try {
 const { stdout, stderr } = await exec(`cd ${local_repo} && git pull`);
 let msg = stderr ? stderr : stdout 
 sendemail(msg, 'mysite.com')
 res.status( ).send('ok')
 } catch (err) {
 console.error(err)
 res.status( ).send('server error sorry about that')
 }
}

we can now test the webhook:

node webhooks.js

express will start up. we can use curl to send some test payloads from a new console session on our local machine:

curl -d '{"key ":"value ", "key ":"value "}' -h "content-type: application/json" -x post https://example.com/gh-library-map

curl -h "x-hub-signature- : blah" -d '{"key ":"value ", "key ":"value "}' -h "content-type: application/json" -x post https://example.com/gh-library-map

both requests should return a with signature is missing or does not match, but in the server console the second one should log a message signature header received but hash did not match.

the last thing we need to do is set up our little express app to run automatically as a background process on the server. we can do this using systemd. i personally find the official documentation rather impenetrable, but there are lots of helpful tutorials online. systemd helps us with two tasks:

keeping the app running
making the environment variables available to the app

first we create a "unit file" called webhooks.service at /etc/systemd/system:

# /etc/systemd/system/webhooks.service
description=keeps the webhooks express server running
after=network.target

[service]
type=simple
execstart=/usr/bin/node webhooks.js
restart=always
restartsec= 
user=username
workingdirectory=/home/username/webhooks
environmentfile=/etc/systemd/system/webhooks.env

[install]
wantedby=multi-user.target

the user is your username, and workingdirectory is wherever you installed your express app. since we're responsible server administrators, we have unattended-upgrades running, so occasionally the server will reboot itself to finish installing security updates. we can ensure the webhooks service always comes back up by setting restart to always.

next we create the environmentfile mentioned in the unit file:

# /etc/systemd/system/webhooks.env
library_map_gh_secret="your github secret here"
email_user="user@mail.example.com"
email_password="top secret password"
smtp_domain="smtp.example.com"
email_send_address="webhooks@mail.example.com"

this is where all those process.env values come from in the webhooks.js file. we could hardcode them, but you might want to share your file in a blog post one day, and you definitely don't want to accidentally leave your hardcoded github secret in the example!

make sure we've stopped the app, so we don't have two conflicting installations, then run:

sudo systemctl enable webhooks.service
sudo systemctl start webhooks.service

our webhooks service should now be running. go back to the github webhooks page in your repository settings and you should see an option to send a "ping event". this simply checks that your webhook is working by sending a test payload. send the ping, wait a few moments, and we should see an email appear in the email_send_address inbox:

screenshot of email from webhook service

what's next?

that was a pretty long and technical post, sorry not sorry. now that i've set up all that automation, it would be great for library people to help correct and complete the data. as for me, i'll be looking for other things i can do with automation. maybe automatically tooting release notes for ephemetoot. we'll see.

- - t : : + : hugh rundle mita williams: weeknote ( ) https://librarian.aedileworks.com/ / / /weeknote- - /

i don’t have much that i can report in this week’s note. you are just going to have to take my word that this week, a large amount of my time was spent at meetings pertaining to my library department, my union, and anti-black racism work.

last year, around this same time, some colleagues from the university and i organized an speaking event called safer communities in a ‘smart tech’ world:

we need to talk about amazon ring in windsor.

windsor’s mayor proposes we be the first city in canada to buy into the ring network.
as residents of windsor, we have concerns with this potential project. seeing no venue for residents of windsor to share their fears of surveillance and loss of privacy through this private-partnership, we hosted an evening of talks on january nd, at the performance hall at the university of windsor’s school of creative arts windsor armories building. our keynote speaker was chris gilliard, heard recently on cbc’s spark.
since that evening, we have been in the media raising our concerns, asking questions, and encouraging others to do the same.

the city of windsor has yet to have entered an agreement with amazon ring. this is good news.

this week, the city of windsor announced that it has entered a one-year deal partnership with ford mobility canada to share data and insights via ford’s safety insights platform.

i don’t think this is good news for reasons outlined in this post called safety insights, data privacy, and spatial justice.

this week i learned a neat tweetdeck hack. if set up a search as column, you can limit the results for that term using the number of ‘engagements’:

i haven’t read this but i have it bookmarked for potential future reference: the weaponization of web archives: data craft and covid- publics:

an unprecedented volume of harmful health misinformation linked to the coronavirus pandemic has led to the appearance of misinformation tactics that leverage web archives in order to evade content moderation on social media platforms. here we present newly identified manipulation techniques designed to maximize the value, longevity, and spread of harmful and non-factual content across social media using provenance information from web archives and social media analytics. after identifying conspiracy content that has been archived by human actors with the wayback machine, we report on user patterns of “screensampling,” where images of archived misinformation are spread via social platforms. we argue that archived web resources from the internet archive’s wayback machine and subsequent screenshots contribute to the covid- “misinfodemic” in platforms. understanding these manipulation tactics that use sources from web archives reveals something vexing about information practices during pandemics—the desire to access reliable information even after it has been moderated and fact-checked, for some individuals, will give health misinformation and conspiracy theories more traction because it has been labeled as specious content by platforms.

i’m going to leave this tweet here because i might pick up this thread in the future:

this reminds me of a talk given in by data & society founder and president, danah boyd called you think you want media literacy… do you?

this essay still haunts me, largely because we still don’t have good answers for the questions that dr. boyd asks of us and the stakes have only gotten higher.

- - t : : + : mita williams david rosenthal: effort balancing and rate limits https://blog.dshr.org/ / /effort-balancing-and-rate-limits.html catalin cimpanu reports on yet another crime wave using bitcoin in as bitcoin price surges, ddos extortion gangs return in force:

in a security alert sent to its customers and shared with zdnet this week, radware said that during the last week of and the first week of , its customers received a new wave of ddos extortion emails.

extortionists threatened companies with crippling ddos attacks unless they got paid between and bitcoins ($ , to $ , )
...
the security firm believes that the rise in the bitcoin-to-usd price has led to some groups returning to or re-prioritizing ddos extortion schemes.

and dan goodin reports on the latest technique the ddos-ers are using in ddosers are abusing microsoft rdp to make attacks more powerful:

as is typical with many authenticated systems, rdp responds to login requests with a much longer sequence of bits that establish a connection between the two parties. so-called booter/stresser services, which for a fee will bombard internet addresses with enough data to take them offline, have recently embraced rdp as a means to amplify their attacks, security firm netscout said.

the amplification allows attackers with only modest resources to strengthen the size of the data they direct at targets. the technique works by bouncing a relatively small amount of data at the amplifying service, which in turn reflects a much larger amount of data at the final target. with an amplification factor of . to , gigabytes-per-second of requests directed at an rdp server will deliver roughly gbps to the target.

i don't know why it took me so long to figure it out, but reading goodin's post i suddenly realized that techniques we described in impeding attrition attacks in p p systems, a follow-up to our award-winning sosp paper on the architecture of the lockss system, can be applied to preventing systems from being abused by ddos-ers. below the fold, brief details.

among the lockss system's defenses against abuse are two relevant to ddos prevention, rate limits and effort balancing.
rate limits i've written before about the importance of rate limits, quoting paul vixie:

every reflection-friendly protocol mentioned in this article is going to have to learn rate limiting. this includes the initial tcp three-way handshake, icmp, and every udp-based protocol. in rare instances it's possible to limit one's participation in ddos reflection and/or amplification with a firewall, but most firewalls are either stateless themselves, or their statefulness is so weak that it can be attacked separately. the more common case will be like dns [response rate limiting], where deep knowledge of the protocol is necessary for a correctly engineered rate-limiting solution applicable to the protocol.

the rdp server being used to ddos sees a flood of authentication requests whose source address has been spoofed to be the target of the ddos. this isn't what they'd see from a real user, so the rdp server should rate-limit sending authentication responses to a client to a reasonable rate for a real client. this would be helpful, but it isn't enough. because the ddos-ers use a large number of systems to mount an attack, even a fairly low rate of reponses can be harmful.
effort balancing in our paper we wrote:

effort balancing. if the effort needed by a requester to procure a service from a supplier is less than the effort needed by the supplier to furnish the requested service, then the system can be vulnerable to an attrition attack that consists simply of large numbers of ostensibly valid service requests. we can use provable effort mechanisms such as memory-bound functions to inflate the cost of relatively “cheap” protocol operations by an adjustable amount of provably performed but otherwise useless effort. by requiring that at each stage of a multi-step protocol exchange the requester has invested more effort in the exchange than the supplier, we raise the cost of an attrition strategy that defects part-way through the exchange. this effort balancing is applicable not only to consumed resources such as computations performed, memory bandwidth used or storage occupied, but also to resource commitments. for example, if an adversary peer issues a cheap request for service and then defects, he can cause the supplier to commit resources that are not actually used and are only released after a timeout (e.g., syn floods). the size of the provable effort required in a resource reservation request should reflect the amount of effort that could be performed by the supplier with the resources reserved for the request.

vixie also noted the economic requirement:

engineering economics requires that the cost in cpu, memory bandwidth, and memory storage of any new state added for rate limiting be insignificant compared with an attacker's effort.

the reason rdp can be used to amplify a ddos attack is that, as goodin wrote:

rdp responds to login requests with a much longer sequence of bits that establish a connection between the two parties.

the obvious application of effort balancing would be to require that rdp's login requests be padded with additional bytes to make them longer than the login reponse. thus the rdp server would act to attenuate the attack, not amplify it. this would satisfy vixie's goal:

attenuation also has to be a first-order goal—we must make it more attractive for attackers to send their packets directly to their victims than to bounce them off a ddos attenuator.

the protocol could specify that the padding bytes not be random, but be computed from the login request parameters by some algorithm making them relatively expensive to generate but cheap to verify (cf. proof-of-work). this would not significantly impact legitimate clients, who issue login requests infrequently, but would increase the cost of using the rdp server to disguise the source of the attack.

- - t : : + : david. (noreply@blogger.com) open knowledge foundation: launching the net zero challenge: a global pitch competition about using open data for climate action https://blog.okfn.org/ / / /launching-the-net-zero-challenge-a-global-pitch-competition-about-using-open-data-for-climate-action/

open knowledge foundation is excited to launch the net zero challenge, a global pitch competition about using open data for climate action.

with a new administration in the usa and the cop meeting in the uk, will be a crucial year for the global climate response.

let’s see how open data can play its part.

tell us how your idea or project uses open data for climate action – and you could win a $ , usd in the first round of the net zero challenge.

full details about the net zero challenge are available at netzerochallenge.info.

this project is funded by our partners at microsoft and the uk foreign, commonwealth & development office. we are extremely grateful for their support.

how are you advancing climate action using open data?

to be eligible for the net zero challenge, your idea or project must do one or more of the following:

understand climate risks
track climate progress
enable informed climate action, or
evaluate climate impact.

some ways in which you might do this include:

making climate relevant data easier to discover, view and understand by the general data user
creating a useful passthrough tool or api for climate-relevant data in any country or jurisdiction
organising climate data so that potential data users (including those who are less data-literate) can see what’s available, and make use of it

we are very open minded about your approach and methodology. what we care about is the outcome, and whether you answer the question.

you might consider whether your idea or project is:

technically achievable
easy to use
easily integrated or can be provided as a tool
scalable
good value for money
published under an open licence which allows free use by others
explainable (this is the key test of the challenge. can you pitch your project in three minutes to a general audience?)

how do i apply?

apply now by filling out this form. all applications must be received by pm pacific standard time on friday th march . late submissions will not be accepted.

applications will be reviewed and a short list invited to pitch their idea to a panel of experts at a virtual pitch contest.

pitches will take the form of a public three-minute presentation via video conference, followed by a question and answer session with our panel of climate data experts.

pitches can be live, or prerecorded but the q&a will be live.

expert guidance for the net zero challenge is provided by our advisory committee: the open data charter, the innovation and open data team at transport for new south wales and the open data day team at open knowledge foundation.

need more information?

if you have any questions about the net zero challenge, please check out the faqs on the netzerochallenge.info website. to contact the net zero challenge team directly, email netzero@okfn.org.

- - t : : + : james hamilton peter sefton: research data management looking outward from it http://ptsefton.com/ / / /rdm aero/index.html

this is a presentation that i gave on wednesday the nd of december at the aero (australian eresearch organizations) council meeting at the request of the chair dr carina kemp).

carina asked:

it would be really interesting to find out what is happening in the research data management space. and i’m not sure if it is too early, but maybe touch on what is happening in the eosc science mesh project.

the audience of the aero council is aero member reps from aaf, aarnet, qcif, caudit, csiro, ga, tpac, the uni of auckland, reannz, adsei, curtin, unsw, apo.

at this stage i was still the eresearch support manager at uts - but i only had a couple of weeks left in that role.

research data management looking outward from it research data management looking outward from it

in this presentation i’m going to start from a naive it perspective about research data.

i would like to acknowledge and pay my respects to the gundungurra and darug people who are the traditional custodians of the land on which i live and work.

research data is special - like snowflakes - and i don’t mean that in a mean way, research data could be anything - any shape any size and researchers are also special, not always % aligned with institutional priorities, they align with their disciplines and departments and research teams.

$data_management != $storage $data_management != $storage

it’s obvious that buying storage doesn’t mean you’re doing data management well but that doesn’t mean it’s not worth restating.

so "data storage is not data management". in fact, the opposite might be true - think about buying a laptop - do you just get one that fits all your stuff and rely on getting a bigger one every few years? or do you get a smaller main drive and learn how to make sure that your data's actually archived somewhere? that would be managing data.

and remember that not all research data is the same “shape” as corporate data - it does not all come in database or tabular form - it can be images, video, text, with all kinds of structures.

💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾👩🏽‍🔬💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾 <p>

there are several reasons we don’t want to just dole-out storage as needed.

it’s going to cost a lot of money and keep costing a lot of money

not everything is stored in “central storage” anyway. there are share-sync services like aarnet’s cloudstor.

emoji of lots of floppy disks with a researcher in the centre

just keeping things doesn’t mean we can find them again

so far we’ve just looked at things from an infrastructure perspective but that’s not actually why we’re here, us with jobs in eresearch. i think we’re here to help researchers do excellent research with integrity, and we need to help our institutions and researchers manage risk.

the australian code for the responsible conduct of research which all research organizations need to adhere to if we get arc or nhmrc grants sets out some institutional responsibilities to provide infrastructure and training
there are risks associated with research data, reputational, financial and risks to individuals and communities about whom we hold data

at uts, we’ve embraced the research data management plan - as a way to assist in dealing with this risk. rdmps have a mixed reputation here in australia - some organizations have decided to keep them minimal and as streamlined as possible but at uts the thinking is that they can be useful in addressing a lot of the issues raised so far.

where’s the data for project x - when there’s an integrity investigation. were procedures followed?
how much storage are we going to need?

<p>image showing a research data management system that can provision research services (workspaces).</p> <p>

inspired by the (defunct?) research data lifecycle project that was conceived by the former organizations that became the australian research data commons (ands, nectar and rdsi) we came up with this architecture for a central research data management system (in our case we use the open source redbox system) loosely linked to a variety of research workspaces, as we call them.

the plan is that over time, researchers can plan and budget for data management in the short, medium and long term, provision services and use the system to archive data as they go.

(diagram by gerard barthelot at uts)

<p>screenshot of the ocfl home page.</p> <p>

uts has been an early adopter of the ocfl (oxford common file layout) specificiation - a way of storing file sustainably on a file system (coming soon: s cloud storage) so it does not need to be migrated. i presented on this at the open repositories conference

<p>screenshot of the ro-crate home page.</p> <p>

and at the same conference, i introduced the ro-crate standards effort, which is a marriage between the datacrate data packaging work we’ve been doing at uts for a few years, and the research object project.

<p>screenshot of the arkisto page.</p> <p>

we created the arkisto platform to bring together all the work we’ve been doing to standardise research data metadata, and to build a toolkit for sustainable data repositories at all scales from single-collection up to institutional, and potentially discipline and national collections.

<p>screenshot from the arkisto use-cases page showing an idealized data-flow for sensor data using the arkisto platform.</p> <p>

this is an example of one of many arkisto deployment patterns you can read more on the arkisto use cases page

<p>screenshot of a search interface to a historical dataset.</p> <p>

this is an example of an arkisto-platform output. data exported from one content management system into an archive-ready ro-crate package, which can then be made into a live site. this was created for ass prof tamson pietsch at uts. the website is ephemeral - the data will be interoperable and reusable (i and r from fair) via the use of ro-crate.

<p>image of chickens roosting on the roof of their house rather than inside it.</p> <p>

now to higher-level concerns: i built this infrastructure for my chooks (chickens) - they have a nice dry box with a roosting loft. but most of the time they roost on the roof.

we know all too well that researchers don’t always use the infrastructure we build for them - you have to get a few other things right as well.

… and have the right policy environment … <p>

… they will come, given the right training, incentives, support etc. <p>

one of the big frustrations i have had as an eresearch manager is that the expectations and aspirations of funders and integrity managers and so on are well ahead of our capacity to deliver the services they want, and then when we do get infrastructure sorted there are organizational challenges to getting people to use it. to go back to my metaphor, we can’t just pick up the researchers from the roof and put them in their loft, or spray water on them to get them to move.

via gavin kennedy and guido aben from aarnet marco la rosa and i are helping out with this charmingly named project which is adding data management service to storage, syncronization and sharing services. contracts not yet in place so won't say much about this yet.

<p>schreenshot of cs mesh eosc homepage.</p> <p>

https://www.cs mesh eosc.eu/about eosc is the european open science cloud

cs mesh eosc - interactive and agile sharing mesh of storage, data and applications for eosc - aims to create an interoperable federation of data and higher-level services to enable friction-free collaboration between european researchers. cs mesh eosc will connect locally and individually provided services, and scale them up at the european level and beyond, with the promise of reaching critical mass and brand recognition among european scientists that are not usually engaged with specialist einfrastructures.

i told carina i would look outwards as well. what are we keeping an eye on?

watch out for the book factory. sorry, the publishing industry.

the publishing industry is going to “help” the sector look after it’s research data.

like, you, know, they did with the copyright in publications. not only did that industry work out how to take over copyright in research works, they successfully moved from selling us hard-copy resources that we could keep in our own libraries to charging an annual rent on the literature - getting to the point where they can argue that they are essential to maintaining the scholarly record and must be involved in the publishing process even when the (sometimes dubious, patchy) quality checks are performed by us who created the literature.

it’s up to research institutions whether this story repeats with research data - remember who you’re dealing with when you sign those contracts!

some questions i will explore post uts how many research institutions in australia have secure, scalable, sustainable research data repositories that support fair practices? when are we going to have national repositories? how many of the eresearch platforms funded under various programs have sustainable data management stories? how much data is locked-up behind apis and will be inaccessible if the software stops running? <p>

in the s the australian national data (ands) service funded investment in metadata stores; one of these was the redbox research data management platform which is alive and well and being sustained by qcif with a subscription maintenance service. but ands didn’t fund development of research data repositories.

credits the uts eresaerch team michael lynch, sharyn wise, fiona tweedie, moises sacal, marco la rosa, pascal tampubolon, anelm motha, michael lake, matthew gaston ,simon kruik, weisi chen from intersect austrlaia and our rdm project sponsor louise wheeler credits <p>

the work i’ve talked about here was all done with with the uts team.

- - t : : + : ptsefton peter sefton: research data management looking outward from it http://ptsefton.com/ / / /rdm aero/index.html

this is a presentation that i gave on wednesday the nd of december at the aero (australian eresearch organizations) council meeting at the request of the chair dr carina kemp).

carina asked:

it would be really interesting to find out what is happening in the research data management space. and i’m not sure if it is too early, but maybe touch on what is happening in the eosc science mesh project.

the audience of the aero council is aero member reps from aaf, aarnet, qcif, caudit, csiro, ga, tpac, the uni of auckland, reannz, adsei, curtin, unsw, apo.

at this stage i was still the eresearch support manager at uts - but i only had a couple of weeks left in that role.

in this presentation i’m going to start from a naive it perspective about research data.

i would like to acknowledge and pay my respects to the gundungurra and darug people who are the traditional custodians of the land on which i live and work.

it’s obvious that buying storage doesn’t mean you’re doing data management well but that doesn’t mean it’s not worth restating.

and remember that not all research data is the same “shape” as corporate data - it does not all come in database or tabular form - it can be images, video, text, with all kinds of structures.

💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾👩🏽‍🔬💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾💾 <p>

there are several reasons we don’t want to just dole-out storage as needed.

it’s going to cost a lot of money and keep costing a lot of money

not everything is stored in “central storage” anyway. there are share-sync services like aarnet’s cloudstor.

just keeping things doesn’t mean we can find them again

the australian code for the responsible conduct of research which all research organizations need to adhere to if we get arc or nhmrc grants sets out some institutional responsibilities to provide infrastructure and training
there are risks associated with research data, reputational, financial and risks to individuals and communities about whom we hold data

where’s the data for project x - when there’s an integrity investigation. were procedures followed?
how much storage are we going to need?

the plan is that over time, researchers can plan and budget for data management in the short, medium and long term, provision services and use the system to archive data as they go.

(diagram by gerard barthelot at uts)

this is an example of one of many arkisto deployment patterns you can read more on the arkisto use cases page

now to higher-level concerns: i built this infrastructure for my chooks (chickens) - they have a nice dry box with a roosting loft. but most of the time they roost on the roof.

we know all too well that researchers don’t always use the infrastructure we build for them - you have to get a few other things right as well.

https://www.cs mesh eosc.eu/about eosc is the european open science cloud

cs mesh eosc - interactive and agile sharing mesh of storage, data and applications for eosc - aims to create an interoperable federation of data and higher-level services to enable friction-free collaboration between european researchers. cs mesh eosc will connect locally and individually provided services, and scale them up at the european level and beyond, with the promise of reaching critical mass and brand recognition among european scientists that are not usually engaged with specialist einfrastructures.

i told carina i would look outwards as well. what are we keeping an eye on?

watch out for the book factory. sorry, the publishing industry.

the publishing industry is going to “help” the sector look after it’s research data.

it’s up to research institutions whether this story repeats with research data - remember who you’re dealing with when you sign those contracts!

the work i’ve talked about here was all done with with the uts team.

- - t : : + : ptsefton hugh rundle: library map part - how https://www.hughrundle.net/library-map-part- /

this is the second in a series of posts about my new library map. you probably should read the first post if you're interested in why i made the map and why it maps the particular things that it does. i expected this to be a two part series but it looks like i might make a third post about automation. the first post was about why i made the map. this one is about how.

the tech stack

the map is built with a stack of (roughly in order):

original shape (shp) and geojson files
qgis
geojson
a bunch of csv files
a tiny python script
topojson
some html, css and javascript
leafletjs and leaflet plugins
map box tile service

boundary files

since i primarily wanted to map things about library services rather than individual library buildings, the first thing i looked for was geodata boundary files. in australia public libraries are usually run by local government, so the best place to start was with local government boundaries.

this is reasonably straightforward to get - either directly from data.gov.au or one of the state equivalents, or more typically by starting there and eventually getting to the website of the state department that deals with geodata. usually the relevant file is provided as shapefile, which is not exactly what we need, but is a vector format, which is a good start. i gradually added each state and data about it before moving on to the next one, but the process would basically have been the same even if i'd had all of the relevant files at the same time. there are two slight oddities at this point that may (or may not 😂) be of interest.

australian geography interlude

the first is that more or less alone of all jurisdictions, queensland provides local government (lga) boundaries for coastal municipalities with large blocks covering the coastal waters and any islands. other states draw boundaries around outlying islands and include the island — as an island — with the lga that it is part of (if it's not "unincorporated", which is often the case in victoria for example). as a result, the national map looks a bit odd when you get to queensland, because the overlay bulges out slightly away from the coast. i'm not sure whether this is something to do with the lga jurisdictions in queensland, perhaps due to the great barrier reef, or whether their cartography team just couldn't be bothered drawing lines around every little island.

secondly, when i got to western australia i discovered two things:

the cocos (keeling) islands are an overseas territory of australia; and
cocos and christmas islands have some kind of jurisdictional relationship with western australia, and are included in the western australia lga files.

i hadn't really considered including overseas territories, but since they were right there in the file, i figured i may as well. later this led to a question about why norfolk island was missing, so i hunted around and found a shapefile for overseas territories, which also included cocos and christmas islands.

shapefiles are a pretty standard format, but i wanted to use leafletjs, and for that we need the data to be in json format. i also needed to both stitch together all the different state lga files, and merge boundaries where local councils have formed regional library services. this seems to be more common in victoria (which has regional library corporations) than other states, but it was required in victoria, new south wales, and western australia. lastly, it turns out there are significant parts of australia that are not actually covered by any local government at all. some of these areas are the confusingly named national parks that are actually governed directly by states. others are simply 'unincorporated' — the two largest areas being the unincorporated far west region of new south wales (slightly larger than hungary), and the pastoral unincorporated area that consists of almost % of the landmass of south australia (slightly smaller than france).

i had no idea these two enormous areas of australia had this special status. there's also a pretty large section of the south of the northern territory that contains no libraries at all, and hence has no library service. if you're wondering why there is a large section of inland australia with no overlays on the library map, now you know.

qgis and geojson

so, anyway, i had to munge all these files — mostly shape but also geojson — and turn them into a single geojson file. i've subsequently discovered mapshaper which i might have used for this, but i didn't know about it at the time, so i used qgis. i find the number of possibilities presented by qgis quite overwhelming, but there's no doubt it's a powerful tool for manipulating gis data. i added each shapefile as a layer, merged local government areas that needed to be merged, either deleted or dissolved (into the surrounding area) the unincorporated areas, and then merged the layers. finally, i exported the new merged layer as geojson, which is exactly what it sounds like: ordinary json, for geodata.

csv data

at this point i had boundaries, but not other data. i mean, this is not actually true, because i needed information about library services in order to know which lgas collectively operate a single library service, but in terms of the files, all i had was a polygon and a name for each area. i also had a bunch of location data for the actual library branches in a variety of formats originally, but ultimately in comma separate values (csv) format. i also had a csv file for information about each library service. the question at this point was how to associate the information i was mapping with each area. there was no way i was going to manually update + rows in qgis. luckily, csv and json are two of the most common open file formats, and they're basically just text.

python script

i'd had a similar problem in a previous, abandoned mapping project, and had a pretty scrappy python script lying around. with a bit more python experience behind me, i was able to make it more flexible and simpler. if we match on the name of the library service, it's fairly straightforward to add properties to each geojson feature (the features being each library service boundaries area, and the properties being metadata about that feature). this is so because the value of properties within each feature is itself simply a json object:

{"type": "featurecollection",
"name": "library_services",
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:epsg:: " } },
"features": 
[{ "type": "feature", "properties" : {"name": "bulloo shire"}
 "geometry": { "type": "multipolygon", "coordinates": [ [ [ [ . ,- . ],[ . ,- . ] ... ]]}

the python script uses python's inbuilt json and csv modules to read both the geojson and the csv file, then basically merge the data. i won't re-publish the whole thing, but the guts of it is:

# for each geojson feature, if a field in the json matches a field in the csv, add new properties to the json
for feature in json_data['features']:
 with open(csv_file, newline='') as f:
 # use dictreader so we can use the header names
 reader = csv.dictreader(f)
 for row in reader:
 # look for match
 if row[csv_match] == feature['properties'][geojson_match]:
 # create new properties in geojson
 for k in row:
 feature['properties'][k] = row[k]

the whole thing is fewer than lines long. this saved me heaps of time, but as you'll discover in my future post on automation, i later worked out how to automate the whole process every time the csv file is updated!

topojson

geojson is pretty cool — it's specifically designed for web applications to read and write gis files in a native web format. unfortunately, geojson can also get very big, especially with a project like mine where there are lots of boundaries over a large area. the final file was about mb — far too big for anyone to reasonably wait for it to load in their browser (and chrome just refused to load it altogether). because of the way i originally wrote the python script, it actually became nearly three times the size, because i put in a two-space indent out of habit. this created literally hundreds of megabytes of empty spaces. "pretty printing" json is helpful if a human needs to read it, but rather unhelpful if you want to keep the file size down.

enter topojson. to be honest i don't really understand the mathematics behind it, but topojson allows you to represent the same information as geojson but in a much, much smaller file. i reduced a mb geojson file (admittedly, about mb being blank spaces) to . mb simply by converting it to topojson! by "quantising" it (essentially, making it less accurate), the file size can be reduced even further, rendering the current file of about . mb - definitely small enough to load in a browser without too much of a wait, albeit not lightning fast.

good old html/css/javascript

at this point we're ready to start putting together the website to display the map. for this i used plain, vanilla html, css, and javascript. the web is awash with projects, frameworks and blog posts explaining how to use them to create your spa (single page app)™️, but we really don't need any of that. the leaflet docs have a pretty good example of a minimal project, and my map is really not much more complex than that.

something that did stump me for a while was how to bring the topojson and csv files into the javascript file as variables. i'm a self-taught javascript coder, and i learned it back to front: initially as a backend scripting language (i.e. nodejs) and then as the front-end browser scripting language it was originally made to be. so sometimes something a front-end developer would consider pretty basic: "how do i import a text file into my javascript and assign it to a variable?" takes me a while to work out. initially i just opened the files in a text editor and copy-pasted the contents between two quote marks, made it the value of a javascript variable, and saved the whole thing as a .js file. but it was obvious even to me that couldn't possibly be the correct way to do it, even though it worked. in nodejs i would use fs.readfile() but the only thing that looked vaguely similar for front end javascript was filereader — which is for reading files on a client, not a server. finally i did a bit of research and found that the answer is to forget that the file is sitting right there in the same directory as all your javascript and html files, and just use ajax like it's a remote file. the modern way to do this is with fetch, so instead of doing this:

// index.html
<script src="./boundaries.js" type="text/javascript"></script>
<script src="./branchescsv.js" type="text/javascript"></script>
<script src="./ikccsv.js" type="text/javascript"></script>
<script src="./mechanics.js" type="text/javascript"></script>
<script src="./nslabranches.js" type="text/javascript"></script>
<script src="./load-map.js" type="text/javascript"></script>

// boundaries.js
const boundaries = `{"contents": "gigantic json string"}`
// branchescsv.js
const branchescsv = `lat,lng,town,address,phone
- . , . ,victor harbor public library service, bay road, 
... etc`
// ikccsv.js
const ikccsv = `lat,lng,town,address,phone
- . , . ,badu island indigenous knowledge centre,nona street , 
...etc`
// mechanics.js
const mechanics = `lat,lng,town,address,phone
- . , . ,ballaarat mechanics institute, sturt street, 
..etc`
// nslabranches.js
const nslabranches = `lat,lng,town,address,phone
- . , . ,state library of victoria," swanston street, melbourne", 
... etc`

// load-map.js
	// boundaries and the other constants are now globals
	const loanperiod = new l.topojson(boundaries, options)

...we do this:

// index.html
<script src="./load-map.js" type="text/javascript"></script>

// load-map.js
const boundaries = fetch('data/boundaries.topo.json')
.then( response => response.json())

const branchescsv = fetch('data/public_library_locations.csv')
.then( response => response.text());

const ikccsv = fetch('data/indigenous_knowledge_centre_locations.csv')
.then( response => response.text());

const mechanics = fetch('data/mechanics_institute_locations.csv')
.then( response => response.text());

const nslabranches = fetch('data/nsla_library_locations.csv')
.then( response => response.text());

// fetch returns a promise so we have to let them all 'settle' before we can use the returned value
promise.all([boundaries, branchescsv, ikccsv, mechanics, nslabranches])
.then( data => {
	// data is an array with the settled values of the fetch() promises
	const loanperiod = new l.topojson(data[ ], options)
}

in the code this doesn't necessarily look much simpler, but in terms of workflow it's a huge improvement that cuts out manually copy-pasting every time a csv or topojson file is updated, and reduces duplication and the total number of files.

so now the site consists of:

the original data as csv and topojson files
an index.html file to display the map
a single css file for basic styling
a single javascript file to load the map

leaflet and friends

finally it's time to actually put all of this stuff into a map using leaflet. this is a really great javascript library, with pretty good documentation. leaflet allows us to plot shapes onto a map, and using javascript to make them interactive - including adding popups, zoom to features when they're clicked, and add interactive overlays.

i won't try to replicate the leaflet docs here and explain the exact steps to making my map, but i do want to highlight how two leaflet plugins really helped with making the map work nicely. leaflet has a fairly strong plugin collection, and they allow the base library to be fairly lightweight whilst the entire system is still quite flexible and fully featured.

i knew from the beginning it would require the whole library community to keep the map up to date over time. there are hundreds of library services across australia, and they don't set their rules or their procurement decisions in stone forever. so it needed to be relatively simple to update the data as it changes. as we've discussed, geojson also takes up a lot of space. ideally, i could store as much data in csv files as possible, and use them directly as data feeding the map. turns out there's a plugin for that - leaflet.geocsv. this allows us to load csv files directly (for library building locations), and it's converted to geojson on the fly. since csv files are much smaller than the equivalent data in json, this is not only easier to maintain, but also loads faster.

the second plugin that really helped was leaflet.pattern. the problem this helped me to solve was how to show both the fines layer and the loan period layer at the same time. typically for a chloropleth map, different colours or shades indicate certain values. but if you add a second overlay on top of the first one, the colours no longer necessarily make much sense and combinations can be difficult or impossible to discern. thinking about this, i figured if i could make one layer semi-transparent colours, and the second layer patterns like differently angled stripes or dots, that might do the trick. leaflet.pattern to the rescue! after some alpha-testing by my go-to volunteer quality assurance tester, i worked out how to make the layers always appear in the same order, regardless of which order they were added or removed, making the combination always look consistent:

animated gif showing overlays

tile service

once all of that's complete, we can load the map. but there's a problem: all we have is a bunch of vector points and lines, there's no underlying geography. for this we need a map tile service. we can use one of several options provided by openstreetmap, but i ended up using the commercial map box service on a free plan (or at least, it will be free as long as thousands of people don't suddenly start using the map all at the same time). their dark and light map styles really suited what i was trying to do, with minimal detail in terms of the underlying geography, but with roads and towns marked at the appropriate zoom level.

so that's it! it took a while to work it all out, but most of the complexity is in getting the data together rather than displaying the map. once i had that done (though there is still a fair bit of information missing), i was able to pay more attention to maintaining the map into the future. that led me to look into some options for automating the merging of data from the library services csv file (when it's updated) into the topojson file, and also automatically refreshing the data on the actual map when the github repository is updated. in my next post i'll explain how that works. while you're waiting for that, you can help me find missing data and make the map more accurate 😀.

- - t : : + : hugh rundle jonathan rochkind: managed solr saas options https://bibwild.wordpress.com/ / / /managed-solr-saas-options/

i was recently looking for managed solr “software-as-a-service” (saas) options, and had trouble figuring out what was out there. so i figured i’d share what i learned. even though my knowledge here is far from exhaustive, and i have only looked seriously at one of the ones i found.

the only managed solr options i found were: websolr; searchstax; and opensolr.

of these, i think websolr and searchstax are more well-known, i couldn’t find anyone with experience with opensolr, which perhaps is newer.

of them all, searchstax is the only one i actually took for a test drive, so will have the most to say about.

why we were looking

we run a fairly small-scale app, whose infrastructure is currently self-managed aws ec instances, running respectively: ) a rails web app ) bg workers for the rails web app ) postgres, and ) solr.

oh yeah, there’s also a redis running one of those servers, on # with pg or # with solr, i forget.

currently we manage this all ourselves, right on the ec . but we’re looking to move as much as we can into “managed” servers. perhaps we’ll move to heroku. perhaps we’ll use hatchbox. or if we do stay on aws resources we manage directly, we’d look at things like using an aws rds postgres instead of installing it on an ec ourselves, an aws elasticache for redis, maybe look into elastic beanstalk, etc.

but no matter what we do, we need a solr, and we’d like to get it managed. hatchbox has no special solr support, aws doesn’t have a solr service, heroku does have a solr add-on but you can also use any solr with it and we’ll get to that later.

our current solr use is pretty small scale. we don’t run “solrcloud mode“, just legacy ordinary solr. we only have around , documents in there (tiny for solr), our index size is only mb. our traffic is pretty low — when i tried to figure out how low, it doesn’t seem we have sufficient logging turned on to answer that specifically but using proxy metrics to guess i’d say k- k requests a day, query as well as add.

this is a pretty small solr installation, although it is used centrally for the primary functions of the (fairly low-traffic) app. it currently runs on an ec t a.small, which is a “burstable” ec type with only g of ram. it does have two vcpus (that is one core with ‘hyperthreading’). the t a.small ec instance only costs $ /month on-demand price! we know we’ll be paying more for managed solr, but we want to do get out of the business of managing servers — we no longer really have the staff for it.

websolr (didn’t actually try out)

websolr is the only managed solr currently listed as a heroku add-on. it is also available as a managed solr independent of heroku.

the pricing in the heroku plans vs the independent plans seems about the same. as a heroku add-on there is a $ “staging” plan that doesn’t exist in the independent plans. (unlike some other heroku add-ons, no time-limited free plan is available for websolr). but once we go up from there, the plans seem to line up.

starting at: $ /month for:

million document limit
k requests/day
index
mb storage
concurrent requests limit (this limit is not mentioned on the independent pricing page?)

next level up is $ /month for:

million document limit
k requests/day
. gb storage
concurrent request limit (again concurrent request limits aren’t mentioned on independent pricing page)

as you can see, websolr has their plans metered by usage.

$ /month is around the price range we were hoping for (we’ll need two, one for staging one for production). our small solr is well under million documents and ~ gb storage, and we do only use one index at present. however, the k requests/day limit i’m not sure about, even if we fit under it, we might be pushing up against it.

and the “concurrent request” limit simply isn’t one i’m even used to thinking about. on a self-managed solr it hasn’t really come up. what does “concurrent” mean exactly in this case, how is it measured? with puma web workers and sometimes a possibly multi-threaded batch index going on, could we exceed a limit of ? seems plausible. what happens when they are exceeded? your solr request results in an http error!

do i need to now write the app to rescue those gracefully, or use connection pooling to try to avoid them, or something? having to rewrite the way our app functions for a particular managed solr is the last thing we want to do. (although it’s not entirely clear if those connection limits exist on the non-heroku-plugin plans, i suspect they do?).

and in general, i’m not thrilled with the way the pricing works here, and the price points. i am positive for a lot of (eg) heroku customers an additional $ * =$ /month is peanuts not even worth accounting for, but for us, a small non-profit whose app’s traffic does not scale with revenue, that starts to be real money.

it is not clear to me if websolr installations (at “standard” plans) are set up in “solrcloud mode” or not; i’m not sure what api’s exist for uploading your custom schema.xml (which we’d need to do), or if they expect you to do this only manually through a web ui (that would not be good); i’m not sure if you can upload custom solrconfig.xml settings (this may be running on a shared solr instance with standard solrconfig.xml?).

basically, all of this made websolr not the first one we looked at.

does it matter if we’re on heroku using a managed solr that’s not a heroku plugin?

i don’t think so.

in some cases, you can get a better price from a heroku plug-in than you could get from that same vendor not on heroku or other competitors. but that doesn’t seem to be the case here, and other that that does it matter?

well, all heroku plug-ins are required to bill you by-the-minute, which is nice but not really crucial, other forms of billing could also be okay at the right price.

with a heroku add-on, your billing is combined into one heroku invoice, no need to give a credit card to anyone else, and it can be tracked using heroku tools. which is certainly convenient and a plus, but not essential if the best tool for the job is not a heroku add-on.

and as a heroku add-on, websolr provides a websolr_url heroku config/env variable automatically to code running on heroku. ok, that’s kind of nice, but it’s not a big deal to set a solr_url heroku config manually referencing the appropriate address. i suppose as a heroku add-on, websolr also takes care of securing and authenticating connections between the heroku dynos and the solr, so we need to make sure we have a reasonable way to do this from any alternative.

searchstax (did take it for a spin)

searchstax’s pricing tiers are not based on metering usage. there are no limits based on requests/day or concurrent connections. searchstax runs on dedicated-to-you individual solr instances (i would guess running on dedicated-to-you individual (eg) ec , but i’m not sure). instead the pricing is based on size of host running solr.

you can choose to run on instances deployed to aws, google cloud, or azure. we’ll be sticking to aws (the others, i think, have a slight price premium).

while searchstax gives you a pricing pages that looks like the “new-way-of-doing-things” transparent pricing, in fact there isn’t really enough info on public pages to see all the price points and understand what you’re getting, there is still a kind of “talk to a salesperson who has a price sheet” thing going on.

what i think i have figured out from talking to a salesperson and support, is that the “silver” plans (“starting at $ a month”, although we’ll say more about that in a bit) are basically: we give you a solr, we don’t don’t provide any technical support for solr.

while the “gold” plans “from $ /month” are actually about paying for solr consultants to set up and tune your schema/index etc. that is not something we need, and $ +/month is way more than the price range we are looking for.

while the searchstax pricing/plan pages kind of imply the “silver” plan is not suitable for production, in fact there is no real reason not to use it for production i think, and the salesperson i talked to confirmed that — just reaffirming that you were on your own managing the solr configuration/setup. that’s fine, that’s what we want, we just don’t want to mangage the os or set up the solr or upgrade it etc. the silver plans have no sla, but as far as i can tell their uptime is just fine. the silver plans only guarantees -hour support response time — but for the couple support tickets i filed asking questions while under a free -day trial (oh yeah that’s available), i got prompt same-day responses, and knowledgeable responses that answered my questions.

so a “silver” plan is what we are interested in, but the pricing is not actually transparent.

$ /month is for the smallest instance available, and if you prepay/contract for a year. they call that small instance an ndn and it has gb of ram and gb of storage. if you pay-as-you-go instead of contracting for a year, that already jumps to $ /month. (that price is available on the trial page).

when you are paying-as-you-go, you are actually billed per-day, which might not be as nice as heroku’s per-minute, but it’s pretty okay, and useful if you need to bring up a temporary solr instance as part of a migration/upgrade or something like that.

the next step up is an “ndn ” which has g of ram and gb of storage, and has an ~$ /month pay-as-you-go — you can find that price if you sign-up for a free trial. the discount price price for an annual contract is a discount similar to the ndn %, $ /month — that price i got only from a salesperson, i don’t know if it’s always stable.

it only occurs to me now that they don’t tell you how many cpus are available.

i’m not sure if i can fit our solr in the g ndn , but i am sure i can fit it in the g ndn with some headroom, so i didn’t look at plans above that — but they are available, still under “silver”, with prices going up accordingly.

all searchstax solr instances run in “solrcloud” mode — these ndn and ndn ones we’re looking at just run one node with one zookeeper, but still in cloud mode. there are also “silver” plans available with more than one node in a “high availability” configuration, but the prices start going up steeply, and we weren’t really interested in that.

because it’s solrcloud mode though, you can use the standard solr api for uploading your configuration. it’s just solr! so no arbitrary usage limits, no features disabled.

the searchstax web console seems competently implemented; it let’s you create and delete individual solr “deployments”, manage accounts to login to console (on “silver” plan you only get two, or can pay $ /month/account for more, nah), and set up auth for a solr deployment. they support ip-based authentication or http basic auth to the solr (no limit to how many solr basic auth accounts you can create). http basic auth is great for us, because trying to do ip-based from somewhere like heroku isn’t going to work. all solrs are available over https/ssl — great!

searchstax also has their own proprietary http api that lets you do most anything, including creating/destroying deployments, managing solr basic auth users, basically everything. there is some api that duplicates the solr cloud api for adding configsets, i don’t think there’s a good reason to use it instead of standard solrcloud api, although their docs try to point you to it. there’s even some kind of webhooks for alerts! (which i haven’t really explored).

basically, searchstax just seems to be a sane and rational managed solr option, it has all the features you’d expect/need/want for dealing with such. the prices seem reasonable-ish, generally more affordable than websolr, especially if you stay in “silver” and “one node”.

at present, we plan to move forward with it.

opensolr (didn’t look at it much)

i have the least to say about this, have spent the least time with it, after spending time with searchstax and seeing it met our needs. but i wanted to make sure to mention it, because it’s the only other managed solr i am even aware of. definitely curious to hear from any users.

here is the pricing page.

the prices seem pretty decent, perhaps even cheaper than searchstax, although it’s unclear to me what you get. does “ solr clusters” mean that it’s not solrcloud mode? after seeing how useful solrcloud apis are for management (and having this confirmed by many of my peers in other libraries/museums/archives who choose to run solrcloud), i wouldn’t want to do without it. so i guess that pushes us to “executive” tier? which at $ /month (billed yearly!) is still just fine, around the same as searchstax.

but they do limit you to one solr index; i prefer searchstax’s model of just giving you certain host resources and do what you want with it. it does say “shared infrastructure”.

might be worth investigating, curious to hear more from anyone who did.

now, what about elasticsearch?

we’re using solr mostly because that’s what various collaborative and open source projects in the library/museum/archive world have been doing for years, since before elasticsearch even existed. so there are various open source libraries and toolsets available that we’re using.

but for whatever reason, there seem to be so many more managed elasticsearch saas available. at possibly much cheaper pricepoints. is this because the elasticsearch market is just bigger? or is elasticsearch easier/cheaper to run in a saas environment? or what? i don’t know.

but there’s the controversial aws elasticsearch service; there’s the elastic cloud “from the creators of elasticsearch”. on heroku that lists one solr add-on, there are three elasticsearch add-ons listed: elasticcloud, bonsai elasticsearch, and searchbox elasticsearch.

if you just google “managed elasticsearch” you immediately see or other names.

i don’t know enough about elasticsearch to evaluate them. there seem on first glance at pricing pages to be more affordable, but i may not know what i’m comparing and be looking at tiers that aren’t actually usable for anything or will have hidden fees.

but i know there are definitely many more managed elasticsearch saas than solr.

i think elasticsearch probably does everything our app needs. if i were to start from scratch, i would definitely consider elasticsearch over solr just based on how many more saas options there are. while it would require some knowledge-building (i have developed a lot of knowlege of solr and zero of elasticsearch) and rewriting some parts of our stack, i might still consider switching to es in the future, we don’t do anything too too complicated with solr that would be too too hard to switch to es, probably.

- - t : : + : jrochkind digital library federation: three new ndsa members https://www.diglib.org/three-new-ndsa-members/

since january , the ndsa coordinating committee unanimously voted to welcome three new members. each of these members bring a host of skills and experience to our group. please help us to welcome:

arkivum: arkivum is recognized internationally for its expertise in the archiving and digital preservation of valuable data and digitized assets in large volumes and multiple formats.
colorado state university libraries: colorado state university libraries’ digital preservation activities has focused on web archiving, targeted born-digital collecting, along with collection development and preservation guidelines for its digital repository.
vassar college libraries: vassar college libraries are committed to supporting a framework of sustainable access to our digital collections and to participate locally, nationally, and globally with other cultural and professional organizations and institutions in efforts to preserve, augment, and disseminate our collective documentary heritage.

each organization has participants in one or more of the various ndsa interest and working groups, so keep an eye out for them on your calls and be sure to give them a shout out. please join me in welcoming our new members. a complete list of ndsa members is on our website.

in future, ndsa is moving to a quarterly process for reviewing membership applications. announcements for new members will be scheduled accordingly.

~ nathan tallman, vice chair of the ndsa coordinating committee

the post three new ndsa members appeared first on dlf.

- - t : : + : nathan tallman duraspace news: fedora migration paths and tools project update: january https://duraspace.org/fedora-migration-paths-and-tools-project-update-january- /

this is the fourth in a series of monthly updates on the fedora migration paths and tools project – please see last month’s post for a summary of the work completed up to that point. this project has been generously funded by the imls.

the grant team has been focused on completing an initial build of a validation utility, which will allow implementers to compare their migrated content with the original fedora .x source material to verify that everything has been migrated successfully. a testable version of this tool is expected to be completed in the coming weeks, at which point the university of virginia pilot team will test and provide feedback on the utility.

the university of virginia team has completed a full migration of their legacy fedora . . repository. they also recently contributed improvements to the fedora aws deployer which have been merged into the codebase. the team is now awaiting a testable version of the validation utility so they can validate their migrated content before moving on to testing this content in a newly installed fedora . instance.

the whitman college pilot team has completed their metadata remediation and mapping work. their process and lessons learned will be shared in a presentation at the upcoming code lib conference. meanwhile, islandora is currently being tested with an alpha build of fedora . , which will be used as the basis for migration testing for the whitman college pilot. work is currently being done in parallel to install islandora using isle and complete work on a new theme. due to the impending end-of-life date of drupal the team decided to proceed directly to drupal , and the theme needed to be updated accordingly. fortunately, the transition from drupal to is relatively minor.

next month we plan to use the validation utility to validate the university of virginia migration before moving on to testing the migrated data in fedora . and updating the application as needed. for the whitman college pilot, once the islandora with fedora . installation is complete we will be able to run a series of test migrations and update the utilities and application as necessary in order to satisfy functional requirements.

stay tuned for future updates!

the post fedora migration paths and tools project update: january appeared first on duraspace.org.

- - t : : + : david wilcox open knowledge foundation: open knowledge justice programme takes new step on its mission to ensure algorithms cause no harm https://blog.okfn.org/ / / /open-knowledge-justice-programme-takes-new-step-on-its-mission-to-ensure-algorithms-cause-no-harm/

today we are proud to announce a new project for the open knowledge justice programme – strategic litigation. this might mean we will go to court to make sure public impact algorithms are used fairly, and cause no harm. but it will also include advocacy in the form of letters and negotiation.

the story so far

last year, open knowledge foundation made a commitment to apply our skills and network to the increasingly important topics of artificial intelligence (ai) and algorithms.

as a result, we launched the open knowledge justice programme in april . our mission is to ensure that public impact algorithms cause no harm.

public impact algorithms have four key features:

they involve automated decision-making
using ai and algorithms
by governments and corporate entities and
have the potential to cause serious negative impacts on individuals and communities.

we aim to make public impact algorithms more accountable by equipping legal professionals, including campaigners and activists, with the know-how and skills they need to challenge the effects of these technologies in their practice. we also work with those deploying public impact algorithms to raise awareness of the potential risks and build strategies for mitigating them. we’ve had some great feedback from our first trainees!

why are we doing this?

strategic litigation is more than just winning an individual case. strategic litigation is ‘strategic’ because it plays a part in a larger movement for change. it does this by raising awareness of the issue, changing public debate, collaborating with others fighting for the same cause and, when we win (hopefully!) making the law fairer for everyone.

our strategic litigation activities will be grounded in the principle of openness because public impact algorithms are overwhelmingly deployed opaquely. this means that experts that are able to unpick why and how ai and algorithms are causing harm cannot do so and the technology escapes scrutiny.

vendors of the software say they can’t release the software code they use because it’s a trade secret. this proprietary knowledge, although used to justify decisions potentially significantly impacting people’s lives, remains out of our reach.

we’re not expecting all algorithms to be open. nor do we think that would necessarily be useful.

but we do think it’s wrong that governments can purchase software and not be transparent around key points of accountability such as its objectives, an assessment of the risk it will cause harm and its accuracy.

openness is one of our guiding principles in how we’ll work too. as far as we are able, we’ll share our cases for others to use, re-use and modify for their own legal actions, wherever they are in the world. we’ll share what works, and what doesn’t, and make learning resources to make achieving algorithmic justice through legal action more readily achievable.

we’re excited to announce our first case soon, so stay tuned! sign up to our mailing list or follow the open knowledge justice programme on twitter to receive updates.

- - t : : + : meg foulkes david rosenthal: isp monopolies https://blog.dshr.org/ / /isp-monopolies.html for at least the last three years (it isn't about the technology) i've been blogging about the malign effects of the way the faangs dominate the web and the need for anti-trust action to mitigate them. finally, with the recent lawsuits against facebook and google, some action may be in prospect. i'm planning a post on this topic. but when it comes to malign effects of monopoly i've been ignoring the other monopolists of the internet, the telcos.

an insightful recent post by john gilmore to dave farber's ip list sparked a response from thomas leavitt and some interesting follow-up e-mail. gilmore was involved in pioneering consumer isps, and leavitt in pioneering web hosting. both attribute the current sorry state of internet connectivity in the us to the lack of effective competition. they and i differ somewhat on how the problem could be fixed. below the fold i go into the details.

i've known gilmore since the early days of sun microsystems, and i'm grateful for good advice he gave me at a critical stage of my career. he has a remarkable clarity of vision and a list of achievements that includes pioneering paid support for free software (the "red hat" model) at cygnus support, and pioneering consumer internet service providers (isps) with the little garden. because in those dial-up days internet connectivity was a commercial product layered on regulated infrastructure (the analog telephone system) there was no lock-in. consumers could change providers simply by changing the number their modem dialed.

this experience is key to gilmore's argument. he writes:

the usa never had "network neutrality" before it was "suspended". what the usa had was , isps. so if an isp did something unfriendly to its customers, they could just stop paying the bad one, and sign up with a different isp that wouldn't screw them. that effectively prevented bad behavior among isps. and if the customer couldn't find an isp that wouldn't screw them, they could start one themselves. i know, because we did exactly that in the s.

anyone could start an isp because by law, everyone had tariffed access to the same telco infrastructure (dialup phone lines, and leased lines at kbit/sec or . mbit/sec or mbit/sec). you just called up the telco and ordered it, and they sent out techs and installed it. we did exactly that, plugged it into our modems and routers and bam, we were an isp: "the little garden".

i was an early customer of the little garden. a sparcstation, a scsi disk and a modem sat on my window-ledge. the system dialed a local, and thus free, number and kept the call up / , enabling me to register a domain and start running my own mail server. years later i upgraded to dsl with stanford as my isp. as gilmore points out, stanford could do this under the same law:

later, dsl lines required installing equipment in telco central offices, at the far end of the wire that leads to your house. but the telcos were required by the fcc to allow competing companies to do that. their central office buildings were / th empty anyway, after they had replaced racks of mechanical relays with digital computers.

gilmore explains how this competitive market was killed:

the telcos figured this out, and decided they'd rather be gatekeepers, instead of being the regulated monopoly that gets a fixed profit margin. looking ahead, they formally asked the fcc to change its rule that telcos had to share their infrastructure with everybody -- but only for futuristic optical fibers. they whined that "fcc wants us to deploy fiber everywhere, but we won't, unless we get to own it and not share it with our competitors." as usual, the regulated monopoly was great at manipulating the public interest regulators. the fcc said, "sure, keep your fibers unshared." this ruling never even mentioned the internet, it is all about the physical infrastructure. if the physical stuff is wires, regulated telcos have to share it; if it's glass, they don't.

the speed of dialup maxed out at kbit/sec. dsl maxed out at a couple of megabits. leased lines worked to mbit/sec but cost thousands of dollars per month. anything over that speed required fiber, not wire, at typical distances. as demand for higher internet speeds arose, any isp who wanted to offer a faster connection couldn't just order one from the telco, because the telco fibers were now private and unshared. if you want a fiber-based internet connection now, you can't buy it from anybody except the guys who own the fibers -- mostly the telcos. most of the , isps could only offer slow internet access, so everybody stopped paying them. the industry consolidated down to just one or a few businesses per region -- mostly the telcos themselves, plus the cable companies that had build their own local monopoly via city government contracts. especially lucky regions had maybe one other competitor, like a wireless isp, or an electrical co-op that ran fibers on its infrastructure.

leavitt makes a bigger point than glimore's:

the only reason the internet exists as we know it (mass consumer access) was the regulatory loophole which permitted the isp industry to flourish in the s. the telcos realized their mistake, as john said, and made sure that there wasn't going to be a repeat of that, so with each generation (dsl, fiber), they made it more and more difficult to access their networks, with the result that john mention--almost no consumer choice, for consumers or business. last office i rented, there was one choice of internet provider: the local cable monopoly, which arbitrarily wanted to charge me much more ($ /mo) to connect my office than it did the apartments upstairs in the same building ($ ). as is the case in most of that county; the only alternatives were a few buildings and complexes wired up by the two surviving local isps, and a relatively expensive wisp.

gilmore concludes:

the telcos' elimination of fiber based competition, and nothing else, was the end of so-called "network neutrality". the rest was just activists, regulators and legislators blathering. there never was an /enforceable federal regulatory policy of network neutrality, so the fcc could hardly suspend it. if the fcc actually wanted us customers to have a choice of isps, they would rescind the fiber rule. and if advocates actually understood how only competition, not regulation, restrains predatory behavior, they would ask fcc for the fiber rule to be rescinded, so a small isp company could rent the actual glass fiber that runs from the telco to (near or inside) your house, for the actual cost plus a regulated profit. then customers could get high speed internet from a variety of vendors at a variety of prices and terms. so far neither has happened.

leavitt shows the insane lengths we are resorting to in order to deliver a modicum of competition in the isp market:

it's ridiculous that it is going to take sending s of thousands of satellites into orbit to restore any semblance of competitiveness to the isp market, when we've had a simple regulatory fix all along. it's not like the telco/cable monopolies suffered as a result of competition... in fact, it created the market they now monopolize. imagine all the other opportunities for new markets that have been stifled by the lack of competition in the isp market over the last two decades!

i have been, and still am, an exception to gilmore's and leavitt's experiences. palo alto owns its own utilities, a great reason to live there. in september palo alto's fiber to the home trial went live, and i was one of citizens who got a mbit/s bidirectional connection, with the city utilities as our isp. we all loved the price, the speed and the excellent customer service. the telcos got worried and threatened to sue the utilities if it expanded the service. the city was on safe legal ground, but that is what they had thought previously when they lost a $ . m lawsuit as part of the fallout from the enron scandal. enron's creditors claimed that the utilities had violated their contract because they stopped paying enron. the utilities did so because enron became unable to deliver them electricity.

so, when the trial ended after i think five years, we loved it so much that we negotiated with motorola to take over the equipment and found an upstream isp. but the utilities were gun-shy and spent iirc $ k physically ripping out the fiber and trashing the equipment. since then, palo alto's approach to municipal fiber has been a sorry story of ineffective dithering.

shortly after we lost our fiber, stanford decided to stop providing staff with dsl, but we again avoided doing business with the telcos. we got dsl and phone service from sonic, a local isp that was legally enabled to rent access to at&t's copper. it was much slower than comcast or at&t, but the upside was sonic's stellar customer service and four static ip addresses. that kept us going quite happily until covid- struck and we had to host our grandchildren for their virtual schools. dsl was not up to the job.

fortunately, it turned out that sonic had recently been able to offer gigabit fiber in palo alto. sonic in its north bay homeland has been deploying its own fiber, as has cruzio in its santa cruz homeland. here they rent access to at&t's fiber in the same way that they rented access to the copper. so, after a long series of delays caused by at&t's inability to get fiber through the conduit under the street that held their copper, we have gigabit speed, home phone and sonic's unmatched customer service all for $ /month.

as a long-time sonic customer, i agree with what the internet advisor website writes:

sonic has maintained a reputation as not only a company that delivers a reliable high-speed connection to its customers but also a company that stands by its ethics. both dane jasper and scott doty have spoken up on numerous occasions to combat the ever-growing lack of privacy on the web. they have implemented policies that reflect this. in , they reduced the amount of time that they store user data to just two weeks in the face of an ever-growing tide of legal requests for its users’ data. that same year, sonic alongside google fought a court order to hand over email addresses who had contacted and had a correspondence with tor developer and wikileaks contributor jacob applebaum. when asked why, ceo dane jasper responded that it was “rather expensive, but the right thing to do.”

sonic has made a habit of doing the right thing, both for its customers and the larger world. it’s a conscientious company that delivers on what is promised and goes the extra mile for its subscribers.

leavitt explained in e-mail how sonic's exception to gilmore's argument came about:

sonic is one of the few independent isps that's managed to survive the regulatory clampdown via stellar customer service and customers willing to go out of their way to support alternative providers, much like cruzio in my home town of santa cruz. they cut some kind of reseller deal with at&t back in that enabled them to offer fiber to a limited number of residents, and again, like cruzio, are building out their own fiber network, but according to [this site], fiber through them is potentially available to only about , customers (in a state with about million households and million businesses); it also reports that they are the th largest isp in the nation, despite being a highly regional provider with access available to only about million households. this says everything about how monopolistic and consolidated the isp market is, given the number of independent cable and telco companies that existed in previous decades, the remaining survivors of which are all undoubtedly offering isp services.

i doubt sonic's deal with at&t was much more lucrative than the dsl deals santa cruz area isps were able to cut.

gilmore attempted to build a fiber isp in his hometown, san francisco:

our model was to run a fiber to about one person per block (what cruzio calls a "champion") and teach them how to run and debug g ethernet cables down the back fences to their neighbors, splitting down the monthly costs. this would avoid most of the cost of city right-of-way crud at every house, which would let us and our champions fiber the city much more broadly and quickly. and would train a small army of citizens to own and manage their own infrastructure.

for unrelated reasons it didn't work out, but it left gilmore with the conviction that, absent repeal of the fiber rule, isp-owned fiber is the way to go. especially in rural areas this approach has been successful; a recent example was described by jon brodkin in jared mauch didn't have good broadband—so he built his own fiber isp. leavitt argues:

i'd like to see multiple infrastructure providers, both private for profit, and municipally sponsored non-profit public service agencies, all with open access networks; ideally, connecting would be as simple as it was back in the dial up days. i think we need multiple players to keep each other "honest". i do agree that a lot of the barriers to building out local fiber networks are regulatory and process, as john mentions. the big incumbent players have a tremendous advantage navigating this process, and the scale to absorb the overhead of dealing with them in conjunction with the capital outlays (which municipalities also have).

i think we all agree that "ideally, connecting would be as simple as it was back in the dial up days". how to make this happen? as gilmore says, there are regulatory and process costs as well as the cost of pulling the fiber. so if switching away from a misbehaving isp involves these costs there is going to a significant barrier. it isn't going to be "as simple as it was back in the dial up days" when the customer could simply re-program their modem.

my experience of municipal fiber leads me to disagree with both gilmore and leavitt. for me, the key is to separate the provision of fiber from the provision of internet services. why would you want to switch providers?

pretty much the only reason why you'd want to switch fiber providers is unreliability. but, absent back-hoes, fiber is extremely reliable.
there are many reasons why you'd want to switch isps, among them privacy, bandwidth caps, price increases.

municipal fiber provision is typically cheap, because they are the regulator and control the permitting process themselves, and because they are accountable to their voters. and if they simply provide the equivalent of an ethernet cable to a marketplace of isps, each of them will be paying the same for their connectivity. so differences in the price of isp service will reflect the features and quality of their service offerings.

the cost of switching isps would be low, simply reprogramming the routers at each end of the fiber. the reason the telcos want to own the fiber isn't because owning fiber as such is a good business, it is to impose switching costs and thus lock in their customers. we don't want that. but equally we don't want switching isps to involve redundantly pulling fiber, because that imposes switching costs too. the only way to make connecting "as simple as it was back in the dial up days" is to separate fiber provision from internet service provision, so that fiber pets pulled once and rented to competing isps. if we are going to have a monopoly at the fiber level, i'd rather have a large number of small monopolies than the duopoly of at&t and comcast. and i'd rather have the monopoly accountable to voters.

- - t : : + : david. (noreply@blogger.com) lucidworks: cast a smarter net with semantic vector search https://lucidworks.com/post/cast-a-smarter-net-semantic-vector-search/

learn how to power the product discovery experience with semantic vector search to eliminate false zero results and accelerate the path to purchase.

the post cast a smarter net with semantic vector search appeared first on lucidworks.

- - t : : + : garrett schwegler digital library federation: virtual ndsa digital preservation recordings available online! https://www.diglib.org/virtual- -ndsa-digital-preservation-recordings-available-online/

session recordings from the virtual ndsa digital preservation conference are now available on ndsa’s youtube channel, as well as on aviary. the full program from digital preservation : get active with digital preservation, which took place online november , , is free and open to the public.

ndsa is an affiliate of the digital library federation (dlf) and the council on library and information resources (clir). each year, ndsa’s annual digital preservation conference is held alongside the dlf forum and acts as a crucial venue for intellectual exchange, community-building, development of good practices, and national agenda-setting for digital stewardship.

enjoy,

tricia patterson; digipres vice-chair, chair

the post virtual ndsa digital preservation recordings available online! appeared first on dlf.

- - t : : + : kussmann terry reese: marcedit . .x/macos . .x timelines https://blog.reeset.net/archives/

i sent this to the marcedit listserv to provide info about my thoughts around timelines related to the beta and release. here’s the info.

dear all,

as we are getting close to feb. (when i’ll make the . beta build available for testing) – i wanted to provide information about the update process going forward.

feb. :

marcedit . download will be released. this will be a single build that includes both the and bit builds, dependencies, and can install if you have admin rights or non-admin rights.

i expect to be releasing new builds weekly – with the goal of taking the beta tag off the build no later than april .

marcedit . .x

i’ll be providing updates for . .x till . comes out of beta. this will fold in some changes (mostly bug fixes) when possible.

marcedit macos . .x

i’ll be providing updates for macos . .x till . is out and out of beta

marcedit macos . .x beta

once marcedit . .x beta is out, i’ll be looking to push a . .x beta by mid-late feb. again, with the idea of taking the beta tag off by april (assuming i make the beta timeline)

march

marcedit macos . .x beta will be out and active (with weekly likely builds)
marcedit . .x beta – testing assessed and then determine how long the beta process continues (with april being the end bookend date)
marcedit . .x – updates continue
marcedit macos . .x – updates continue

april

marcedit . .x comes out of beta
marcedit . .x is deprecated
marcedit macos . .x beta assessed – end bookend date is april ^th if above timelines are met

may

marcedit macos . .x is out of beta
marcedit macos . .x is deprecated

let me know if you have questions.

- - t : : + : reeset jez cope: a new font for the blog https://erambler.co.uk/blog/new-font/

i’ve updated my blog theme to use the quasi-proportional fonts iosevka aile and iosevka etoile. i really like the aesthetic, as they look like fixed-width console fonts (i use the true fixed-width version of iosevka in my terminal and text editor) but they’re actually proportional which makes them easier to read.
https://typeof.net/iosevka/

- - t : : + : jez cope jez cope: training a model to recognise my own handwriting https://erambler.co.uk/blog/training-a-handwriting-model/

if i’m going to train an algorithm to read my weird & awful writing, i’m going to need a decent-sized training set to work with. and since one of the main things i want to do with it is to blog “by hand” it makes sense to focus on that type of material for training. in other words, i need to write out a bunch of blog posts on paper, scan them and transcribe them as ground truth. the added bonus of this plan is that after transcribing, i also end up with some digital text i can use as an actual post — multitasking!

so, by the time you read this, i will have already run it through a manual transcription process using transkribus to add it to my training set, and copy-pasted it into emacs for posting. this is a fun little project because it means i can:

write more by hand with one of my several nice fountain pens, which i enjoy
learn more about the operational process some of my colleagues go through when digitising manuscripts
learn more about the underlying technology & maths, and how to tune the process
produce more lovely content! for you to read! yay!
write in a way that forces me to put off editing until after a first draft is done and focus more on getting the whole of what i want to say down.

that’s it for now — i’ll keep you posted as the project unfolds.

addendum

tee hee! i’m actually just enjoying the process of writing stuff by hand in long-form prose. it’ll be interesting to see how the accuracy turns out and if i need to be more careful about neatness. will it be better or worse than the big but generic models used by samsung notes or onenote. maybe i should include some stylus-written text for comparison.

- - t : : + : jez cope lucidworks: learning from einstein’s brain https://lucidworks.com/post/activate-learning-from-einsteins-brain/

einstein’s remarkable brain has an important lesson about balance for all of us in technology and machine learning.

the post learning from einstein’s brain appeared first on lucidworks.

- - t : : + : ellen leanse meredith farkas: making customizable interactive tutorials with google forms https://meredith.wolfwater.com/wordpress/ / / /making-customizable-interactive-tutorials-with-google-forms/

in september, i gave a talk at oregon state university’s instruction librarian get-together about the interactive tutorials i built at pcc last year that have been integral to our remote instructional strategy. i thought i’d share my slides and notes here in case others are inspired by what i did and to share the amazing assessment data i recently received about the impact of these tutorials that i included in this blog post. you can click on any of the slides to see them larger and you can also view the original slides here (or below). at the end of the post are a few tutorials that you can access or make copies of.

i’ve been working at pcc for over six years now, but i’ve been doing online instructional design work for years and i will freely admit that it’s my favorite thing to do. i started working at a very small rural academic library where i had to find creative and usually free solutions to instructional problems. and i love that sort of creative work. it’s what keeps me going.

i’ve actually been using survey software as a teaching tool since i worked at portland state university. there, my colleague amy hofer and i used qualtrics to create really polished and beautiful interactive tutorials for students in our university studies program.

i also used qualtrics at psu and pcc to create pre-assignments for students to complete prior to an instruction session that both taught students skills and gave me formative assessment data that informed my teaching. so for example, students would watch a video on how to search for sources via ebsco and then would try searching for articles on their own topic.

a year and a half ago, the amazing anne-marie dietering led my colleagues in a day of goal-setting retreat for our instruction program. in the end, we ended up selecting this goal, identify new ways information literacy instruction can reach courses other than direct instruction, which was broad enough to encompass a lot of activities people valued. for me, it allowed me to get back to my true love, online instructional design, which was awesome, because i was kind of in a place of burnout going into last fall.

at pcc, we already had a lot of online instructional content to support our students. we even built a toolkit for faculty with information literacy learning materials they could incorporate into their classes without working with a librarian.

farkas_googleformspresentation ( )

the toolkit contains lots of handouts, videos, in-class or online activities and more. but it was a lot of pieces and they really required faculty to do the work to incorporate them into their classes.

what i wanted to build was something that took advantage of our existing content, but tied it up with a bow for faculty. so they really could just take whatever it is, assign students to complete it, and know students are learning and practicing what they learned. i really wanted it to mimic the sort of experience they might get from a library instruction session. and that’s when i came back to the sort interactive tutorials i built at psu.

so i started to sketch out what the requirements of the project were. even though we have qualtrics at pcc, i wasn’t % sure qualtrics would be a good fit for this. it definitely did meet those first four criteria given that we already have it, it provides the ability to embed video, for students to get a copy of the work they did, and most features of the software are ada accessible. but i wanted both my colleagues in the library and disciplinary faculty members to be able to easily see the responses of their students and to make copies of the tutorial to personalize for the particular course. and while pcc does have qualtrics, the majority of faculty have never used it on the back-end and many do not have accounts. so that’s when google forms seemed like the obvious choice and i had to give up on my fantasy of having pretty tutorials.

i started by creating a proof of concept based on an evaluating sources activity i often use in face-to-face reading and writing classes. you can view a copy of it here and can copy it if you want to use it in your own teaching.

in this case, students would watch a video we have on techniques for evaluating sources. then i demonstrate the use of those techniques, which predate caulfield’s four moves, but are not too dissimilar. so they can see how i would go about evaluating this article from the atlantic on the subject of daca.

the students then will evaluate two sources on their own and there are specific questions to guide them.

during fall term, i showed my proof of concept to my colleagues in the library as well as at faculty department meetings in some of my liaison areas. and there was a good amount of enthusiasm from disciplinary faculty – enough that i felt encouraged to continue.

one anthropology instructor who i’ve worked closely with over the years asked if i could create a tutorial on finding sources to support research in her online biological anthropology classes – classes i was going to be embedded in over winter term. and i thought this was a perfect opportunity to really pilot the use of the google form tutorial concept and see how students do.

so i made an interactive tutorial where students go through and learn a thing, then practice a thing, learn another thing, then practice that thing. and fortunately, they seemed to complete the tutorial without difficulty and from what i heard from the instructor, they did a really good job of citing quality sources in their research paper in the course. later in the presentation, you’ll see that i received clear data demonstrating the impact of this tutorial from the anthropology department’s annual assessment project.

so my vision for having faculty make copies of tutorials to use themselves had one major drawback. let’s imagine they were really successful and we let a thousand flowers bloom. well, the problem with that is that you now have a thousand versions of your tutorials lying around and what do you do when a video is updated or a link changes or some other update is needed? i needed a way to track who is using the tutorials so that i could contact them when updates were made.

so here’s how i structured it. i created a qualtrics form that is a gateway to accessing the tutorials. faculty need to put in their name, email, and subject area. they then can view tutorials and check boxes for the ones they are interested in using.

once they submit, they are taking to a page where they can actually copy the tutorials they want. so now, i have the contact information for the folks who are using the tutorials.

this is not just useful for updates, but possibly for future information literacy assessment we might want to do.

the individual tutorials are also findable via our information literacy teaching materials toolkit.

so when the pandemic came just when i was ready to expand this, i felt a little like nostradamus or something. the timing was very, very good during a very, very bad situation. so we work with biology every single term in week to teach students about the library and about what peer review means, why it matters, and how to find peer-reviewed articles.

farkas_googleformspresentation ( )
as soon as it became clear that spring term was going to start online, i scrambled to create this tutorial that replicates, as well as i could, what we do in the classroom. so they do the same activity we did in-class where they look at a scholarly article and a news article and list the differences they notice. and in place of discussions, i had them watch videos and share insights. i then shared this with the biology faculty on my campus and they assigned it to their students in week . it was great! [you can view the biology tutorial here and make a copy of it here]. and during spring term i made a lot more tutorials.

the biggest upside of using google forms is its simplicity and familiarity. nearly everyone has created a google form and they are dead simple to build. i knew that my colleagues in the library could easily copy something i made and tailor it to the courses they’re working with or make something from scratch. and i knew faculty could easily copy an existing tutorial and be able to see student responses. for students, it’s a low-bandwidth and easy-to-complete online worksheet. the barriers are minimal. and on the back-end, just like with libguides, there’s a feature where you can easily copy content from another google form.

the downsides of using google forms are not terribly significant. i mean, i’m sad that i can’t create beautiful, modern, sharp-looking forms, but it’s not the end of the world. the formatting features in google forms are really minimal. to create a hyperlink, you actually need to display the whole url. blech. then in terms of accessibility, there’s also no alt tag feature for images, so i just make sure to describe the picture in the text preceding or following it. i haven’t heard any complaints from faculty about having to fill out the qualtrics form in order to get access to the tutorials, but it’s still another hurdle, however small.

this spring, we used google form tutorials to replace the teaching we normally do in classes like biology , writing , reading , and many others. we’ve also used them in addition to synchronous instruction, sort of like i did with my pre-assignments. but word about the google form tutorials spread and we ended up working with classes we never had a connection to before. for example, the biology faculty told the anatomy and physiology instructors about the tutorial and they wanted me to make a similar one for a&p. and that’s a key class for nursing and biology majors that we never worked with before on my campus. lots of my colleagues have made copies of my tutorials and tailored them to the classes they’re working with or created their own from scratch. and we’ve gotten a lot of positive feedback from faculty, which really felt good during spring term when i know i was working myself to the bone.

since giving this presentation, i learned from my colleagues in anthropology that they actually used my work as the basis of their annual assessment project (which every academic unit has to do). they used a normed rubric to assess student papers in anthropology and compared the papers of students who were in sections in which i was embedded (where they had access to the tutorial) to students in sections where they did not have an embedded librarian or a tutorial. they found that students in the class sections in which i was involved had a mean score of / and students in other classes had a mean score of / . that is significant!!! i am so grateful that my liaison area did this project that so validates my own work.

here’s an excerpt from one email i received from an anatomy and physiology instructor: “i just wanted to follow up and say that the library assignment was a huge success! i’ve never had so many students actually complete this correctly with peer-reviewed sources in correct citation format. this is a great tool.” at the end of a term where i felt beyond worked to the bone, that was just the sort of encouragement i needed.

i made copies of a few other tutorials i’ve created so others can access them:

- - t : : + : meredith farkas journal of web librarianship: the culture of digital scholarship in academic libraries https://www.tandfonline.com/doi/full/ . / . . ?ai= dl&mi=co bk&af=r .
- - t : : + : bradford lee eden hugh rundle: library map part - why https://www.hughrundle.net/library-map-part- /

this weekend past i ran the generous & open galleries, libraries, archives & museums (go glam) miniconf at linuxconf.au, with bonnie wildie. being a completely online conference this year, we had an increased pool of people who could attend and also who could speak, and managed to put together what i think was a really great program. i certainly learned a lot from all our speakers, and i'll probably share some thoughts on the talks and projects later this year.

i also gave a short talk, about my new library map project and some thoughts on generosity in providing open data. unfortunately, alissa is completely right about my talk. the tone was wrong. i spoke about the wrong things and in the wrong way. it was an ungenerous talk on the virtues of generosity. i allowed my frustration at underfunded government bureaucracies and my anxiety about the prospect of giving a "technical" talk that "wasn't technical enough" for lca to overwhelm the better angels of my nature. i won't be sharing the video of my own talk when it becomes available, but here is a short clip of me not long after i delivered it:

via giphy

so i'm trying again. in this post i'll outline the basic concepts, and the why of library map - why i wanted to make it, and why i made the architecture and design choices i've made. in the next post, i'll outline how i built it - some nuts and bolts of which code is used where (and also, to some extent, why). you may be interested in one, or the other, or neither post 🙂.

library map

the library map is a map of libraries in australia and its external territories. there are three 'layers' to the map:

libraries

the libraries layer shows every public library in australia, plus an indicative m radius around it. also mapped on additional overlays are state and national libraries, indigenous knowledge centres, and most still-operating mechanics institutes.

rules

the rules layer has two overlays.

the fines overlay colour-codes each library service area according to whether they charge overdue fines for everyone, only for adults, or not at all.

the loan periods overlay uses patterns (mostly stripes) to indicate the standard loan period in weeks ( , , , or as it turns out).

library management software

the library management software layer works basically the same as the rules layer, except it colour codes library services according to which library management system (a.k.a intergrated library system) they use.

what is the library map for?

i've wanted something like this map at various times in the past. there is a fair amount of information around at the regional and state level about loan periods, or fine regimes, and even library management systems. but a lot of this is in people's heads, or in lists within pdf documents. i'm not sure i'd call myself a 'visual learner' but sometimes it is much clearer to see something mapped out visually than to read it in a table.

the intended audience for the map is actually a little bit "inside baseball". i'm not trying to build a real-time guide for library users to find things like current opening hours. google maps does a fine job of that, and i'm not sure a dedicated site for every public library but only libraries is a particularly useful tool. it would also be a nightmare to maintain. the site ultimately exists because i wanted to see if i could do it, but i had — broadly — two specific use cases in mind:

mapping library management systems to visualise network effects

my talk at lca was called who else is using it? — in reference to a question library managers often ask when confronting a suggestion to use a particular technology, especially something major like a library management system. this is understandable — it's reassuring to know that one's peers have made similar decisions ("nobody gets fired for buying ibm"), but there are also genuine advantages to having a network of fellow users you can talk to about shared problems or desired features. i was interested in whether these sorts of networks and aggregate purchasing decisions might be visible if they were mapped out, in a different way to what might be clear from a list or table. especially at a national level — i suspected there were strong trends within states in contrasts between them, but didn't have a really clear picture.

the state library of queensland was invaluable in this regard, because they have a list of every library service in the state and which library management system they use. when visiting library service websites it turned out that identifying the lms was often the easiest piece of data to find — much easier than finding out whether they charge overdue fines! it turns out there are very strong trends within each state — stronger than i expected — but western australia is a much more fractured and diverse market than i had thought. i also discovered a bunch of library management systems i had never heard of, so that was fun. this layer is the most recent — i only added it today — so there may still be some improvements to be made in terms of how the data is displayed.

mapping overdue fines

the second thing i wanted to map was whether and how libraries charge overdue fines, but my reason was different. i actually started the map with this layer, as part of a briefing i gave to some incoming victorian local government councillors about what they should know about public libraries.

here, the goal is mapping as an advocacy tool, using the peer pressure of "who else is charging it?" to slowly flip libraries to go fine-free. fines for overdue library books are regressive and counter-productive. i have found no compelling or systematic evidence that they have any effect whatsoever on the aggregate behaviour of library users in terms of returning books on time. they disproportionally hurt low income families. they need to go.

in victoria there has been a growing movement in the last few years for public libraries to stop charging overdue fines. i wasn't really aware of the situation in other states, but it turns out the whole northern territory has been fine-free for over a decade, and most libraries in queensland seem to also be fine-free. i'm still missing a fair bit of data for other states, especially south and western australia. what i'm hoping the map can be used for (once the data is more complete) is to identify specific libraries that charge fines but are near groups of libraries that don't, and work with the local library networks to encourage the relevant council to see that they are the odd ones out. i've worked in public libraries and know how difficult this argument can be to make from the inside, so this is a tool for activists but also to support library managers to make the case.

as if often a problem in libraries, i had to define a few terms and therefore "normalise" some data in order to have it make any sense systematically. so "no fines for children" is defined as any system that has a "younger than" exclusion for library fines or an exclusion for items designated as "children's books". some libraries are fine free for users under , others for those under , some only for children's book loans and so on. on my map they're all the same. the other thing to normalise was the definition of "overdue fine", which you might think is simple but turns out to be complex. in the end i somewhat arbitrarily decided that if there is no fee earlier than days overdue, that is classified as "no overdue fines". some libraries charge a "notice fee" after two weeks (which does count), whilst others send an invoice for the cost of the book after days (which doesn't).

colonial mode

as the project has progressed, some things have changed, especially how i name things. when i first added the libraries layer, i was only looking at victoria, using the directory of public library services in victoria. this includes mechanics institutes as a separate category, and that seemed like a good idea, so i had two overlays, in different colours. then i figured i should add the national library, and the state libraries, as a separate layer, since they operate quite differently to local public libraries.

once i got to queensland, i discovered that the state library of queensland not only provides really good data on public libraries, but also had broadly classified them into three categories: "rlq" for rural libraries queensland, a reciprocal-borrowing arrangement; "ind" for independent library services, and "ikc" for "indigenous knowledge centre". the immediate question for me was whether i would also classify any of these libraries as something different to a "standard" public library.

the main thing that distinguishes the rlq network from the "independents" is that it is a reciprocal lending network. in this regard, it's much the same as libraries victoria (formerly the swift consortium), or shorelink. there are other ways that rural libraries in queensland operate differently to urban libraries in queensland, but i don't think these differences make them qualitatively different in terms of their fundamental nature.

but what about indigenous knowledge centres? i admit i knew very little about them, and i still only know what i've gleaned from looking at ikc websites. the torres strait island regional council website seems to be fairly representative:

our indigenous knowledge centres endeavour to deliver new technology, literacy and learning programs to empower our communities through shared learning experiences. we work with communities to preserve local knowledge and culture and heritage, to keep our culture strong for generations.

the big difference between an ikc and a typical public library is that the focus is on preserving local indigenous knowledge and culture, which does happen through books and other library material, but is just as likely to occur through classes and activities such as traditional art and dance.

but the more i looked at this difference, the less different it seemed to be. public libraries across the world have begun focussing more on activities and programs in the last two decades, especially in weird countries. public libraries have always delivered new technology, literacy and learning programs. and the ‌directory of public library services in victoria amusingly reports that essentially every library service in victoria claims to specialise in local history. what are public libraries for, if not to "keep our culture strong for generations"?

yet it still felt to me that indigenous knowledge centres are operating from a fundamentally different mental model. finally it dawned on me that the word "our" is doing a lot of work in that description. our indigenous knowledge centres, keep our culture strong for generations. i was taken back to a conversation i've had a few times with my friend baruk jacob, who lives in aotearoa but grew up in a minority-ethnicity community in india. baruk maintains that public libraries should stop trying to be universally "inclusive" — that they are fundamentally eurocentric institutions and need to reconcile themselves to staying within that sphere. in this line of thinking, public libraries simply can't serve indigenous and other non-"western" people appropriately as centres of knowledge and culture. i could see where baruk was coming from, but i was troubled by his argument, and the implication that different cultural traditions could never be reconciled. as i struggled to decide whether indigenous knowledge centres were public libraries, or something else, i think i started to understand what baruk meant.

i'd been thinking about this back to front. indigenous knowledge centre is a usefully descriptive term. these places are centres for indigenous knowledge. the problem wasn't how to classify ikcs, but rather how to classify the other thing. the activities might be the same, but the our is different. i thought about what a non-indigenous knowledge centre might be. what kind of knowledge does it want to "keep strong for generations"? i thought about all those local history collections full of books about "pioneers" and family histories of "first settlers". if it's not indigenous knowledge, it must be settler knowledge. when i first saw this term being used by aboriginal activists in reference to non-indigenous residents generally, and white australians specifically, i bristled. i mean, sure, the modern culture is hopelessly dismissive of , years of human occupation, culture and knowledge, but how could i be a "settler" when i have five or six generations of australian-born ancestors? but a bit of discomfort is ok, and i have rather hypocritical ideas about other settler-colonial communities. it's exactly the right term to describe the culture most australians live in.

so i renamed "public libraries" as "settler knowledge centres". i initially renamed the national & state libraries to "imperial knowledge centres", but later decided it was more accurate to call them "colonial knowledge centres". i also briefly renamed mechanics institutes to worker indoctrination centres, but that's not entirely accurate and i realised i was getting carried away. i wasn't completely oblivious to the fact that this nomenclature could be a bit confusing, so i cheekily created two views: the "general" view which would be the default, and a second view which would appear on clicking "view in white fragility mode". this second mode would show the more familiar names "public libraries" and "national & state libraries".

while i was doing some soul searching this morning about my go glam talk, i continued to work on the map. my cheeky joke about "white fragility mode" had made me slightly uncomfortable from the moment i'd created it, but i initially brushed it off as me worrying too much about being controversial. but i realised today that the real problem was that calling it "white fragility mode" sabotages the entire point of the feature. the default language of "settler knowledge centre" and "colonial knowledge centre" sitting next to "indigenous knowledge centre" is intended to invite map users to think about the work these institutions do to normalise certain types of knowledge, and to "other" alternative knowledge systems and lifeworlds. the point is to bring people in to sit with the discomfort that comes from seeing familiar things described in an unfamiliar way. calling it "white fragility mode" isn't inviting, it's smug. it either pushes people away, or invites them to think no more about it because they're already woke enough to get it.

so today i changed it to something hopefully more useful. general mode is now called standard mode, and white fragility mode is now called colonial mode. it's the mode of thinking that is colonial, not the reader. flicking to colonial mode is ok if you need the more familiar terms to get your bearings: but hopefully by making it the non-standard view, users of the map are encouraged to think about libraries and about australia in a slightly different way. they don't have to agree that the "standard mode" terminology is better.

so that's some background behind why i started building the map and why i made some of the decisions i have about how it works. you can check it out at librarymap.hugh.run and see (most of) the code and data i used to build it on github. next time join me for a walk through how i made it.

- - t : : + : hugh rundle nick ruest: four fucking years of donald trump https://ruebot.net/post/ -four-fucking-years-of-donald-trump/

nearly four years ago i decided to start collecting tweets to donald trump out of morbid curiosity. if i was a real archivist, i would have planned this out a little bit better, and started collecting on election night in , or inaguration day . i didn’t. using twarc, i started collecting with the filter (streaming) api on may , . that process failed, and i pivoted to using the search api. i dropped that process into a simple bash script, and pointed cron at it to run every days. here’s what the bash script looked like:

#!/bin/bash date=`date +"%y_%m_%d"` cd /mnt/vol /data_sets/to_trump/raw /usr/local/bin/twarc search 'to:realdonaldtrump' --log donald_search_$date.log > donald_search_$date.json

it’s not beautiful. it’s not perfect. but, it did the job for the most part for almost four years save and except a couple twitter suspensions on accounts that i used for collection, and an absolutely embarassing situtation where i forgot to setup cron correctly on a machine i moved the collecting to for a couple weeks while i was on family leave this past summer.

in the end, the collection ran from may , - january , , and collected , , unique tweets; . t of line-delminted json! the final created_at timestamp was wed jan : : + , and the text of that tweet very fittingly reads, “@realdonaldtrump you’re fired!“

the “dehydrated” tweets can be found here. in that dataset i decided to include in a number of derivatives created with twut which, i hope rounds out the dataset. this update is the final update on the dataset.

i also started working on some notebooks here where i’ve been trying to explore the dataset a bit more in my limited spare time. i’m hoping to have the time and energy to really dig into this dataset sometime in the future. i’m especially curious at what the leadup to the storming of the united states capitol looks like in the dataset, as well as the sockpuppet frequency. i’m also hopeful that others will explore the dataset and that it’ll be useful in their research. i have a suspicion folks can do a lot smarter, innovative, and creative things with the dataset than i did here, here, here, here, or here.

for those who are curious what the tweet volume for the last few months looked like (please note that the dates are utc), check out these bar charts. january is especially fun.

tweets to @realdonaldtrump by day in october

tweets to @realdonaldtrump by day in november

tweets to @realdonaldtrump by day in december

tweets to @realdonaldtrump by day in january

- -

- - t : : + : lucidworks: consider a new application for ai in retail https://lucidworks.com/post/consider-a-new-application-for-ai-in-retail/

how companies can plan for by weaving ai and machine learning into their digital experiences.

the post consider a new application for ai in retail appeared first on lucidworks.

- - t : : + : garrett schwegler mita williams: weeknote ( ) https://librarian.aedileworks.com/ / / /weeknote- - /

hey. i missed last week’s weeknote. but we are here now.

this week i gave a class on searching scientific literature to a group of biology masters students. while i was making my slides comparing the advanced search capabilities of web of science and scopus, i discovered this weird behaviour of google scholar: a phrase search generated more hits than not.

i understand that google scholar performs ‘stemming’ instead of truncation in generating search results but this still makes no sense to me.

new to me: if you belong to an organization that is already a member of crossref, you are eligible to use a similarity check of documents for an additional fee. perhaps this is a service we could provide to our ojs editors.

i’m still working through the canadian journal of academic librarianship special issue on academic libraries and the irrational.

long time readers know that i have a fondness for the study of organizational culture and so it should not be too surprising that the first piece i wanted to read was the digital disease in academic libraries. it begins….

though several recent books and articles have been written about change and adaptation in contemporary academic libraries (mossop ; eden ; lewis ), there are few critical examinations of change practices at the organizational level. one example, from which this paper draws its title, is braden cannon’s ( ) the canadian disease, where the term disease is used to explore the trend of amalgamating libraries, archives, and museums into monolithic organizations. though it is centered on the impact of institutional convergence, cannon’s analysis uses an ethical lens to critique the bureaucratic absurdity of combined library-archive-museum structures. this article follows in cannon’s steps, using observations from organizational de-sign and management literature to critique a current trend in the strategic planning processes and structures of contemporary academic libraries. my target is our field’s ongoing obsession with digital transformation beyond the shift from paper-based to electronic resources, examined in a north american context and framed here as the digital disease.

i don’t want to spoil the article but i do want to include this zinger of a symptom which is the first of several:

if your library’s organizational chart highlights digital forms of existing functions, you might have the digital disease.
kris joseph, the digital disease in academic libraries, canadian journal of academic librarianship, vol ( )

ouch. that truth hurts almost as much as this tweet did:

- - t : : + : mita williams jez cope: blogging by hand https://erambler.co.uk/blog/blogging-by-hand/

i wrote the following text on my tablet with a stylus, which was an interesting experience:

so, thinking about ways to make writing fun again, what if i were to write some of them by hand? i mean i have a tablet with a pretty nice stylus, so maybe handwriting recognition could work. one major problem, of course, is that my handwriting is awful! i guess i’ll just have to see whether the ocr is good enough to cope…

it’s something i’ve been thinking about recently anyway: i enjoy writing with a proper fountain pen, so is there a way that i can have a smooth workflow to digitise handwritten text without just typing it back in by hand? that would probably be preferable to this, which actually seems to work quite well but does lead to my hand tensing up to properly control the stylus on the almost-frictionless glass screen.

i’m surprised how well it worked! here’s a sample of the original text:

and here’s the result of converting that to text with the built-in handwriting recognition in samsung notes:

writing blog posts by hand
so, thinking about ways to make writing fun again, what if i were to write some of chum by hand? i mean, i have a toldest winds a pretty nice stylus, so maybe handwriting recognition could work.
one major problems, ofcourse, is that my , is awful! iguess
i’ll just have to see whattime the ocu is good enough to cope…
it’s something i’ve hun tthinking about recently anyway: i enjoy wilting with a proper fountain pion, soischeme a way that i can have a smooch workflow to digitise handwritten text without just typing it back in by hand?
that wouldprobally be preferableto this, which actually scams to work quito wall but doers load to my hand tensing up to properly couldthe stylus once almost-frictionlessg lass scream.

it’s pretty good! it did require a fair bit of editing though, and i reckon we can do better with a model that’s properly trained on a large enough sample of my own handwriting.

- - t : : + : jez cope tara robertson: diversity, equity and inclusion core competencies: get cross functional projects done (part of ) https://tararobertson.ca/ /dei-core-competencies-get-projects-done/

latina woman writing on a poster board paper while black women offer ideas

this is the last post in a weeklong series exploring dei professional competencies. again, i believe the five key competencies for dei professionals are:

be strategic
translate academic research into action and measure the impact of initiatives
meet people where they are at and help them move to be more inclusive
influence others
get cross functional projects done.

yesterday’s post was about influencing others. this post will explore getting cross functional projects done. i’ll also share some other dei career resources.

great ideas without action are totally meaningless. as a dei leader you’ll be working across departments and functions to get stuff done. strong project management skills and collaboration are key in making change to existing processes and developing new ways of doing things. here’s two examples to illustrate this competency.

one of my first projects at mozilla was working with people ops and a tableau expert in it to build a dashboard to track our diversity metrics, which was more difficult and time consuming than i first thought. when i started the project was off the rails, so i suggested we restart by introducing ourselves, what we thought we brought to the table and then developed a rasci for the project. with these foundations in place we shifted us to be a very effective team. we completed the project and became friends. having a dashboard for diversity metrics was important as leaders owned accountability goals and needed to know how they were doing.

engineers started mozilla’s first mentorship program. i joined the team and was the only non-technical person and marvelled at some of the skills and ways of thinking that the others brought. it was one of those wonderful experiences where we were more than the sum of our parts. we were a small group of people with different backgrounds, doing different jobs, at various job levels and we were able to stand up and support a mentorship program for about people. i credit the leadership of melissa o’connor, senior manager of data operations. she often said “tell me what i’m missing here” to invite different options and ran the most efficient meetings i’ve ever attended in my life.

great ideas without action are totally meaningless. turning thoughts into actions as a leader in dei is a necessary art–to get things done you’ll need to effectively collaborate with people at different levels and in different functions.

additional resources on dei careers

i’m excited to be one of the panelists for andrea tatum’s dei careers panel tomorrow, january . the event is sold out but she’ll be simulcasting live on youtube at january at am pacific. andrea also introduced me to russell reynold’s list of competencies of a chief diversity officer.

aubrey blanche’s post how can i get a job in d&i? starts by trying to talk the reader out of going into this line of work then gets into five key areas of expertise.

dr. janice gassam’s dirty diversity podcast has an episode where she interviews lambert odeh, diversity and inclusion manager at olo inc. on how to land a career in dei.

the post diversity, equity and inclusion core competencies: get cross functional projects done (part of ) appeared first on tara robertson consulting.

- - t : : + : tara robertson open knowledge foundation: how to run your open data day event online in https://blog.okfn.org/ / / /how-to-run-your-open-data-day-event-online-in- /

for open data day on saturday th march, the open knowledge foundation is offering support and funding for in-person and online events anywhere in the world via our mini-grant scheme.

open data day normally sees thousands of people getting together at hundreds of events all over the world to celebrate and use open data in their communities but this year has not been a normal year.

with many countries still under lockdown or restricted conditions due to the covid- pandemic, we recognise that many people will need to celebrate open data day by hosting online events rather than getting together for in-person gatherings.

to support the running of events, anyone can apply to our mini-grant scheme to receive $ usd towards the running of your open data day event whether it takes place in-person or online. applications must be submitted before pm gmt on friday th february by filling out this form.

if you’re applying for a mini-grant for an online event, we will accept applications where the funds are allocated to cover any of the following costs:

fees for online tools needed to help with the running of your event
essential equipment needed to help with the running of your event
reimbursing speakers or participants for mobile data costs incurred during event
paying for the printing and posting of physical materials to event participants
other costs associated with running the event

it might feel challenging to plan a great online event if you are used to running events in the real world. but many people and organisations have overcome these challenges this year, and there are many tools that can help you plan your event. here are some tips and tools that we use for remote events that we think will help with your preparations.

open knowledge foundation is a remote working organisation with our team spread around the world. we use zoom, google meet or slack to host our internal and external video meetings and rely on google docs, github, gitter and discourse to allow us to share documents and talk in real-time. many of these tools are free and easy to set up.

two members of our team are also on the organisation team of csv,conf, an annual community conference for data makers which usually hosts several hundred people for a two-day event. for csv,conf,v in may , the team decided to make their event online-only and it proved to be a great success thanks to lots of planning and the use of good online tools. read this post – https://csvconf.com/ /going-online – to learn more about how the team organised their first virtual conference including guidance about the pros and cons of using tools like crowdcast, zenodo, zoom and spatial chat for public events.

other organisations – including the center for scientific collaboration and community engagement and the mozilla festival team – have also shared their guidebooks and processes for planning virtual events.

we hope some of these resources will help you in your planning. if you have any further questions relating to an open data day mini-grant application, please email opendataday@okfn.org.

- - t : : + : stephen abbott pugh ed summers: trump's tweets https://inkdroid.org/ / / /trumps-tweets/

tldr: trump’s tweets are gone from twitter.com but still exist spectrally in various states all over the web. after profiting off of their distribution twitter now have a responsibility to provide meaningful access to the trump tweets as a read only archive.

this post is also published on the documenting the now medium where you can comment, if the mood takes you.

so trump’s twitter account is gone. finally. it’s strange to have had to wait until the waning days of his presidency to achieve this very small and simple act of holding him accountable to twitter’s community guidelines…just like any other user of the platform.

better late than never, especially since his misinformation and lies can continue to spread after he has left office.

but isn’t it painful to imagine what the last four years (or more) could have looked like if twitter and the media at large had recognized their responsibility and acted sooner?

when twitter suspended trump’s account they didn’t simply freeze it and prevent him from sending more hateful messages. they flipped a switch that made all the tweets he has ever sent disappear from the web.

these are tweets that had real material consequences in the world. as despicable as trump’s utterances have been, a complete and authentic record of them having existed is important for the history books, and for holding him to account.

twitter’s suspension of donald trump’s account has also removed all of his thousands of tweets sent over the years. i personally find it useful as a reporter to be able to search through his tweets. they are an important part of the historical record. where do they live now?
— olivia nuzzi (@olivianuzzi) january ,

where indeed? one hopes that they will end up in the national archives (more on that in a moment). but depending on how you look at it, they are everywhere.

twitter removed trump’s tweets from public view at twitter.com. but fortunately, as shawn jones notes, embedded tweets like the one above persist the tweet text into the html document itself. when a tweet is deleted from twitter.com the text stays behind elsewhere on the web like a residue, as evidence (that can be faked) of what was said and when.

it’s difficult to say whether this graceful degradation was an intentional design decision to make their content more resilient, or it was simply a function of twitter wanting their content to begin rendering before their javascript had loaded and had a chance to emboss the page. but design intent isn’t really what matters here.

what does matter is the way this form of social media content degrades in the web commons. kari kraus calls this process “spectral wear”, where digital media “help mitigate privacy and surveillance concerns through figurative rather than quantitative displays, reflect and document patterns of use, and promote an ethics of care.” (kraus, ). this spectral wear is a direct result of tweet embed practices that twitter itself promulgates while simultaneously forbidding it developer terms of service:

if twitter content is deleted, gains protected status, or is otherwise suspended, withheld, modified, or removed from the twitter applications (including removal of location information), you will make all reasonable efforts to delete or modify such twitter content (as applicable) as soon as possible.

fortunately for history there has probably never been a more heavily copied social media content than donald trump’s tweets. we aren’t immediately dependent on twitter.com to make this content available because of the other other places on the web where it exists. what does this copying activity look like?

i intentionally used copied instead of archived above because the various representations of trump’s tweets vary in terms of their coverage, and how they are being cared for.

given their complicity in bringing trump’s messages of division and hatred to a worldwide audience, while profiting off of them, twitter now have a responsibility to provide as best a representation of this record for the public, and for history.

we know that the trump administration have been collecting the @realdonaldtrump twitter account, and plan to make it available on the web as part of their responsibilities under the presidential records act:

the national archives will receive, preserve, and provide public access to all official trump administration social media content, including deleted posts from @realdonaldtrump and @potus. the white house has been using an archiving tool that captures and preserves all content, in accordance with the presidential records act and in consultation with national archives officials. these records will be turned over to the national archives beginning on january , , and the president’s accounts will then be made available online at nara’s newly established trumplibrary.gov website.

nara is the logical place for these records to go. but it is unclear what shape these archival records will take. sure the library of congress has (or had) it’s twitter archive. it’s not at all clear if they are still adding to it. but even if they are lc probably hasn’t felt obligated to collect the records of an official from the executive branch, since they are firmly lodged in the legislative. then again they collect gifs so, maybe?

reading between the lines it appears that a third party service is being used to collect the social media content: possibly one of the several e-discovery tools like archivesocial or hanzo. it also looks like the trump administration themselves have entered into this contract, and at the end of its term (i.e. now) will extract their data and deliver it to nara. given their past behavior it’s not difficult to imagine the trump administration not living up to this agreement in substantial ways.

this current process is a slight departure from the approach taken by the obama administration. obama initiated a process where platforms [migrate official accounts] to new accounts that were then managed going forward by nara (acker & kriesberg, ). we can see that this practice is being used again on january , when biden became president. but what is different is that barack obama retained ownership of his personal account @barackobama, which he continues to use.nara has announced that they will be archiving trump’s now deleted (or hidden) personal account.

a number of trump administration officials, including president trump, used personal accounts when conducting government business. the national archives will make the social media content from those designated accounts publicly available as soon as possible.

the question remains, what representation should be used, and what is twitter’s role in providing it?

meanwhile there are online collections like the trump archive, the new york times’ complete list of trump’s twitter insults, propublica’s politwoops and countless github repositories of data which have collected trump’s tweets. these tweets are used in a multitude of ways including things as absurd as a source for conducting trades on the stock market.

but seeing these tweets as they appeared in the browser, with associated metrics and comments is important. of course you can go view the account in the wayback machine and browse around. but what if we wanted a list of all the trump tweets? how many times were these tweets actually archived? how complete is the list?

after some experiments with the internet archive’s api it’s possible to get a picture of how the tweets from the @realdonaldtrump account have been archived there. there are a few wrinkles because a given tweet can have many different url forms (e.g. tracking parameters in the url query string). in addition just because there was a request to archive a url for something that looks like a realdonaldtrump tweet url doesn’t mean it resulted in a successful response. success here means a ok from twitter.com when resolving the url. factoring these issues into the analysis it appears the wayback machine contains (at least) , , snapshots of trump’s tweets.

https://twittter.com/realdonaldtrump/status/{tweet-id}

of these millions of snapshots there appear to be , unique tweets. this roughly correlates with the k total tweets suggested by the last profile snapshots of the account. the maximum number of times in one day that his tweets were archived was , times on february , . here’s what the archive snapshots of trump’s tweets look like over time (snapshots per week).

it is relatively easy to use the csv export from the [trump archive] project to see what tweets they know about that the internet archive does not and vice-versa (for the details see the jupyter notebook and sqlite database here).

it looks like there are tweet ids in the trump archive that are missing from the internet archive. but further examination shows that many of these are retweets, which in twitter’s web interface, have sometimes redirected back to the original tweet. removing these retweets to specifically look at trump’s own tweets there are only tweets in the trump archive that are missing from the internet archive. of these are in fact retweets that have been miscategorized by the trump archive.

one of the three is this one, which is identified in the trump archive as deleted, and wasn’t collected quick enough by the internet archive before it was deleted:

roger stone was targeted by an illegal witch hunt tha never should have taken place. it is the other side that are criminals, including the fact that biden and obama illegally spied on my campaign - and got caught!"

sure enough, over at the politwoops project you can see that this tweet was deleted seconds after it was sent:

flipping the table it’s also possible to look at what tweets are in the internet archive but not in the trump archive. it turns out that there are , tweet identifiers in the wayback machine for trump’s tweets which do not appear in the trump archive. looking a bit closer we can see that some are clearly wrong, because the id itself is too small a number, or too large. and then looking at some of the snapshots it appears that they often don’t resolve, and simply display a “something went wrong” message:

yes, something definitely went wrong (in more ways than one). just spot checking a few there also appear to be some legit tweets in the wayback that are not in the trump archive like this one:

notice how the media will not play there? it would take some heavy manual curation work to sort through these tweet ids to see which ones are legit, and which ones aren’t. but if you are interested here’s an editable google sheet.

finally, here is a list of the top ten archived (at internet archive) tweets. the counts here reflect all the variations for a given tweet url. so they will very likely not match the count you see in the wayback machine, which is for the specific url (no query paramters).

the point of this rambling data spelunking, if you’ve made it this far, is to highlight the degree to which trump’s tweets have been archived (or collected), and how the completeness and quality of those representations is very fluid and difficult to ascertain. hopefully twitter is working with nara to provide as complete a picture as possible of what trump said on twitter. as much as we would like to forget, we must not.

references

acker, a., & kriesberg, a. ( ). tweets may be archived: civic engagement, digital preservation and obama white house social media data. proceedings of the association for information science and technology, ( ), – .

kraus, k. ( ). the care of enchanted things. in m. k. gold & l. f. klein (eds.), debates in the digital humanities . retrieved from https://www.jstor.org/stable/ . /j.ctvg hk.

- - t : : + : lucidworks: lucidworks announces updated integration with drupal https://lucidworks.com/post/updated-integration-with-drupal/

updated fusion integration with drupal provides easier setup and additional monitoring.

the post lucidworks announces updated integration with drupal appeared first on lucidworks.

- - t : : + : tom allen virtual ndsa digital preservation recordings available online! - dlf clir fellowships & grants dlf publications clir global join give clir programs digital library federation dlf forum digital library of the middle east digitizing hidden special collections and archives recordings at risk mellon fellowships for dissertation research leading change institute dlf eresearch network postdoctoral fellowship program dlf digital library federation toggle navigation about about dlf our staff our members governance dlf code of conduct events dlf year-round forum: online past forums dlf forum news social events checklist child care fund resources dlf organizers’ toolkit blog and news #dlfcontribute series digitizing special formats dlf cost calculator dlf jobs board groups dlf working groups clir/dlf affiliates dlf membership cohorts get involved with groups opportunities grants and fellowships authenticity project dlf community calendar data curation postdocs community/capacity awards post a job/find a job contact get in touch stay connected join dlf our members benefits dlf digital library federation toggle navigation about about dlf our staff our members governance dlf code of conduct events dlf year-round forum: online past forums dlf forum news social events checklist child care fund resources dlf organizers’ toolkit blog and news #dlfcontribute series digitizing special formats dlf cost calculator dlf jobs board groups dlf working groups clir/dlf affiliates dlf membership cohorts get involved with groups opportunities grants and fellowships authenticity project dlf community calendar data curation postdocs community/capacity awards post a job/find a job contact get in touch stay connected join dlf our members benefits virtual ndsa digital preservation recordings available online! january , kussmann ndsa session recordings from the virtual ndsa digital preservation conference are now available on ndsa’s youtube channel, as well as on aviary. the full program from digital preservation : get active with digital preservation, which took place online november , , is free and open to the public. ndsa is an affiliate of the digital library federation (dlf) and the council on library and information resources (clir). each year, ndsa’s annual digital preservation conference is held alongside the dlf forum and acts as a crucial venue for intellectual exchange, community-building, development of good practices, and national agenda-setting for digital stewardship. enjoy, tricia patterson; digipres vice-chair, chair related conference digipres ndsa previous next what's the dlf? networked member institutions and a robust community of practice—advancing research, learning, social justice, & the public good through the creative design and wise application of digital library technologies contact clir+dlf north union street suite -pmb alexandria, va e: info@diglib.org elsewhere twitter facebook linkedin youtube email rss community calendar geo lib camp february , – february , online https://www.eventbrite.com/e/geo libcamp- -tickets- nycdh week february , – february , https://nycdh.org/dhweek/ college art association annual conference february , – february , online https://www.collegeart.org/programs/conference/conference more events & links here. on twitter rt @clirnews: celebrating #archiveshashtagparty #archivesblackeducation and #blackhistorymonth with "our story: photographs and publication…yesterday have a spare minute? make sure to complete our question survey about attending fall events today! https://t.co/h mxokpxqz days ago rt @lellyjz: really looking forward to this! https://t.co/ o gkknzqq days ago unless otherwise indicated, content on this site is available for re-use under cc by-sa . license clir skip to content open toolbar accessibility tools increase text decrease text grayscale high contrast negative contrast light background links underline readable font reset planet code lib search the bloggers of planet code lib using google custom search. utf- code lib library peter@ohiolink.edu data:image/png;base ,ivborw kggoaaaansuheugaaabaaaaaqcamaaaaolq taaaaaxnsr iars c qaaaarnqu baacxjwv yquaaaagy hstqaaeiyaaiceaad aaaagogaahuwaadqyaaaopgaabdwnlprpaaaawbqtfrf////s ozndq skpkioqkbgxsoaghe t kskp enpwfhyhr dzmzml exymjivr + /pzpj + traaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaavwdikqaaaalwsflzaaalewaacxmbajqcgaaaai jrefukfnvtlscwyaie smldsv+x wlup svmisnlgqlgttcui/gtmgli osdkmuzfbagq/cxdqo zejwmmo aihimjmqweuydolwehplpc a kx jlv fc za lvfu helmotva+tyfvk/ro zz zkplb q b pyi lpafc bw atdy l + /rhquihczntwaaaabjru erkjggg== false http://planet.code lib.org wayback machine see what's new with book lending at the internet archive a line drawing of the internet archive headquarters building façade. an illustration of a magnifying glass. an illustration of a magnifying glass. an illustration of a horizontal line over an up pointing arrow. upload an illustration of a person's head and chest. sign up | log in an illustration of a computer application window wayback machine an illustration of an open book. books an illustration of two cells of a film strip. video an illustration of an audio speaker. audio an illustration of a . " floppy disk. software an illustration of two photographs. images an illustration of a heart shape donate an illustration of text ellipses. more an icon used to represent a menu that can be toggled by interacting with this icon. about blog projects help donate an illustration of a heart shape contact jobs volunteer people search metadata search text contents search tv news captions search archived websites advanced search sign up for free log in the wayback machine requires your browser to support javascript, please email info@archive.org if you have any questions about this. the wayback machine is an initiative of the internet archive, a (c)( ) non-profit, building a digital library of internet sites and other cultural artifacts in digital form. other projects include open library & archive-it.org. your use of the wayback machine is subject to the internet archive's terms of use. digital library federation - dlf clir fellowships & grants dlf publications clir global join give clir programs digital library federation dlf forum digital library of the middle east digitizing hidden special collections and archives recordings at risk mellon fellowships for dissertation research leading change institute dlf eresearch network postdoctoral fellowship program dlf digital library federation toggle navigation about about dlf our staff our members governance dlf code of conduct events dlf year-round forum: online past forums dlf forum news social events checklist child care fund resources dlf organizers’ toolkit blog and news #dlfcontribute series digitizing special formats dlf cost calculator dlf jobs board groups dlf working groups clir/dlf affiliates dlf membership cohorts get involved with groups opportunities grants and fellowships authenticity project dlf community calendar data curation postdocs community/capacity awards post a job/find a job contact get in touch stay connected join dlf our members benefits dlf digital library federation toggle navigation about about dlf our staff our members governance dlf code of conduct events dlf year-round forum: online past forums dlf forum news social events checklist child care fund resources dlf organizers’ toolkit blog and news #dlfcontribute series digitizing special formats dlf cost calculator dlf jobs board groups dlf working groups clir/dlf affiliates dlf membership cohorts get involved with groups opportunities grants and fellowships authenticity project dlf community calendar data curation postdocs community/capacity awards post a job/find a job contact get in touch stay connected join dlf our members benefits thanks for joining us! learn more and apply miss a great talk? we've got you covered. dlf forum plenary recordings forum child care fund you can make a difference. please give. more than just a conference. find your people all year long. join the dlf today meet the authenticity project participants! volunteers and fellows have been matched in yearlong partnerships. learn more what’s the dlf? networked member institutions and a robust community of practice—advancing research, learning, social justice, & the public good through the creative design and wise application of digital library technologies dlf as community find your people, year-round. grassroots, pragmatic, and mission-driven, dlf is a space where ideas are road-tested and shared strategies and visions emerge. dlf as platform get things done. we foster active, open, and welcoming working groups dedicated to building better libraries, museums, and archives for the digital age. dlf as crossroads meet up. our annual dlf forum serves as meeting place, marketplace, and congress for diglib practitioners from member institutions and the community at large. read more selected posts what’s new on the dlf blog cfps are here! dlf forum and affiliated events dlf forum community journalist reflection: lisa covington three questions on irus-usa dlf forum and fall events move online metadata during covid february , gayle assessment, community this post was written by members of the metadata working group, a subgroup of dlf’s assessment interest group. digital collections work has changed in a number of ways during the covid- pandemic. for many libraries and archives, this has meant... more three new ndsa members january , nathan tallman ndsa since january , the ndsa coordinating committee unanimously voted to welcome three new members. each of these members bring a host of skills and experience to our group. please help us to welcome: arkivum: arkivum is recognized internationally for... more virtual ndsa digital preservation recordings available online! january , kussmann ndsa session recordings from the virtual ndsa digital preservation conference are now available on ndsa’s youtube channel, as well as on aviary. the full program from digital preservation : get active with digital preservation, which took place... more more posts jobs working groups ndsa organizers' toolkit community calendar dlf contribute dlf events grants & fellowships digitization cost calculator contact clir+dlf north union street suite -pmb alexandria, va e: info@diglib.org elsewhere twitter facebook linkedin youtube email rss community calendar geo lib camp february , – february , online https://www.eventbrite.com/e/geo libcamp- -tickets- nycdh week february , – february , https://nycdh.org/dhweek/ college art association annual conference february , – february , online https://www.collegeart.org/programs/conference/conference more events & links here. on twitter rt @clirnews: celebrating #archiveshashtagparty #archivesblackeducation and #blackhistorymonth with "our story: photographs and publication…yesterday have a spare minute? make sure to complete our question survey about attending fall events today! https://t.co/h mxokpxqz days ago rt @lellyjz: really looking forward to this! https://t.co/ o gkknzqq days ago unless otherwise indicated, content on this site is available for re-use under cc by-sa . license clir skip to content open toolbar accessibility tools increase text decrease text grayscale high contrast negative contrast light background links underline readable font reset wayback machine see what's new with book lending at the internet archive a line drawing of the internet archive headquarters building façade. an illustration of a magnifying glass. an illustration of a magnifying glass. an illustration of a horizontal line over an up pointing arrow. upload an illustration of a person's head and chest. sign up | log in an illustration of a computer application window wayback machine an illustration of an open book. books an illustration of two cells of a film strip. video an illustration of an audio speaker. audio an illustration of a . " floppy disk. software an illustration of two photographs. images an illustration of a heart shape donate an illustration of text ellipses. more an icon used to represent a menu that can be toggled by interacting with this icon. about blog projects help donate an illustration of a heart shape contact jobs volunteer people search metadata search text contents search tv news captions search archived websites advanced search sign up for free log in the wayback machine requires your browser to support javascript, please email info@archive.org if you have any questions about this. the wayback machine is an initiative of the internet archive, a (c)( ) non-profit, building a digital library of internet sites and other cultural artifacts in digital form. other projects include open library & archive-it.org. your use of the wayback machine is subject to the internet archive's terms of use. podcast | drjanicegassam janice z. gassam asare, ph.d. driving diversity increasing inclusion building bridges log in home about services clients podcast contact products shop more listen on apple podcasts listen on spotify listen on stitcher have you downloaded the dirty diversity podcast yet? what are you waiting for? i don't want you to miss a single episode. are you listening on apple podcasts? it would mean the world to me if you left a review so that other people can find the dirty diversity podcast. share with your friends, family and colleagues. contact if you would like to enlist my services for your organization, conference, institution or university, please fill out this form or contact me directly email: janice@bwgbusinesssolutions.com. tel: - - name * email * subject message success! message received. send © by janice z gassam asare. all rights reserved. none product manager vs product marketing manager vs product owner navigate back to the homepagecopied product manager vs product marketing manager vs product owner damilola ajiboye, november rd, · min read copied this is a question that often arises in the product world and it requires our time to unpack it. though these roles are somewhat similar and they work towards achieving the same set of goals, we then ought to look at the clear line distinction in the line of duty of the individuals and performance metric used to evaluate the success achieved by each role to help us understand them better and also help you figure out the most suitable for you. these roles vary from company to company as in the case of a startup, a product manager is often tasked with the duty of these roles while in larger or enterprise companies, there are or more different roles in the product team working at different capacities to ensure that the right product is shipped and meet the demands of users. while the product manager’s role is one that has come to limelight, the product marketing manager and product owner role have also been adopted by a lot of companies depending on their product and company culture. in this article, we will explain the difference between these roles and the job requirement of the individual involved. the product manager these individuals are often referred to as mini ceos of a product. they conduct customer surveys to figure out the customer’s pain and build solutions to address it. the pm also prioritizes what features are to be built next and prepares and manages a cohesive and digital product roadmap and strategy. the product marketing manager the pmm communicates vital product value — the “why”, “what” and “when” of a product to intending buyers. he manages the go-to-market strategy/roadmap and also oversees the pricing model of the product. the primary goal of a pmm is to create demand for the products through effective messaging and marketing programs so that the product has a shorter sales cycle and higher revenue. the product owner this role exists in a scrum environment — scrum is a framework for project management that emphasizes teamwork, accountability, and iterative progress toward a well-defined goal. a product owner (po) maximizes the value of a product through the creation and management of the product backlog, creation of user stories for the development team. the product owner is the customer’s representative to the development team. he addresses customer’s pain points by managing and prioritizing a visible product backlog. the po is the first point of call when the development team needs clarity about interpreting a product feature to be implemented. going more practical, we will use a hypothetical case study of a tech company, slyde, to explain these roles to clearly understand their day-to-day duties. slyde is a financial technology company based in lagos, nigeria with coverage in other african countries. the company has been providing payment solutions through its web-based application but has noticed a large churn rate in areas with poor internet connection which in turn delay users from completing their transaction. the product team made this observation and has been tasked to come up with a solution. where does the product manager, product marketing manager, and the product owner come in? pm: the product manager will interface with the users through user interviews/feedback surveys or other means to hear directly from the users. they will come up with hypotheses alongside the team and validate them through prototyping and user testing. they will then create a strategy on the feature and align the team and stakeholders around it. the pm who is also the chief custodian of the entire product roadmap will, therefore, be tasked with the duty of prioritization. before going ahead to carry out research and strategy, they will have to convince the stakeholders if it is a good choice to build the feature in context at that particular time or wait a bit longer based on the content of the roadmap. pmm: the product marketing manager is tasked with market feasibility and discovering if the features being built align with the company’s sales and revenue plan for the period. they also make research on how sought-after the feature is being anticipated and how it will impact the budget. they communicate the values of the feature; the why, what, and when to potential buyers — in this case users in countries with poor internet connection. po: the product owner will first have to prioritize the backlog to see if there are no important tasks to be executed and if this new feature is worth leaving whatever is being built currently. they will also consider the development effort required to build the feature i.e the time, tools, and skill set that will be required. they will be the one to tell if the expertise of the current developers is enough or if more engineers or designers are needed to be able to deliver at the scheduled time. the product owner is also armed with the task of interpreting the product/feature requirements for the development team. they serve as the interface between the stakeholders and the development team. lastly, the goal of a product team is to delight its user by providing an excellent solution to their pain points regardless of the job title/role. irrespective of your job role on a product team, you should always be driven by user empathy and the company’s goal as this will in-turn make you collaborate better among the team. a product person should first and foremost see themselves as a product leader — one who makes sure the user’s need is always advocated for, despite top executive declination, the product person is to persuasively align the executive to this. product owner is a role you play on a scrum team. product manager is the job. — melissa perri so, regardless of your job role, a great product person will always act in the stead of a product manager and never streamline herself to the jd only but be actively involved directly or indirectly in every phase of the product development cycle. when the role is clearly understood and the duties implemented collaboratively as a team, every aspect of the product comes together in one beautiful piece. this article first appeared on my medium thanks to olufisayo babalola for reviewing this article. sources: hero image by airfocus on unsplash share: share the selected text share the selected text musings on building user-centric products directly in your inbox monthly get notified of new posts as they get published ✨ subscribe more articles from damilolaa product-led growth - what it is and how you can leverage it product led growth (plg) is a go-to-market strategy that relies on product usage as the primary driver of acquisition, conversion and expansion. january th, · min read product manager vs product marketing manager vs product owner product owner is a role you play on a scrum team. product manager is the job. — melissa perri. november rd, · min read © damilolaa | about me | contact link to $https://twitter.com/steve_dammylink to $https://github.com/mcsteve link to $https://www.linkedin.com/in/damilola-ajiboye/link to $https://dribbble.com/damilolaa samvera samvera season’s greetings from the samvera community as draws to a close, we want to share our gratitude for each and every person who has helped the samvera community thrive in this difficult year of profound loss and challenge. we could not be the vibrant, welcoming, and valuable open source solution community we are without each and every person who contributed... read more &# ; the post season&# ;s greetings from the samvera community appeared first on samvera. save the date for samvera virtual connect mark your calendar for samvera virtual connect !tuesday, april &# ; wednesday, april , : am &# ; : pm edt / : am – : am pdt / : - : bst / : - : utc watch for more information coming in early including a call for program committee participation and a call for proposals. the post save the date for samvera virtual connect appeared first on samvera. developer resources: bug hunting in hyrax; adding blacklight advanced search to hyku bess sadler from notch has created two excellent guides that may be helpful to developers working in hyrax or hyku applications: bug hunting in hyrax: a well-documented process for finding a bug in a hyrax application adding blacklight_advanced_search to hyku: a how-to guide for adding blacklight advanced search to a hyku application have you or... read more &# ; the post developer resources: bug hunting in hyrax; adding blacklight advanced search to hyku appeared first on samvera. fedora alpha release available for download and testing fedora . alpha- is now available for download and testing. the primary goals for fedora are robust migration support, enhanced digital preservations features, and improved performance and scale. the fedora team will ask the samvera community for testing assistance when the full version is available in early . in the meantime, you can learn... read more &# ; the post fedora alpha release available for download and testing appeared first on samvera. samvera tech : a beginner-friendly overview of samvera samvera connect on-line included an excellent, beginner-friendly overview of “samvera tech ” presented by alisha evans and shana moore, software engineers at notch . evans has turned this presentation into a blog post walking through the technologies used in the samvera community. check out the post on the notch blog: samvera tech the post... read more &# ; the post samvera tech : a beginner-friendly overview of samvera appeared first on samvera. samvera connect presentations if you are not one of the + people who registered to enjoy samvera connect on-line, or if you are but missed a session you wanted to see, links to the recordings and session slide packs are being added to the samvera wiki here. our grateful thanks to all the organizers and speakers who... read more &# ; the post samvera connect presentations appeared first on samvera. things are already happening for samvera connect! samvera connect ‘proper’ is a little over a week away but the poster exhibition is now available here! there is a slack channel #connect-posters for asynchronous comment or discussion and each presenter has a minute video conferencing slot monday th – wednesday st october for live discussion. see sched for details. need a slack... read more &# ; the post things are already happening for samvera connect! appeared first on samvera. samvera connect on-line is nearly here! samvera&# ;s annual connect conference has gone virtual this year, like so many others. nevertheless, we&# ;ve put together an exciting program of workshops, presentations, posters and community social events that we hope will make up for not being able to meet in person. the main events are on friday / and monday &# ; thursday / &# ;... read more &# ; the post samvera connect on-line is nearly here! appeared first on samvera. connect on-line calls for proposals open the program committee for samvera connect on-line is pleased to announce its call for proposals (cfp) of workshops, presentations and lightning talks. as in the past, our goal is to serve the needs of attendees from an ever-widening range of experience and background (potential adopters, new adopters, expert samverans; developers, managers, sysops, metadata librarians,... read more &# ; the post connect on-line calls for proposals open appeared first on samvera. samvera community manager samvera is hiring for its inaugural community manager. we are seeking a highly organized individual who wants to join this grass-roots, open source community that creates best in class repository solutions for digital content stewarded by libraries, archives, and museums. we are a vibrant and welcoming community of information and technology professionals who share challenges,... read more &# ; the post samvera community manager appeared first on samvera. mark e. phillips journal mark e. phillips journal menu skip to content home about metadata events system: accounting for time leave a reply in the last post, i mentioned that there four primary things that we have needed in order to move a large portion of our student and staff workers to full remote work during this quarantine. in this post, i wanted to jump ahead a bit and talk a bit about how we are accounting for remote workers’ time. this is one of the things that the university wrestled with when thinking about moving students offline, how would we be able to account for time. well, one of the ways we are trying to track time is by using the activity in the metadata editing system to help managers understand what their workers are working on. over the past two weeks, the software development unit at the unt libraries has been working hard to push out quite a few changes to a system we call events. this system has been sitting in the background of our metadata editing infrastructure for a number of years collecting what we call “edit events”. an edit event is logged in the event system and contains information about the username, which record they edited, the timestamp for when it was edited, and how long the metadata edit window was open while they were doing metadata. this is then aggregated for users in a dashboard that they can view. for a few years now this system has been unusable because of some code that needed to be refactored now that we have almost . million edit events. that’s what we have been working on for the past few weeks. the first thing to note for users of our edit system is “how do you get to the events pages”. well in the upper left corner of the screen, if you click on the “home” icon you will see an events option in the dropdown. getting to edit events from edit search dashboard this will take you to the events landing page, where you are greeted with an overview of what is going on with the events system. all data is divided into today, this month, and all time and gives you statistics for the number of edits, the number of active users, and the number of unique items that have been edited. clicking on the different buttons will take you to different places. i am going to walk through by clicking on any of the blue buttons for today. edit events overall stats by clicking on any of the today buttons, you are presented with statistics for the edit events that have happened in the system today. you can get an overview of everything that has happened in the system. you can also navigate to different days by clicking on the previous day button. daily stats view for march , at the top of the page, you will see an overview of stats for the day. this includes the number of edits, the number of unique records edited, and how much aggregate time has been spent during the day editing. we also show the first and most recent edit, the number of users that have edited during the day, and then we list the user who has the most edits along with their edit count. events daily stats detail next up is a view of the activity by hour section. i have been amazed to see the time of day when users are editing records. for example, on saturday the th there were users who edited records in of the hours during the day. events hourly stats detail below that block is the activity by user for the day. you can see a listing of all of the users who have edited during the day as well as some statistics related to their activity including the number of edits, number of records, total editing duration and the average time per edit, and the average time per record. clicking on any of the names will take you to an overview of that user’s activity. daily activity by user a user’s page gives information going back a little over a month so that managers can easily verify the time for pay periods that are either every two weeks or one month. in addition to an overview of all of the user’s activity in the events system, you can see a breakdown of what days they have edited records as well as statistics about what activity they completed during that day. by clicking on the day link you can see information specific to that day for that user. user daily activity view the user hourly detail view presents statistics for a user on a specific day. it includes similar information as the overview pages mentioned previously. there is also an hourly table that shows when the user was editing records, including how many edits, hours, duration and average time per edit and record for a given hour. user hourly detail view below the hourly breakdown of activity, you see all of the edits performed by the user on that day. you can link directly to the edit event or you can view information about the record that they have edited. below you will see the detail for a record in the events system. you see how many total edits have taken place with a record including when, and which user performed the edits. there is a link on the page to view the records summary in the edit system to see more information about the record that was edited. record activity view when you click on that link you are taken to the record summary page in the edit interface. record in edit system if you want to dig deeper into what happened with the record, you can click on the view history link and view different versions of the record to see what changes were made. metadata history page there are two other views in the edit events interface that can be useful. if you had clicked on the orange users button on the edit landing page you would see a list of all of the users who have been active during the current week (starting sunday). user activity for this week if you click on the orange items button on the events landing page you get to a view that shows a listing of the records that were edited this week. it also includes the number of edits and the duration of editing for that week. record activity for this week we are hoping that the improved events pages will be useful for managers as they begin to review timesheets for students that they supervise. i know that i have been pleasantly surprised by the data when i can view the number of edits we are getting at all hours of the day. i think it shows the opportunities that we have with our digital library systems for providing engaging, meaningful work to a wide range of users during this quarantine. i skipped forward in my list of components we have needed for this process. over the next blog posts, i will go back and pick up where i left off describing the infrastructure we have in place to communicate instructions and documentation, and finally how we are identifying work that needs to be done in the system. if you have questions or comments about this post, please let me know via twitter. this entry was posted in thinking outloud and tagged covid- , metadata editing on march , by vphill. managing metadata editing for telecommuting. leave a reply many of us in the us, and around the world for that matter, are now sitting at home working remotely trying to maintain some semblance of normalcy during quarantine for covid- . i’m going to write up a few of the things that we have been working on her at the unt libraries to try and provide as many of the library employees, including student employees, with activities that they can do remotely. on march th it became quite clear that there was going to be a large number of folks from the library needing to work remotely, many of us have activities that we can do remotely, and some of us even prefer to work remotely when we have the opportunity. there are however a large group of people at the library that don’t have jobs that directly transfer to working remotely, or in the situation where there aren’t students on campus to directly serve, the work that they would be doing isn’t available for them. we wanted to provide an option for these individuals to create metadata for the unt libraries digital collections if they were interested in doing so. additionally, this would give supervisors some meaningful activity that could be verified in the event that we had to move the workforce remotely because of this pandemic. what we needed there were a few things that we needed to have in place in order to move people online to edit metadata records. web-based system for editing metadata clear instructions and guides a way of identifying work to be done a way of coordinating activity a way of verifying/tracking activity for timekeeping web-based editing system. there were a number of things that we have in place that allow us to move a large number of people into the metadata creation workflow. first, we have been using a web-based metadata editing system for the entire time that we have had our digital collections. here at unt libraries we creatively call this system edit. this system is built around the metadata format that we have been using locally called untl. we can add a user in the system, assign them to a subset of the collection, usually based on collections, and give them permission to begin editing metadata. we have a system in place to register new users in batches and then invite them to join the editing system. in addition to our own work here in the libraries, we have made use of this metadata editing system for a number of metadata classes of library school students to give them real-world experience writing metadata records in a production system. because of these previous experiences, the task of creating - new accounts for users in the library wasn’t too daunting. for setting up user accounts we make use of a django app that we wrote a few years ago called django_invite (https://github.com/unt-libraries/django-invite). generally, the workflow we follow for new accounts is that we are given one or more new users who need accounts. the information we need is just a name, an email address, and the scope of the collection they need access to. we can enter multiple names at once if they have the same permissions. the permissions for this app are based on the standard django permission and group concepts. django invite once you submit users, the system sends out an invitation email for the user to complete the registration process by picking a name. we are able to see who has established their account and if needed, resend the invitation email. this process makes it fairly straightforward to get people set up in the system. editing records as i said, we have been using a web-based editing system for the unt libraries digital collections for over a decade now. the editing system (edit) starts a user with a view of all of the records that they can access based on their permissions. we call this view the “search dashboard”. edit system: search dashboard from here a user can search the metadata for a record, sort results, and limit their result sets based on any of the facets on the left-hand side of the screen. for those interested, the facets include. system collections partners resource type visibility (hidden or not) date validity (valid edtf dates) my edits (records you have edited) record completeness location data (with/without placenames, geocodes, or bounding boxes) recently edited records (last h, h, d, d, d, d, d) edit system: limiting to collection from there a user gets back their results where they can be further refined by sorting. edit system: limited collection the current sort options include: title (default) date added (newest/oldest creation date (newest/oldest) date last modified (newest/oldest) ark identifier (lowest/highest) completeness (lowest/highest) they are also able to see basic information about the object including a thumbnail, the system, collections, and partner in which the item belongs. they can see the accession date and the last date that the item was edited. finally, they can see a visibility flag, green for visible to the public and a red check for not visible. they can choose to go to one of two places for a record, the edit view or the summary view. i will start with an overview of the summary view. edit system: item summary the summary view provides an overview of the record including a compact version of the record itself. we provide an edit timeline to get a better sense of when the item has been edited, and when it became available online. additionally, it has links and other information that are helpful for metadata creators. i will walk through a few of those links now. first up is the view item screen. edit system: view item this view is for the metadata creator to interact with the object itself, they are able to see all of the pages of the item and zoom in to look at the details. for audio and video, they have the ability to view the item in a player as well as download the media files as needed. we also have the ability to see the history of edits that have occurred for a record. this history page presents information about who, when, and a high-level overview of what has happened to the item over time. you might notice a number of edits for this record. one of the things we have noticed in our metadata editing practice is that we tend to take a “column” approach to the editing of records instead of a “row” approach. we will find an issue, maybe an incorrectly formatted name, and fix all of those instances in the system. this results in many edits per record but allows editors to focus on a single task. as you can see, all of the record edits are versioned so it is possible to go back and view what was changed and by whom. edit system: item history the final view is the metadata editor itself. i’ve mentioned this in a number of other posts over time so i won’t go into too much detail here. basically all of the work of editing gets done here. users can add, subtract, reorder, and edit elements. most elements have qualifiers to designate the type of element being used such as the main title, added title, serial title, or series title for the title element. some element such as creator, contributor, and publisher have a type ahead that pull from our name authority system (unt names) and include information about the type of name (personal/organization), the role (author, photographer, editor) and an info field for other bits of info about the agent. all dropdown values are pulled from a centralized vocabulary management system. some of the fields have popup modals for controlled vocabularies, picking locations from a map, or assigning bounding box information to an object. from here users can mark an object as hidden or visible, and publish the record in order to save it back into the system. edit system: edit record as i mentioned above there are a number of other components that are proving to be important as we move a large number of works into our metadata system. in the past week, we have created over new accounts for students and staff in the library so that they can begin to incorporate metadata editing into their work. in the next few posts, i will go over how we are attempting to manage who is doing what and how we are providing social and technical infrastructure to help managers keep track of what is going on with the folks they are responsible for. if you have questions or comments about this post, please let me know via twitter. this entry was posted in thinking outloud and tagged covid- , metadata editing on march , by vphill. user session analysis: unt scholarly works this is a continuation of a series of posts that i never got around to writing earlier this semester. i posted the first and second post in the series in february but never got around to writing the rest of them. this time i am looking at if users use items from multiple collections in the unt digital library. the dataset i am using is a subset of the , , user sessions logged in the unt libraries digital collections in . this subset is specifically for the unt scholarly works repository. there are a total of , sessions in this dataset and these have been processed in the same way as was mentioned in previous posts. of these , sessions, there were , that were sessions that involved interactions with a single item. this means % of the time when a user made use of an object in the unt scholarly works repository, it was for just one item. this leaves us , of the sessions in that would be interesting to look at for our further analysis. items accessed sessions percentages of all sessions , . % , . % , . % , . % , . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % . % based on what i see in this table i’m choosing item uses as the cutoff point for further analysis. this means that i will be looking at all of the sessions that have – items per session. this is % of the , unt scholarly works sessions and % of the sessions that have more than one item used. this represents , user sessions we are analyzing in the rest of this post.. looking at the sessions with the dataset layout we have we can easily go through and look at the partners, resource types, and finally the collections used per session. let’s get started. the first category we will look at are the partners present in the sessions. in the unt scholarly works collection there is generally a single partner field for a record. this partner is usually the contributing college, department, or center on campus where the author of the resource is contributing from. the model is flat and doesn’t allow for any nuance for multiple authors from different colleges but seems to work pretty well for many of the items in the repository. as i said there is generally a one-to-one relationship between an object and a partner field in the dataset. unt scholarly works: partners per session from the partners per session graph we can see that there are many sessions that make use of items from multiple partners in a single session. in fact . % of the sessions that accessed - items made use of items from more than one partner. to me that is really telling that there are discoveries being made that span disciplines in this collection. so a user could pull an article that was contributed by the college of information and in the same session pull up something that is from the college of music. that’s pretty cool. the next thing we can look at is the number of different resource types that are used in a given session. there is generally one resource type per digital object. these resource types could be an article, a book chapter, a report, a poster, or a presentation. we are interested in seeing how often a session will include multiple different types of resources. unt scholarly works: types per session in looking at the graph above we can see that for sessions that included between and items there were % of the sessions where users made use of items that were different types. the final area we will look is at the collections per session. this is a little bit messier to explain because it is possible (and common) for a single digital object to have multiple collections. we had to take this into account in the way that we counted the collections per session. unt scholarly works: collections per session this graph matches the same kind of pattern that we saw for partners and resource types. for sessions that used between two and eleven items % of the sessions used two or more different collection combinations. this means that when a user looked at two or more different records there was a very high chance that they were going to be pulling up a digital object that was from another collection in the unt digital library. so how does this happen? i can come up with four different ways that this can happen. a user is using the main search on https://digital.library.unt.edu/ and just pulls up items that are from a number of collections. a user is searching one of the combined search interfaces we have for the library that includes all million metadata records in the unt libraries digital collections. a user is coming to our content from a google search that lands them in a collection and they navigate more broadly to get to a resource. a user has multiple different browser tabs open and might even have two different search tasks going on but they are getting combined into one session because of the way we are grouping things for this analysis. there are probably other ways that this is happening which might be a good thing to look at in more depth in the future. i looked briefly at the full list of collections that get used together and some of the combinations aren’t immediately interpretable with a logical story of how these items were viewed together within a session. the web gets messy. cross-collection sessions in looking at the number of sessions that spanned more than one collection i was interested in understanding which collections were most used with the unt scholarly works repository collection. i took all of the collections present in each session and created pairs of collections in the form of (‘untsw’, ‘untetd’) or (‘untsw’, ‘trail’). these were then grouped and then the results placed into a table to show how everything matches up. untsw untetd osti trail ota tdnp mdid crsr jnds untgw untsw , , , , , untetd , osti , trail , ota , tdnp , mdid crsr jnds untgw the table needs just one piece of information to keep in mind. when you start comparing collections that don’t include untsw you need to remember that because we limited our dataset to sessions that included untsw you should always add that into your interpretation. for example if you were looking at how often do items from untetd (our theses and dissertation collection) get used with trail (technial report archive and image library collection) you will get sessions. but you also have to add into that untsw so it is really, how often does untetd, trail and untsw get used together which is . just looking at the first column of the table will give us the collections that are most often accessed within sessions with the unt scholarly works repository. i pulled those out into the chart below. cross-collection sessions by far the most commonly used collection with the unt scholarly works repository collection is the unt theses and dissertations collection. this occurs % of the time when there are two or more collections used in a session. the other collections drop off very quickly after untetd. this analysis is just another quick stab at understanding how the digital collections are being accessed by our users. i think that there is more that we can do with this data and hopefully i’ll get around to doing a bit more analysis this summer. there are still a few research questions from our original post that we haven’t answered. if you have questions or comments about this post, please let me know via twitter. this entry was posted in thinking outloud on june , by vphill. introducing sampling and new algorithms to the clustering dashboard one of the things that we were excited about when we adding the clustering dashboard to the unt libraries’ edit system was the ability to experiment with new algorithms for grouping or clustering metadata values. i gave a rundown of the cluster dashboard in a previous blog post this post is going to walk through some of things that we’ve been doing to try and bring new data views to the metadata we are managing here at the unt libraries. the need to sample i’m going to talk a bit about the need to sample values first and then get to the algorithms that make use of it in a bit. when we first developed the cluster dashboard we were working with a process that would take all of the values of a selected metadata element, convert those values into a hash value of some sort and then and identify where there were more than one value that produces the same hash. we were only interested in the instances that contained multiple values that had the same hash. while there were a large number of clusters for some of the elements, each cluster had a small number of values. i think the biggest cluster i’ve seen in the system had values. this is easy to display to the user in the dashboard so that’s what we did. moving forward we wanted to make use of some algorithms that would result in hundreds, thousands, and even tens of thousands of values per cluster. an example of this is trying to cluster on the length of a field. in our dataset there are , different creator values that are twelve characters in length. if we tried to display all of that to the user we would quickly blow up the browser for the user which is never any fun. what we have found is that there are some algorithms we want to use that will always return all of the values and not only when there are multiple values that share a common hash. for these situations we want to be proactive and sample the cluster members so that we don’t overwhelm the users interface. sampling options in cluster dashboard you can see in the screenshot above that there are a few different ways that you can sample the values of a cluster. random first alphabetically last alphabetically most frequent least frequent this sampling allows us to provide some new types of algorithms but still keep the system pretty responsive. so far we’ve found this works because when you are using these cluster algorithms that return so many value you generally aren’t interested in the clusters that are the giant clusters. you are typically looking for anomalies that show up in smaller clusters, like really long or really short values for a field. cluster options in dashboard showing sampled and non-sampled clustering algorithms. we divided the algorithm selection dropdown into two parts to try and show the user the algorithms that will be sampled and the ones that don’t require sampling. the option to select a sample method will only show up when it is required by the algorithm selected. new algorithms as i mentioned briefly above we’ve added a new set of algorithms to the cluster dashboard. these algorithms have been implemented to find anomalies in the data that are a bit hard to find other ways. first on the list is the length algorithm. this algorithm uses the number of characters or length of the value as the clustering key. generally the very short and the very long values are the ones that we are interested in. i’ll show some screenshots of what this reveals about our subject element. i always feel like i should make some sort of defense of our metadata when i show these screenshots but i have a feeling that anyone actually reading this will know that metadata is messy. subject clustered by length (shortest) subject clustered by length (longest) so quickly we can get to values that we probably want to change. in this case the subject values that are only one character in length or those that are over , characters in length. a quick story about how this is useful. we had a metadata creator a few years back accidentally pasted the contents of a personal email into the title field of a photograph because they just got their clipboard mixed up. they didn’t notice this so it went unnoticed for a few weeks until it was stumbled on by another metadata editor. this sort of thing happens from time to time and can show up with this kind of view. there are a few variations on the length that we provide. instead of the number of characters we have another view that is the count of tokens in the metadata value. so a value of “university of north texas” would have a token count of . this gives a similar but different view as the length. beyond that we provide some algorithms that look at the length of tokens within the values. so the value of “university of north texas” would have an average token length of . . i’ve honestly not found a good use for the average token length, median token length, token length mode, or token length range yet but maybe we will? finally there is the pattern mask algorithm that was implemented primarily for the date field in our metadata records. this algorithm takes in the selected metadata element values and converts all digits to and all of the letters to an a. it leaves all punctuation characters alone. so a value of “ ” maps to “ ” or a value of “july , ” maps to “aaaa , ”. pattern mask on date element in the example above you can quickly see the patterns that we will want to address as we continue to clean up our date element. as i mentioned at the beginning of the post, one of the things that we were excited about when we implemented the cluster dashboard was the ability to try out different algorithms for looking at our metadata. this is our first set of “new” algorithms for the system. we also had to add the ability to sample the clusters because the can quickly get crazy with the number of values. hopefully we will be able to add additional clustering algorithms to the system in the future. are there any ideas that you have for us that you would like us to try out in the interface? if so please let me know, we would love to experiment a bit. if you have questions or comments about this post, please let me know via twitter. this entry was posted in thinking outloud on april , by vphill. user session analysis: investigating sessions in the previous post in this series i laid out the work that we were going to do with session data from the unt libraries’ digital collections. in order to get the background that this post builds from take a quick look at that post. in this post we are going to look at the data for the , , user sessions that we generated from the apache access logs from the unt libraries digital collections. items per sessions the first thing that we will take a look at in the dataset is information about how many different digital objects or items are viewed during a session. items accessed sessions percentage of all sessions , , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % - , . % - , . % - , . % - , . % - . % - . % - . % - . % - , . % - . % - . % - . % - . % - . % - . % - . % - . % + . % i grouped the item uses per session in order to make the table a little easier to read. with % of sessions being single item accesses that means we have % of the sessions that have more than one item access. this is still , , sessions that we can look at in the dataset so not bad. you can also see that there are a few sessions that have a very large number of items associated with them. for example there are sessions that have over , items being used. i would guess that this is some sort of script or harvester that is masquerading as a browser. here are some descriptive statistics for the items per session data. n min median max mean stdev , , , . . for further analysis we will probably restrict our sessions to those that have under items used in a single session. while this might remove some legitimate sessions that used a large number of items, it will give us numbers that we can feel a bit more confident about. that will leave , , or % of the sessions with more than one item used still in the dataset for further analysis. duration of sessions the next thing we will look at is the duration of sessions in the dataset. we limited a single session to all interactions by an ip address in a thirty minute window so that gives us the possibility of sessions up to , seconds. minutes sessions percentage of sessions , , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % , . % the table above groups a session into buckets for each minute. the biggest bucket by number of sessions is the bucket of minutes. this bucket has sessions that are up to seconds in length and accounts for , , or % of the sessions in the dataset. duration sessions percent of sessions under min sec , , % - sec , , % - sec , % - sec , % - sec , % - sec , % - sec , % you might be wondering about those sessions that lasted only zero seconds. there are , , of them which is % of the sessions that were under one minute. these are almost always sessions that used items as part of an embedded link, a pdf view directly from another site (google, twitter, webpage) or a similar kind of view. next steps this post helped us get a better look at the data that we are working with. there is a bit of strangeness here and there with the data but this is pretty normal for situations where you work with access logs. the web is a strange place full of people, spiders, bots, and scripts. next up we will actually dig into some of the research questions we had in the first post. we know how we are going to limit our data a bit to get rid of some of the outliers in the number of items used and we’ve given a bit of information about the large number of very short duration sessions. so more to come. if you have questions or comments about this post, please let me know via twitter. this entry was posted in thinking outloud on february , by vphill. user session analysis: connections between collections, type, institutions i’ve been putting off some analysis that a few of us at the unt libraries have wanted to do with the log files of the unt libraries digital collections. this post (and probably a short series to follow) is an effort to get back on track. there are three systems that we use to provide access to content and those include: the portal to texas history, the unt digital library, and the gateway to oklahoma history. in our digital collections there are a few things that we’ve said over time that we feel very strongly about but which we’ve never really measured. first off we have said that there is value in co-locating all of our content in the same fairly uniform system instead of building visually and functionally distinct systems for different collections of items. so instead of each new project or collection going into a new system, we’ve said there is not only cost savings, but real value in putting them all together in a single system. we’ve said “there is an opportunity for users to not only find content from your collection, but they could find useful connections to other items in the overall digital library”. another thing we’ve said is that there is value in putting all different types of digital objects together into our digital systems. we put the newspapers, photographs, maps, audio, video, and datasets together and we think there is value in that. we’ve said that users will be able to find newspaper issues, photographs, and maps that might meet their need. if we had a separate newspaper system, separate video or audio system some of this cross-type discovery would never take place. finally we’ve said that there is great value in locating collections from many institutions together in a system like the portal to texas history. we thought (and still think) that users would be able to do a search and it will pull resources together from across institutions in texas that have matching resources. because of the geography of the state, you might be finding things that are physically located or hours away from each other at different institutions. in the portal, these could be displayed together, something that would be challenging if they weren’t co-located in a system. in our mind these aren’t completely crazy concepts but we do run into other institutions and practitioner that don’t always feel as strongly about this as we do. the one thing that we’ve never done locally is look at the usage data of the systems and find out: do users discover and use items from different collections? do users discover and use items that are different types? do users discover and use items that are from different contributing partners? this blog post is going to be the first in a short series that takes a look at the usage data in the unt libraries digital collections in an attempt to try and answer some of these questions. hopefully that is enough background, now let’s get started: how to answer the questions. in order to get started we had to think a little bit about how we wanted to pull together data on this. we have been generating item-based usage for the digital library collections for a while. these get aggregated into collection and partner statistics that we make available in the different systems. the problem with this data is that it just shows what items were used and how many times in a day they were used. it doesn’t show what was used together. we decided that we needed to go back to the log files from the digital collections and re-create user sessions to group item usage together. after we have information about what items were used together we can sprinkle in some metadata about those items and start answering our questions. with that as a plan we can move to the next step. preparing the data we decided to use all of the log files for from our digital collections servers. this ends up being , , , lines of apache access logs (geez, over . billion, or . million server requests a day). the data came from two different servers that collectively host all of the application traffic for the three systems that make up the unt libraries’ digital collections. we decided that we would define a session as all of the interactions that a single ip address has with the system in a minute window. if a user uses the system for more than minutes, say minutes, that would count as one thirty minute session and one fifteen minute session. we started by writing a script that would do three things. first it would ignore lines in the log file that were from robots and crawlers. we have a pretty decent list of these bots so that was easy to remove. next we further reduced the data by only looking at digital object accesses. specifically lines that looked something like ‘/ark:/ /metapth /`. this pattern in our system denotes an item access and these are what we were interested in. finally we only were concerned with accesses that returned content so we only looked at lines that returned a status code. we filtered the log files down to three columns of data. the first column was the timestamp for when the http access was made, the second column was the has of the hashed ip address used to make the request, and the final column was the digital item path requested. this resulted in a much smaller dataset to work with, from , , , down to , , individual lines of data. here is what a snipped of data looks like dce e d a e a b a ec e /ark:/ /metadc /m / /high_res_d/bulletin .pdf fa cf b a fe f /ark:/ /metadc /m / /high_res/ fa cf b a fe f /ark:/ /metadc /m / /high_res/ b e b aadb d c ab /ark:/ /metadc /m / /high_res_d/dissertation.pdf accb d f ef d bfb /ark:/ /metacrs /m / /high_res_d/rs _ may .pdf decabc fc bad b /ark:/ /metadc /m / /small_res/ f b f b fd c beee c /ark:/ /metadc / f b f b fd c beee c /ark:/ /metadc /m / /small_res/ c a f b a fabd f c /ark:/ /metadc /m / /med_res_d/ f b f b fd c beee c /ark:/ /metadc /m / /med_res/ f b f b fd c beee c /ark:/ /metadc /m / /small_res/ f b f b fd c beee c /ark:/ /metadc /m / /small_res/ f b f b fd c beee c /ark:/ /metadc /m / /small_res/ ef c dd cb b b f b /ark:/ /metadc /m / /small_res/ a ce e cd e b c df d a /ark:/ /metadc /m / /med_res/ ea ba d a d ff c e e /ark:/ /metadc /m / /high_res/ you can see the three columns in the data there. the next step was actually to sort all of this data by the timestamp in the first column. you might notice that not all of the lines are in chronological order in the sample above. by sorting on the timestamp, things will fall into order based on time. the next step was to further reduce this data down into sessions. we created a short script that we could feed the data into and it would keep track of the ip addresses it came across, note the objects that the ip hash used, and after a thirty minute period of time (based on the timestamp) it would start the aggregation again. the result was a short json structure that looked like this. { "arks": ["metapth ", "metapth "], "ip_hash": " ebfe f b c b e ead e ", "timestamp_end": , "timestamp_start": } this json has the ip hash, the starting and ending timestamp for that session, and finally the items that were used. each of these json structures were placed into a file, a line-oriented set of json “files” that would get used in the following steps. this new line-oriented json file is , , lines long, with one line representing a single user session for the unt libraries’ digital collections. i think that’s pretty cool. i think i’m going to wrap up this post but in the next post i will take a look at what these users sessions look like with a little bit of sorting, grouping, plotting, and graphing. if you have questions or comments about this post, please let me know via twitter. this entry was posted in thinking outloud on february , by vphill. metadata quality interfaces: cluster dashboard (openrefine clustering baked right in) this is the last of the updates from our summer’s activities in creating new metadata interfaces for the unt libraries digital collections. if you are interested in the others in this series you can view the past few posts on this blog where i talk about our facet, count, search, and item interfaces. this time i am going to talk a bit about our cluster dashboard. this interface took a little bit longer than the others to complete. because of this, we are just rolling it out this week, but it is before autumn so i’m calling it a summer interface. i warn you that there are going to be a bunch of screenshots here, so if you don’t like those, you probably won’t like this post. cluster dashboard for a number of years i have been using openrefine for working with spreadsheets of data before we load them into our digital repository. this tool has a number of great features that help you get an overview of the data you are working with, as well as identifying some problem areas that you should think about cleaning up. the feature that i have always felt was the most interesting was their data clustering interface. the idea of this interface is that you choose a facet, (dimension, column) of your data and then group like values together. there are a number of ways of doing this grouping and for an in-depth discussion of those algorithms i will point you to the wonderful openrefine clustering documentation. openrefine is a wonderful tool for working with spreadsheets (and a whole bunch of other types of data) but there are a few challenges that you run into when you are working with data from our digital library collections. first of all our data generally isn’t rectangular. it doesn’t easily fit into a spreadsheet. we have some records with one creator, we have some records with dozens of creators. there are ways to work with these multiple values but things get complicated. the bigger challenge we generally have is that while many systems can generate a spreadsheet of their data for exporting, very few of them (our system included) have a way of importing those changes back into the system in a spreadsheet format. this means that while you could pull data from the system, clean it up in openrefine, when you were ready to put it back in the system you would run into the problem that there wasn’t a way to get that nice clean data back into the system. a way that you could use openrefine was to identify records to change and then have to go back into the system and change records there. but that is far from ideal. so how did we overcome this? we wanted to use the openrefine clustering but couldn’t get data easily back into our system. our solution? bake the openrefine clustering right into the system. that’s what this post is about. the first thing you see when you load up the cluster dashboard is a quick bit of information about how many records, collections, and partners you are going to be working on values from. this is helpful to let you know the scope of what you are cluster, both to understand why it might take a while to generate clusters, but also because it is generally better to run these clustering tools over the largest sets of data that you can because it can pull in variations from many different records. other than that you are presented with a pretty standard dashboard interface from the unt libraries’ edit system. you can limit to subsets of records with the facets on the left side and the number of items you cluster over will change accordingly. cluster dashboard the next thing that you will see is a little help box below the clustering stats. this is a help interface that helps to explain how to use the clustering dashboard and a little more information about how the different algorithms work. metadata folks generally like to know the fine details about how the algorithms work, or at least be able to find that information if they want to know it later. cluster dashboard help the first thing you do is select a field/element/facet that you are interested in clustering. in the example below i’m going to select the contributor field. choosing an element to cluster once you make a selection you can further limit it to a qualifier, in this case you could limit it to just the contributors that are organizations, or contributors that are composers. as i said above, using more data generally works better so we will just run the algorithms over all of the values. you next have the option of choosing an algorithm for your clustering. we recommend to people that they start with the default fingerprint algorithm because it is a great starting point. i will discuss the other algorithms later in this post. choosing an algorithm after you select your algorithm, you hit submit and things start working. you are given a screen that will have a spinner that tells you the clusters are generating. generating clusters depending on your dataset size and the number of unique values of the selected element, you could get your results back on a second or dozens of seconds. the general flow of data after you hit submit is to query the solr backend for all of the facet values and their counts. these values are then processed with the chosen algorithm that creates a “key” for that value. another way to think about it is that the values are placed into a bucket that groups similar values together. there are some calculations that are preformed on the clusters and then they are cached for about ten minutes by the system. after you wait for the clusters to generate the first time they are much quicker for the next ten minutes. in the screen below you can see the results of this first clustering. i will go into detail about the values and options you have to work with the clusters. contributor clusters with fingerprint key collision hashing the first thing that you might want to do is sort the clusters in a different way. by default they are sorted with the value of the cluster key. sometimes this makes sense, sometimes it doesn’t make sense as to why something is in a given order. we thought about displaying the key but found that it was also distracting in the interface. different ways of sorting clusters one of the ways that i like to sort the clusters is by the number of cluster members. the image below shows the clusters with this sort applied. contributor field sorted by members here is a more detailed view of a few clusters. you can see that the name of the russian composer shostakovich has been grouped into a cluster of members. this represents different records in the system with a contributor element for this composer. next to each member value you will see a number in parenthesis, this is the number of records that uses that variation of the value. contributor cluster detail you can also sort based on the number of records that a cluster contains. this brings up the most frequently used values. generally there are a large number that have a value and then a few records that have a competing value. usually pretty easy to fix. contributor element sorted by records sorting by the average length variation can help find values that are strange duplications of themselves. repeated phrases, a double copy and paste, strange things like that come to the surface. contributor element sorted by average length variation finally sorting by average length is helpful if you want to work with the longest or shortest values that are similar. contributor element sorted by average length different algorithms i’m going to go through the different algorithms that we currently have in production. our hope is that as time moves forward we will introduce new algorithms or slight variations of algorithms to really get at some of the oddities of the data in the system. first up is the fingerprint algorithm. this is a direct clone of the default fingerprint algorithm used by openrefine. contributor element clustered using fingerprint key collision a small variation we introduced was instead of replacing punctuation with a whitespace character, the fingerprint-ns (no space) just removes the punctuation without adding whitespace. this would group f.b.i with fbi where the other fingerprint algorithm wouldn’t group them together. this small variation surfaces different clusters. we had to keep reminding ourselves that when we created the algorithms that there wasn’t such a thing as “best”, or “better”, but instead they were just “different”. contributor element clustered using fingerprint (no space) key collision one thing that is really common for names in bibliographic metadata is that they have many dates. birth, death, flourished, and so on. we have a variation of the fingerprint algorithm that removes all numbers in addition to punctuation. we call this one fingerprint-nd (no dates). this is helpful for grouping names that are missing dates with versions of the name that have dates. in the second cluster below i pointed out an instance of mozart’s name that wouldn’t have been grouped with the default fingerprint algorithm. remember, different, not better or best. contributor element clustered using fingerprint (no dates) key collision from there we branch out into a few simpler algorithms. the caseless algorithm just lowercases all of the values and you can see clusters that only differ in ways that are related to upper case or lower case values. contributor element clustered using caseless (lowercase) key collision next up is the ascii algorithm which tries to group together values that only differ in diacritics. so for instance the name jose and josé would be grouped together. contributor element clustered using ascii key collision the final algorithm is just a whitespace normalization called normalize whitespace, it removes consecutive whitespace characters to group values. contributor element clustered using normalized whitespace key collision you may have noticed that the number of clusters went down dramatically from the fingerprint algorithms to the caseless, ascii, or normalize whitespace, we generally want people to start with the fingerprint algorithms because they will be useful most of the time. other example elements here are a few more examples from other fields. i’ve gone ahead and sorted them by members (high to low) because i think that’s the best way to see the value of this interface. first up is the creator field. creator element clustered with fingerprint algorithm and sorted by members next up is the subject field. we have so so many ways of saying “ou football” subject element clustered with fingerprint algorithm and sorted by members the real power of this interface is when you start fixing things. in the example below i’m wanting to focus in on the value “football (o u )”. i do this by clicking the link for that member value. subject element cluster detail you are taken directly to a result set that has the records for that selected value. in this case there are two records with “football (o u )”. selected records all you have to do at this point is open up a record, make the edit and publish that record back. many of you will say “yeah but wouldn’t some sort of batch editing be faster here?” and i will answer “absolutely, we are going to look into how we would do that!” (but it is a non-trivial activity due to how we manage and store metadata, so sadface 🙁 ) subject value in the record there you have it, the cluster dashboard and how it works. the hope is to empower our metadata creators and metadata managers to better understand and if needed, clean up the values in our metadata records. by doing so we are improving the ability for people to connect different records based on common valuse between the records. as we move forward we will introduce a number of other algorithms that we can use to cluster values. there are also some other metrics that we will look at for sorting records to try and tease out “which clusters would be the most helpful to our users to correct first”. that is always something we are keeping in the back of our head, how can we provide a sorted list of things that are most in need of human fixing. so if you are interested in that sort of thing stay tuned, i will probably talk about it on this blog. if you have questions or comments about this post, please let me know via twitter. this entry was posted in thinking outloud on september , by vphill. metadata interfaces: search dashboard this is the next blog post in a series that discusses some of the metadata interfaces that we have been working on improving over the summer for the unt libraries digital collections. you can catch up on those posts about our item views, facet dashboard, and element count dashboard if you are curious. in this post i’m going to talk about our search dashboard. this dashboard is really the bread and butter of our whole metadata editing application. about % of the time a user who is doing some metadata work will login and work with this interface to find the records that they need to create or edit. the records that they see and can search are only ones that they have privileges to edit. in this post you will see what i see when i login to the system, the nearly . million records that we are currently managing in our systems. let’s get started. search dashboard if you have read the other post you will probably notice quite a bit of similarity between the interfaces. all of those other interfaces were based off of this search interfaces. you can divide the dashboard into three primary sections. on the left side there are facets that allow you to refine your view in a number of ways. at the top of the right column is an area where you can search for a term or phrase in a record you are interested in. finally under the search box there is a result set of items and various ways to interact with those results. by default all the records that you have access to are viewable if you haven’t refined your view with a search or a limiting facet. edit interface search dashboard the search section of the dashboard lets you find a specific record or set of records that you are interested in working with. you can choose to search across all of the fields in the metadata record or just a specific metadata field using the dropdown next to where you enter your search term. you can search single words, phrases, or unique identifiers for records if you have those. once you hit the search button you are on your way. search and view options for records once you have submitted your search you will get back a set of results. i’ll go over these more in depth in a little bit. record detail you can sort your results in a variety of ways. by default they are returned in title order but you can sort them by the date they were added to the system, the date the original item was created, the date that the metadata record was last modified, the ark identifier and finally by a completeness metric. you also have the option to change your view from the default list view to the grid view. sort options here is a look at the grid view. it presents a more visually compact view of the records you might be interested in working with. grid view the image below is a detail of a record view. we tried to pack as much useful information into each row as we could. we have the title, a thumbnail, several links to either the edit or summary item view on the left part of the row. following that we have the system, collection, and partner that the record belongs to. we have the unique ark identifier for the object, the date that it was added to the unt libraries’ digital collections, and the date the metadata was last modified. finally we have a green check if the item is visible to the public or a red x if the item is hidden from the public. record detail facet section there are a number of different facets that a user can use to limit the records they are working with to a smaller subset. the list is pretty long so i’ll first show you it in a single image and then go over some of the specifics in more detail below. facet options the first three facets are the system, collection and partner facets. we have three systems that we manage records for with this interface, the portal to texas history, the unt digital library, and the gateway to oklahoma history. each digital item can belong to multiple collections and generally belongs to a single partner organization. if you are interested in just working on the records for the kxas-nbc new collection you can limit your view of records by selecting that value from the collections facet area. system, collections and partners facet options next are the resource type and visibility facets. it is often helpful to limit to just a specific resource type, like maps when you are doing your metadata editing so that you don’t see things that you aren’t interested in working with. likewise there are some kinds of metadata editing that you want to focus primarily on items that are already viewable to the public and you don’t want the hidden records to get in the way. you can do this with the visibility facet. resource type and visibility facet options next we start getting into the new facet types that we added this summer to help identify records that need some metadata uplift. we have the date validity, my edits, and location data facets. date validity is a facet that allows you to identify records that have dates in them that are not valid according to the extended date time format (edtf). there are two different fields in a record that are checked, the date field and the coverage field (which can contain dates). if any of these aren’t valid edtf strings then we mark the whole record as having invalid dates. you can use this facet to identify these and go in a correct those values. next up is a facet for just the records that you have edited in the past. this can be helpful for a number of reasons. i use it from time to time to see if any of the records that i’ve edited have developed any issues like dates that aren’t valid since i last edited them. it doesn’t happen often but can be helpful. finally there is a section of location data. this set of facets is helpful for identifying records which have or don’t have a place name, place point, or place box in the record. helpful if you are working through a collection trying to add geographic information to the records. date validity, my edits, and location data facet options the final set of facets are recently edited records, and record completeness. the first is the recently edited records which is pretty straight forward. this just a listing of how many records have been edited in the past h, h, d, d, d, d in the system. one note that causes a bit of confusion here is that these are records that are edited by anyone in the past period of time. it is often misunderstood as “your edits” in a given period of time which isn’t true. still very helpful but can get you into some strange results if you think about it the other way. the last facet value is for the record completeness. we really have two categories, records that have a completeness of . (complete records) or records that are less than . (incomplete records). this metric is calculated when the item is indexed in the system and based on our notion of a minimally viable record. recently edited records and record completeness facet options this finishes this post about the search dashboard for the unt libraries digital collections. we have been working to build out this metadata environment for about the last eight years and have slowly refined it to the metadata creation and editing workflows that seem to work for the widest number of folks here at unt. there are always improvements that we can make and we have been steadily chipping away at those over time. there are a few other things that we’ve been working on over the summer that i will post about in the next week or so, so stay tuned for more. if you have questions or comments about this post, please let me know via twitter. this entry was posted in thinking outloud on september , by vphill. metadata quality interfaces: element count dashboard next up in our review of the new metadata quality interfaces we have implemented this summer is our element count dashboard. the basics of this are that whenever we index metadata records in our solr index we go ahead and count the number of instances of a given element, or a given element with a specific qualifier and store those away in the index. this results in hundreds of fields that are the counts of element instances in those fields. we built an interface on top of these counts because we had a hunch that we would be able to use this information to help us identify problems in our metadata records. it feels like i’m showing some things in our metadata that we probably don’t want to really highlight but it is all for helping others understand. so onward! element count dashboard the dashboard is similar to other dashboards in the edit system. you have the ability to limit your view to just the collection, partner or system you are interested in working with. count dashboard from there you can select an element you are interested in viewing counts for. in the example below i am interested in looking at the description element or field. select an element to view counts once your selection is made you are presented with the number of instances of the description field in a record. this is a little more helpful if you know that in our metadata world, a nice clean record will generally have two description fields. one for a content description and one for a physical description of the item. more than two is usually strange and less than one is usually bad. counts for description elements to get a clearer view you can see the detail below. this again is for the top level description element where we like to have two descriptions. detail of description counts you can also limit to a qualifier specifically. in the example below you see the counts of description elements with a content qualifier. the , records that have two description elements with a content qualifier are pretty strange. we should probably fix those. detail of description counts for content qualifier next we limit to just the physical description qualifier. you will see that there are a bunch that don’t have any sort of physical description and then that have two. we should fix both of those record sets. detail of description counts for physical qualifier because of the way that we index things we can also get at the description elements that don’t have either a content or physical qualifier selected. these are identified with a value of none for the qualifier. you can see that there are , , records that have zero description elements with a none qualifier. that’s awesome. you can also see that have one element and that have two elements that are missing qualifiers. that’s not awesome. detail of description counts for none qualifier i’m hoping you are starting to see how this kind of interface could be useful to drill into records that might look a little strange. when you identify something strange all you have to do is click on the number and you are taken directly to the records that match what you’ve asked for. in the example below we are seeing all of the records that have two physical descriptions because this is something we are interested in correcting. records with multiple duplicate physical qualifiers if you open up a record to edit you will see that yes, in fact there are two physical descriptions in this record. it looks like the first one should actually be a content description. example of two physical descriptions that need to be fixed once we change that value we can hit the publish button and be on our way fixing other metadata records. the counts will update about thirty seconds later to reflect the corrections that you have made. fixed physical and content descriptions even more of a good thing. because i think this is a little different than other interfaces you might be used to, it might be good to see another example. this time we are looking at the creator element in the element count dashboard. creator counts you will see that there are different counts from zero way up into way way too many creators on an item (silly physics articles). i was curious to see what the counts looked like for creator elements that were missing a role qualifier. these are identified by selecting the none value from the qualifier dropdown. creator counts for missing qualifiers you can see that the majority of our records don’t have creator elements missing the role qualifier but there are a number that do. we can fix those. if you wanted to look at those records that have five different creator elements that don’t have a role you would end up getting to records that loo like the one below. example of multiple missing types and roles you will notice that when a record has a problem there are often multiple things wrong with it. in this case not only is it missing role information for each of these creator elements but there is also name type information that is missing. once we fix those we can move along and edit some more. and a final example. i’m hoping you are starting to see how this interface could be useful. here is another example if you aren’t convinced yet. we are completing a retrospective digitization of theses and dissertations here at unt. not only is this a bunch of digitization but it is quite a bit of metadata that we are adding to both the unt digital library as well as our traditional library catalog. let’s look at some of those records. you can limit your dashboard view to the collection you are interested in working on. in this case we choose the unt theses and dissertations collection. next up we take a look at the number of creator elements per record. theses and dissertations are generally authored by just one person. it would be strange to see counts other than one. creator counts for these and dissertations collection it looks lie there are records that are missing creator elements and a single record that for some reason has two creator elements. this is strange and we should take a look. below you will see the view of the records that are missing a creator element. sadly at the time of writing there are seven of these that are visible to the public so that’s something we really need to fix in a hurry. example theses that are missing creators that’s it for this post about our element count dashboard. i hope that you find this sort of interface interesting. i’d be interested to hear if you have interfaces like this for your digital library collections or if you think something like this would be useful in your metadata work. if you have questions or comments about this post, please let me know via twitter. this entry was posted in thinking outloud on september , by vphill. metadata quality interfaces: facet dashboard this is the second post in a series that discusses the new metadata interfaces we have been developing for the unt libraries’ digital collections metadata editing environment. the previous post was related to the item views that we have created. this post discusses our facet dashboard in a bit of depth. let’s get started. facet dashboard a little bit of background is in order so that you can better understand the data that we are working with in our metadata system. the unt libraries uses a locally-extended dublin core metadata element set. in addition to locally-extending the elements to include things like collection, partner, degree, citation, note, and meta fields we also qualify many of the fields. a qualifier usually specifics what type of value is represented. so a subject could be a keyword, or an lcsh value. a creator could be an author, or a photographer. many of the fields have the ability to have one qualifier for the value. when we index records in our solr instance we store strings of each of these elements, and each of the elements plus qualifiers, so we have fields we can facet on. this results in facet fields for creator as well as specifically creator_author, or creator_photographer. for fields that we expect the use of a qualifier we also capture when there isn’t a qualifier in a field like creator_none. this results in many hundreds of fields in our solr index but we do this for good reason, to be able to get at the data in ways that are helpful for metadata maintainers. the first view we created around this data was our facet dashboard. the image below shows what you get when you go to this view. default facet dashboard on the left side of the screen you are presented with facets that you can make use of to limit and refine the information you are interested in viewing. i’m currently looking at all of the records from all partners and all collections. this is a bit over . million records. the next step is to decide which field you are interested in seeing the facet values for. in this case i am choosing the creator field. selecting a field to view facet values after you make a selection you are presented with a paginated view of all of the creator values in the dataset ( , unique values in this case). these are sorted alphabetically so the first values are the ones that generally start with punctuation. in addition to the string value you are presented the number of records in the system that have that given value. all creator values because there can be many many pages of results sometimes it is helpful to jump directly to a subset of the records. this can be accomplished with a “begins with” dropdown in the left menu. i’m choosing to look at only facets that start with the letter d. limit to a specific letter after making a selection you are presented with the facets that start with the letter d instead of the whole set. this makes it a bit easier to target just the values you are looking for. creator values starting with d sometimes when you are looking at the facet values you are trying to identify values that fall next to each other but that might differ only a little bit. one of the things that can make this a bit easier is having a button that can highlight just the whitespace in the strings themselves. highlight whitespace button once you click this button you see that the whitespace is now highlighted in green. this highlighting in combination with using a monospace font makes it easier to see when values only differ with the amount of whitespace. highlighted whitespace once you have identified a value that you want to change the next thing to do is just click on the link for that facet value. identified value to correct you are taken to a new tab in your browser that has just the records that have the selected value. in this case there was just one record with “d & h photo” that we wanted to edit. record with identified value we have a convenient highlighting of visited rows on the facet dashboard so you know which values you have clicked on. highlighted reminder of selected value in addition to just seeing all of the values for the creator field you can also limit your view to a specific qualifier by selecting the qualifier dropdown when it is available. select an optional qualifier you can also look at items that don’t have a given value, for example creator values that don’t have a name type designated. this is identified with a qualifier value of none-type. creator values without a designated type you get just the + values in the system that don’t have a name type designated. all of this can be performed on any of the elements or any of the qualified elements of the metadata records. while this is a useful first step in getting metadata editors directly to both the values of fields and their counts in the form of facets, it can be improved upon. this view still requires users to scan a long long list of items to try and identify values that should be collapsed because they are just different ways of expressing the same thing with differences in spacing or punctuation. it is only possible to identify these values if they are located near each other alphabetically. this can be a problem if you have a field like a name field that can have inverted or non-inverted strings for names. so there is room for improvement of these interfaces for our users. our next interface to talk about is our count dashboard. but that will be in another post. if you have questions or comments about this post, please let me know via twitter. this entry was posted in thinking outloud on august , by vphill. post navigation ← older posts search for: recent posts metadata events system: accounting for time managing metadata editing for telecommuting. user session analysis: unt scholarly works introducing sampling and new algorithms to the clustering dashboard user session analysis: investigating sessions recent comments government data at risk – uc portal on how many of the eot pdf files were harvested in eot news roundup | lj infodocket on how do metadata records change over time? news roundup | lj infodocket on how do metadata records change over time? news roundup | lj infodocket on how do metadata records change over time? the internet archive has preserved tb of government website data during transition to trump administration - techsqrd.com on how many of the eot pdf files were harvested in eot proudly powered by wordpress newcardigan – making glam workers more awesome skip to content newcardigan making glam workers more awesome menu about who is newcardigan? announcements cardimail newcardigan timeline code of conduct & anti-harassment policy projects aus glam blogs cardicast cardiparties cardishorts glam blog club quarantinis donate join newcardigan is a social and professional group for people who work in galleries, libraries, archives and museums – and for those who like hanging around with glam types. we are an incorporated association registered in victoria, australia. log in home about join donate subscribe to podcast code of conduct & anti-harassment policy twitter facebook instagram email aus glam space aus glam blogs newcardigan inc po box abbotsford, vic , australia abn overview of various statistics about reproducible builds debian navigation suite/architecture overviews tested architectures: amd i arm armhf tested suites: unstable bullseye buster stretch experimental test results statistics results for unstable/amd unreproducible packages: with notes without notes other package states: package sets recently tested packages: last h last h all tested packages packages with .buildinfo files packages without .buildinfo files scheduled for amd maintainers of in unstable reproducible debian overview dashboard categorized issues bugs filed variations tested packages with notifications enabled ⚑ repositories overview backend related broken pieces documentation (eg. on manual scheduling) performance stats health monitoring node overview job overview daily graphs weekly graphs monthly graphs yearly graphs the reproducible builds project reproducible-builds.org reproducible builds - docs• news reproducible builds in debian - wiki source_date_epoch specification ci tests overview of various statistics about reproducible builds please also visit the more general website reproducible-builds.org where reproducible builds are explained in more detail than just bit by bit identical rebuilds to enable verifcation of the sources used to build. we think that reproducible builds should become the norm, so we wrote how to make your software reproducible . also aimed at the free software world at large, is the first specification we have written: the source_date_epoch specification . these pages are showing the potential of reproducible builds of debian packages . the results shown were obtained by several jobs running on jenkins.debian.net. thanks to ionos for donating the virtual machines this is running on! https://tests.reproducible-builds.org/ we are reachable via irc (#debian-reproducible and #reproducible-builds on oftc), or email, and we care about free software in general, so whether you are an upstream developer or working on another distribution, or have any other feedback - we'd love to hear from you! besides debian we are also testing coreboot , openwrt , netbsd , freebsd , and arch linux though not as thoroughly as debian yet. opensuse , nixos and f-droid are also being tested, though elsewhere. as far as we know, the guix challenge is not yet run systematically anywhere. testing of fedora has sadly been suspended for now. we can test more projects, if you contribute! suite all source packages reproducible packages unreproducible packages packages failing to build packages timing out packages in depwait state not for this architecture blacklisted bullseye/amd / . % / . % / . % / % / % / . % / % bullseye/i / . % / . % / . % / % / . % / . % / % bullseye/arm / . % / . % / . % / % / . % / . % / % bullseye/armhf / . % / . % / . % / . % / . % / . % / . % unstable/amd / . % / . % / . % / % / . % / . % / % unstable/i / . % / . % / . % / % / . % / . % / % unstable/arm / . % / . % / . % / % / . % / . % / % unstable/armhf / . % / . % / . % / % / . % / . % / . % experimental/amd / . % / . % / . % / . % / . % / . % / % experimental/i / . % / . % / . % / % / . % / . % / % experimental/arm / . % / . % / . % / . % / . % / . % / % experimental/armhf ( . % tested) / . % / . % / . % / . % / . % / . % / . % various reproducibility statistics source based amd i arm armhf identified distinct and categorized issues total number of identified issues in packages packages with notes about these issues packages in unstable with issues but without identified ones / . % / . % / . % / . % - unreproducible ones / . % / . % / . % / . % - failing to build / . % / . % / . % / . % packages in unstable which need to be fixed / . % / . % / . % / . % packages in bullseye with issues but without identified ones / . % / . % / . % / . % - unreproducible ones / . % / . % / . % / . % - failing to build / . % / . % / . % / . % packages in bullseye which need to be fixed / . % / . % / . % / . % committers to notes.git (in the last three months) committers to notes.git (in total) packages modified in our toolchain (in unstable) build network performance stats and an overview about the old suites 'stretch' and 'buster' are also available. this page was built by the jenkins job reproducible_html_dashboard which is configured via this git repo. there is more information about jenkins.debian.net and about reproducible builds of debian available elsewhere. please send technical feedback about jenkins to the debian jenkins development list, or as a bug report against the jenkins.debian.org package. feedback about specific job results should go to their respective lists and/or the bts. the code of jenkins.debian.net is mostly gpl- licensed. the weather icons are public domain and were taken from the tango icon library. copyright - holger levsen and many others. last update: - - : utc electronic frontier foundation | defending your rights in the digital world skip to main content about contact press people opportunities eff th anniversary issues free speech privacy creativity and innovation transparency international security our work deeplinks blog press releases events legal cases whitepapers take action action center electronic frontier alliance volunteer tools privacy badger https everywhere surveillance self-defense certbot atlas of surveillance cover your tracks crocodile hunter donate donate to eff shop other ways to give membership faq donate donate to eff shop other ways to give search form search email updates on news, actions, and events in your area. join eff lists copyright (cc by) trademark privacy policy thanks electronic frontier foundation donate eff turns ! learn more about us, and how you can help. eff turns ! learn more. electronic frontier foundation about contact press people opportunities eff th anniversary issues free speech privacy creativity and innovation transparency international security our work deeplinks blog press releases events legal cases whitepapers take action action center electronic frontier alliance volunteer tools privacy badger https everywhere surveillance self-defense certbot atlas of surveillance cover your tracks crocodile hunter donate donate to eff shop other ways to give membership faq donate donate to eff shop other ways to give search form search the leading nonprofit defending digital privacy, free speech, and innovation for years and counting! donate the leading nonprofit defending digital privacy, free speech, and innovation for years and counting! featured update twitter and interoperability: some thoughts from the peanut gallery late in , twitter ceo jack dorsey floated "project blue sky," a plan for an interoperable, federated, standardized twitter that would let users (or toolsmiths who work on behalf of users) gain more control over their participation in the twitter system. this was an exciting moment for us, a significant lurch towards an interoperable , decentralized social media world. we were especially excited to see dorsey cite mike masnick's excellent protocols, not products paper. it’s been more than a year,... featured update it’s not you hate, it’s oligopolies as we continue to hear calls to repeal or change section , it appears that many people have conflated a law that affects the tech giants (among many others) with big tech as a whole. section is not a gift to big tech, nor is repealing it a panacea for the problems big tech is causing—to the contrary repealing it will only exacerbate those problems. the thing you hate is not . it’s lack of competition. section stands... featured update amazon ring’s end-to-end encryption: what it means almost one year after eff called on amazon’s surveillance doorbell company ring to encrypt footage end-to-end, it appears they are starting to make this necessary change. this call was a response to a number of problematic and potentially harmful incidents, including larger concerns about ring’s security and reports that employees were fired for watching customers’ videos. now, ring is finally taking a necessary step—making sure that the transmission of footage from your ring camera to your phone cannot be viewed... join our newsletter! email updates on news, actions, and events in your area. email address postal code (optional) anti-spam question: enter the three-letter abbreviation for electronic frontier foundation: don't fill out this field (required) thanks, you're awesome! please check your email for a confirmation link. oops something is broken right now, please try again later. banner graphic: resource surveillance self-defense description: surveillance self-defense is eff's online guide to defending yourself and your friends from surveillance by using secure technology and developing careful practices. banner graphic: browser add-on privacy badger description: privacy badger is a browser add-on that stops advertisers and other third-party trackers from secretly tracking where you go and what pages you look at on the web. if an advertiser seems to be tracking you across multiple websites without your permission, privacy badger automatically blocks that advertiser from loading... issues covid- and digital rights many of our digital rights are impacted by covid- . now more than ever, eff is dedicated to ensuring that technology supports freedom, justice, and innovation for all the people of the world.surveillance. governments around the world are demanding extraordinary new surveillance powers that many hope will contain the virus’... the latest deeplinks blog by katharine trendacosta, cara gagliano | february , some answers to questions about the state of copyright in in all the madness that made up the last month of , a number of copyright bills and proposals popped up—some even became law before most people had any chance to review them. so now that the dust has settled a little and we have some better idea what the... deeplinks blog by jason kelley | february , online-only vaccine distribution will leave too many behind as the rollout of covid- vaccines has begun across the u.s., there have been numerous reports of people having trouble getting it—not just because of its limited availability, but also because some counties and states have chosen to require computer and internet access to sign up. this... deeplinks blog by jillian c. york, david greene | february , facebook's latest proposed policy change exemplifies the trouble with moderating speech at scale hateful speech presents one of the most difficult problems of content moderation. at a global scale, it’s practically impossible. that’s largely because few people agree about what hateful speech is—whether it is limited to derogations based on race, gender, religion, and other personal characteristics historically subject to hate, whether it... deeplinks blog by joe mullin | february , incoming biden administration officials should change course on encryption to have privacy and security in the digital world, encryption is an indispensable ingredient. without it, we’re all at risk of exploitation—by authoritarian governments, over-reaching police, nosy corporations, and online criminals.but for some years now, federal law enforcement has paid lip service to “cybersecurity,” while actually seeking to make... deeplinks blog by kit walsh | february , section ’s harm to security research shown by mixed decision in corellium case under traditional copyright law, security research is a well-established fair use, meaning it does not infringe copyright. when it was passed in , section of the digital millennium copyright act upset the balance of copyright law. since then, the balance has been further upset as it has been interpreted... deeplinks blog by kit walsh | february , no secret evidence in our courts if you’re accused of a crime, you have a right to examine and challenge the evidence used against you. in an important victory, an appeals court in new jersey agreed with eff and the aclu of nj that a defendant is entitled to see the source code of software... deeplinks blog by katitza rodriguez, veridiana alimonti | february , despite progress, metadata still under "second class" protection in latam legal safeguards this post is the fourth in a series about our new state of communications privacy laws report, a set of questions and answers about privacy and data protection in argentina, brazil, chile, colombia, mexico, paraguay, panama, peru, and spain. the... deeplinks blog by katitza rodriguez, veridiana alimonti | february , when law enforcement wants your private communications, what legal safeguards are in place in latin america and spain? this post is the third in a series about our new state of communications privacy laws report, a set of questions and answers about privacy and data protection in argentina, brazil, chile, colombia, mexico, paraguay, panama, peru, and spain. the... deeplinks blog by dave maass, matthew guariglia | february , san francisco takes small step to establish oversight over business association surveillance the san francisco board of supervisors last week voted unanimously in favor of requiring all special business districts—such as the union square business improvement district (usbid)—to bring any new surveillance plans to the board before adopting new technologies. the resolution—passed in the wake of an eff investigation, a... deeplinks blog by rebecca jeschke | february , can government officials block you on social media? a new decision makes the law murkier, but users still have substantial rights this blog post was co-written by eff legal fellow houston davidson.it’s now common practice for politicians and other government officials to make major policy announcements on twitter and other social media forums. that’s continuing to raise important questions about who can read those announcements, and what happens when... more updates back to top follow eff: twitter facebook instagram youtube flicker rss contact general legal security membership press about calendar volunteer victories history internships jobs staff issues free speech privacy creativity & innovation transparency international security updates blog press releases events legal cases whitepapers effector newsletter press press contact donate join or renew membership online one-time donation online shop other ways to give copyright (cc by) trademark privacy policy thanks javascript license information wayback machine see what's new with book lending at the internet archive a line drawing of the internet archive headquarters building façade. an illustration of a magnifying glass. an illustration of a magnifying glass. an illustration of a horizontal line over an up pointing arrow. upload an illustration of a person's head and chest. sign up | log in an illustration of a computer application window wayback machine an illustration of an open book. books an illustration of two cells of a film strip. video an illustration of an audio speaker. audio an illustration of a . " floppy disk. software an illustration of two photographs. images an illustration of a heart shape donate an illustration of text ellipses. more an icon used to represent a menu that can be toggled by interacting with this icon. about blog projects help donate an illustration of a heart shape contact jobs volunteer people search metadata search text contents search tv news captions search archived websites advanced search sign up for free log in the wayback machine requires your browser to support javascript, please email info@archive.org if you have any questions about this. the wayback machine is an initiative of the internet archive, a (c)( ) non-profit, building a digital library of internet sites and other cultural artifacts in digital form. other projects include open library & archive-it.org. your use of the wayback machine is subject to the internet archive's terms of use. hanzo solutions hanzo for ediscovery hanzo for compliance hanzo for investigations products hanzo hold hanzo dynamic capture company about news customers careers hanzo helps resources blog whitepapers + ebooks webinars contact book a demo there’s a better way to control enterprise data technology that ediscovery and compliance teams rely on to manage tomorrow's biggest data challenges today. our solutions ediscovery simplifying ediscovery for enterprise collaborative data compliance dynamic web archiving to satisfy sec, finra and state regulatory compliance requirements easily collect & control data from enterprise collaboration apps & dynamic websites archive interactive websites targeted data preservation insight & control defensible process archive interactive websites contextually collect dynamic and interactive data sources like collaboration applications and interactive websites so that you have the data you need in the format you want to meet ediscovery and compliance requirements. targeted data preservation why capture everything if you don’t have to? limit data risk and reduce costs by targeting preservation precisely with best practice scoping and granular hold controls. insight & control your unstructured data has a flow, links, and it has interactive elements. get better insights and control your risk with complete context. whether you’re conducting early case assessment of a slack or google workspace collection or are archiving complex and dynamic web content with monitoring and alerts, your team will have the insights they need to make better decisions and more timely responses. defensible process peace of mind that your captures are complete and highest fidelity is priceless. be fully confident in the defensibility of the collection and preservation of dynamic data via audit reports and worm storage that is commensurate with iso standards. the context & understanding of enterprise data you need. channel-based communications and collaboration have fueled productivity and have transformed the way we work and communicate. the information on these platforms is mission critical. problem is, these applications were not designed for discovery, investigations, nor compliance. collaboration and dynamic data is the new normal. so what are you going to do about it? security that instills peace of mind we focus on security and certifications so you don't have to. soc type certified architected to be secure from the ground up enterprise grade controls enterprise collaboration essentials learn what you need to know for managing enterprise collaboration data in this -page guide. this guide shows how to: identify the potentially relevant data using custodian or subject names, dates, and keywords capture the data and store it externally establish and follow a records retention schedule for the rest of your data review and use your captured data when email was first on the scene, everyone in the industry was trying to ‘paper-ize’ electronic documents so they could treat them like the documents they were used to. now, with collaboration apps, we’re trying to ‘email-ize’ their content so it’s more familiar. deanna blomquist, dish network ready to see hanzo in action? contact us for a demo and see why some of the most recognizable brands in the world trust hanzo to manage the challenges of modern enterprise ediscovery and compliance today. solutions ediscovery compliance investigations products hanzo hold hanzo dynamic capture company about news customers careers hanzo helps contact privacy resources blog whitepapers + ebooks webinars copyright . all rights reserved | privacy policy about internet security research group - internet security research group about us about isrg policies careers contact projects let's encrypt prio services radiant award blog contribute current sponsors and donors become a sponsor get involved donate now donate now about internet security research group mission our mission is to reduce financial, technological, and educational barriers to secure communication over the internet. history isrg was founded in may of to serve as a home for public-benefit digital infrastructure projects, the first of which was the let's encrypt certificate authority. isrg's founding directors were josh aas and eric rescorla. the group's founding sponsors and partners were mozilla, the electronic frontier foundation, the university of michigan, cisco, and akamai. structure isrg is a california public benefit corporation, and is recognized by the irs as a tax-exempt organization under section (c)( ) of the internal revenue code. our ein is - . funding isrg is proudly sponsored by a diverse group of organizations, from small businesses and other non-profits to fortune companies. we aim to set an example for how everyone interested in a more secure internet can work together to provide digital infrastructure for the public’s benefit. see this page for more on our sponsors. to learn more about isrg and the history of let's encrypt, please check out let's encrypt, an automated certificate authority to encrypt the entire web, which was published for the acm conference on computer and communications security (ccs) in . board of directors josh aas (isrg) richard barnes (cisco) vicky chin (mozilla) jennifer granick (aclu) aanchal gupta (independent) j. alex halderman (univ. of michigan) pascal jaillon (ovh) david nalley (amazon) erica portnoy (eff) christine runnegar (internet society) technical advisory board rich salz (akamai) joe hildebrand (mozilla) jacob hoffman-andrews (eff) yueting lee (facebook) j.c. jones (mozilla) russ housley (independent) ryan hurst (google) stephen kent (independent) karen o'donoghue (internet society) ivan ristic (independent) privacy policy | trademark policy © internet security research group. all rights reserved. market st, pmb , san francisco, california - skip to content menu home services services overview site audit service training and education vocabulary extension blog about founder contact structured data / schema.org site audit service launched check out our new fixed price service to find out how your site is performing! find out more data value liberating business, social, economic, enterprise, and financial value from data of all types delivers benefits to the media, governments, commerce, academia and individuals. data liberate, and its founder richard wallis, focus on introducing, simplifying, and demystifying, these often superficially complex techniques and technologies. advice, guidance, evaluation, training, consultancy services, writing, podcasting, conference keynotes and presentations are some of the ways that data liberate can help you and your organisation identify and release value from your data, within the enterprise and on the public web of data recent postings from the blog… library metadata evolution: the final mile posted on may , may , by richard wallis when schema.org arrived on the scene i thought we might have arrived at the point where library metadata could finally blossom; adding value outside of library systems to help library curated resources become first class citizens, and hence results, in the global web we all inhabit. but as yet it has not happened. read more ... posted in bibframe, data liberate, knowledge graph, libraries, schema.org, structured datatagged bibframe schema.org, libraries, library of congress comment something for archives in schema.org posted on april , by richard wallis the recent release of the schema.org vocabulary (version . ) includes new types and properties, proposed by the w c schema architypes community group, specifically target at facilitating the web sharing of archives data to aid discovery. when the group, which i have the privilege to chair, approached the challenge of building a proposal to make schema.org useful for archives, it was identified that the vocabulary could be already used to describe the things & collections that you find in archives. what was missing was the ability to identify the archive holding organisation, and the fact that an item is being held … read more ... posted in archives, data publishing, linked data, schema.org, webtagged archives, schema.orgleave a comment bibframe – schema.org – chocolate teapots posted on august , august , by richard wallis in a session at the ifla wlic in kuala lumpur – my core theme being that there is a need to use two [linked data] vocabularies when describing library resources — bibframe for cataloguing and [linked] metadata interchange — schema.org for sharing on the web for discovery. read more ... posted in bibframe, libraries, linked data, marc, schema.org, structured datatagged libraries, linked data, schema.orgleave a comment schema.org introduces defined terms posted on june , june , by richard wallis do you have a list of terms relevant to your data? things such as subjects, topics, job titles, a glossary or dictionary of terms, blog post categories, ‘official names’ for things/people/organisations, material types, forms of technology, etc. read more ... posted in data publishing, schema.org, uncategorizedtagged #linkeddata, schema.org, seo comments schema.org significant updates for tourism and trips posted on june , june , by richard wallis the latest release of schema.org ( . ) includes some significant enhancements for those interested in marking up tourism, and trips in general. for tourism markup two new types touristdestination and touristtrip have joined the already useful touristattraction read more ... posted in data publishing, knowledge graph, schema.org, seo, tourism, webtagged schema.org, seo, tourismleave a comment the three linked data choices for libraries posted on may , may , by richard wallis we are [finally] on the cusp of establishing a de facto linked data approach for libraries and their system suppliers – not there yet but getting there. we have a choice between bibframe . , schema.org, linky marc and doing nothing. read more ... posted in bibframe, data publishing, libraries, linked data, marc, schema.org, semantic web, structured datatagged bibframe, libraries, linked data, marc, schema.ogleave a comment structured data: helping google understand your site posted on november , may , by richard wallis add schema.org structured data to your pages because during indexing, we will be able to better understand what your site is about. read more ... posted in data publishing, google, linked data, schema.org, structured data schema.org for tourism posted on september , june , by richard wallis these touristattraction enhancements have significantly improved the capability for describing tourist attractions and hopefully enabling more tourist discoveries read more ... posted in data publishing, schema.org, seo, tourismtagged linkeddata, schema.org, seo, tourism comment schema.org: describing global corporations local cafés and everything in-between posted on september , may , by richard wallis there have been discussions in schema.org about the way organizations their offices, branches and other locations can be marked up; they exposed a lack of clarity in the way to structure descriptions of organizations and their locations, offices, branches , etc. to address that lack of clarity i thought it would be useful to share some examples here. read more ... posted in data publishing, schema.org, seotagged schema.og, seo a discovery opportunity for archives? posted on june , june , by richard wallis so why am i now suggesting that there maybe an opportunity for the discovery of archives and their resources? read more ... posted in archives, linked data, open data, schema.orgtagged #linkeddata, archives, schema.orgleave a comment follow twitter linkedin rss client engagements developer advocate at working on: schema.org/bibframe education, consultancy, & implementation for: technology evangelist and advisor for: schema.org consultancy for europeana fibo schema.org working group schema.org consultancy for the british library schema.org consultancy for: w c community groups bibframe schema.org - chair schema bib extend - chair schema architypes - chair tourism structured web data - co-chair schema course extension financial industry business ontology sport schema tweets data liberate @dataliberate #structureddata https://t.co/cch oectub (about days ago) data liberate @dataliberate post: library metadata evolution: the final mile? https://t.co/cibwjuf o many of the elements of this process are… https://t.co/a ddjxvpmz (about days ago) data liberate @dataliberate post: something for archives in https://t.co/ kzbg oj archives focussed additions in new #schema.org ( . ) versi… https://t.co/ qxispxmm (about days ago) follow @dataliberate speaking calandar archives may april august june may november september june september august march february august june april december september april march february september june may april march december november august june april march february january november october september august july march january december january copyright © •fabulous fluid by catch themes search for: search scroll up home services services overview site audit service training and education vocabulary extension blog about founder contact this site uses cookies: find out more.ok wayback machine see what's new with book lending at the internet archive a line drawing of the internet archive headquarters building façade. an illustration of a magnifying glass. an illustration of a magnifying glass. an illustration of a horizontal line over an up pointing arrow. upload an illustration of a person's head and chest. sign up | log in an illustration of a computer application window wayback machine an illustration of an open book. books an illustration of two cells of a film strip. video an illustration of an audio speaker. audio an illustration of a . " floppy disk. software an illustration of two photographs. images an illustration of a heart shape donate an illustration of text ellipses. more an icon used to represent a menu that can be toggled by interacting with this icon. about blog projects help donate an illustration of a heart shape contact jobs volunteer people search metadata search text contents search tv news captions search archived websites advanced search sign up for free log in the wayback machine requires your browser to support javascript, please email info@archive.org if you have any questions about this. the wayback machine is an initiative of the internet archive, a (c)( ) non-profit, building a digital library of internet sites and other cultural artifacts in digital form. other projects include open library & archive-it.org. your use of the wayback machine is subject to the internet archive's terms of use. is the ssliverse a safe place? peter eckersley jesse burns @ eff @ isec partners special thanks to chris palmer @ eff for help with data wrangling + publication so, https will save the web but... encryption security ≤ ability to identify the other party certificate authorities (cas) say ”this key belongs to mail.google.com” (browsers trust the cas) https uses certificates : vulnerabilities due to ca mistakes : evidence of governments compelling cas generally: too many trusted parties! we are afraid of cas because: also afraid of x. designed in s by the itu (!), before http (!!!) + extremely flexible & general - extremely flexible & general - extremely ugly - history of implementation vulnerabilities x. : security via digital paperwork x. certs can (and do) contain just about anything what to do about it? . write alternative browser code? . study ca behaviour and detect problems is hard → let's do first scanned all allocated ipv space (port ) built a system for analysing the data initial results presented at defcon eff ssl observatory this talk: brief overview of what we reported at defcon new results from a re-scan a tutorial on using our datasets design for a decentralised observatory . m ips were listening on port . m started an ssl handshake . +m used valid cert chains . +m distinct valid leaves size of the ssliverse lots of cas! , cas trustable by microsoft or mozilla , disinct issuer strings organisations noteworthy subordinate cas u.s. department of homeland security u.s. defence contractors cnnic, (why debate their root ca?) etisalat gemini observatory exposure to many jurisdictions cas are located in these ~ countries: ['ae', 'at', 'au', 'be', 'bg', 'bm', 'br', 'ca', 'ch', 'cl', 'cn', 'co', 'cz', 'de', 'dk', 'ee', 'es', 'eu', 'fi', 'fr', 'gb', 'hk', 'hu', 'ie', 'il', 'in', 'is', 'it', 'jp', 'kr', 'lt', 'lv', 'mk', 'mo', 'mx', 'my', 'nl', 'no', 'pl', 'pt', 'ro', 'ru', 'se', 'sg', 'si', 'sk', 'tn', 'tr', 'tw', 'uk', 'us', 'uy', 'ww', 'za'] vulnerabilities ~ , servers use broken keys ~ had valid ca signatures, including: diplomatie.be yandex.ru lawwebmail.uchicago.edu (now fixed/expired) other whackiness certificates that were and were not ca certs lots of certs for ”localhost”, ”mail” and various ips violations of extended validation rules also, we've published the data, so you can do further research on it the data available from https://www.eff.org/observatory gb download / gb mysql db ~ hours to import on a fast pc definitely a version release :) the database schema is fairly baroque. in part: blame x. in part: only . of us but let's show you how to use it! get the torrent file from https://www.eff.org/observatory bittorrent ssl-database-paths-fixed-ext.sql.lzma.torrent mysqladmin -u root -p create observatory unlzma -c ssl-database-paths-fixed-ext.sql.lzma | mysql -u root -p ( ~ hours later ) now you have a database of certs valid_certs } indexed by certid or all_certs } fingerprint (sha ) names } common names + subject anames } alternative names -> certids certs_seen: maps (time, ip) -> fingerprint (also stores chain order) main db tables some simple examples: select rsa_modulus_bits, count(*) from valid_certs group by rsa_modulus_bits order by cast(rsa_modulus_bits as decimal); +------------------+----------+ | rsa_modulus_bits | count(*) | +------------------+----------+ | | | | | | | | | | | | | | | | | | | | | | ... | ... | +------------------+----------+ select `signature algorithm`, count(*) from valid_certs where startdate > ” ” group by `signature algorithm`; +--------------------------+----------+ | signature algorithm | count(*) | +--------------------------+----------+ | md withrsaencryption | | | sha withrsaencryption | | | sha withrsaencryption | | | sha withrsaencryption | | +--------------------------+----------+ select distinct issuer from valid_certs where stardate > ” ” and `signature algorithm`= " md withrsaencryption"; +------------------------------------------------------------------------+ | issuer | +------------------------------------------------------------------------+ | o=ministere de la justice, cn=autorite de certification serveurs | | c=us, o=anthem inc, ou=ecommerce, cn=anthem inc certificate authority | +------------------------------------------------------------------------+ (fortunately, these cas don't robo sign) caveats... some fields (name, ip) in the _certs tables are correct but not comprehensive select count(distinct ip) from all_certs -- , , select count(distinct ip) from seen -- , , (the former undercounts due to certs seen on multiple ips) some columns have unintuitive semantics; moz_valid, ms_valid are the outputs of: openssl verify -capath -untrusted cert ; eg: yes yes self-signed: ok self-signed: /cn=sw-mhs-ser- - ./unstructuredname=sw-mhs-ser- - . error at depth lookup:certificate has expired ok yes yes self-signed: in certificate chain self-signed: ok no: 'stdin: /c=us/st=default state/l=default locality/o=american power conversion corp/ou=internally generated certificate/cn=za \\n error at depth lookup:unable to get local issuer certificate\\n'none so: select count(*) from valid_certs where moz_valid=”yes” → , , select count(*) from valid_certs where not moz_valid=”yes” → , select count(*) from valid_certs where not ms_valid=”yes” → , firefox and ie cache intermediate ca certificates... so openssl can't necessarily say whether a cert is valid in these browsers (!!!) even worse... ”transvalidity” valid, but only if the browser cached the right intermediate ca certs first → we catch all / almost all transvalid certs explaining transvalidity.py first, find invalid certs where a plausible, valid intermediate cert was seen somewhere in the ssliverse: select certs .path, certs .id, valid_certs.path, certs .fingerprint, certs .fetchtime from certs join valid_certs on certs .issuer = valid_certs.subject and ( (certs .`authority key identifier:keyid` is null and valid_certs.`subject key identifier` is null) or certs .`authority key identifier:keyid` = valid_certs.`subject key identifier` ) where not certs .valid and (locate("unable to get local issuer certificate", certs .moz_valid) or locate("unable to get local issuer certificate", certs .ms_valid) ) group by certs .fingerprint, valid_certs.path note: some variable names were simplified in this query: certs is an example raw input certs table, authority key ids have longer column names transvalidity.py (ct'd) once we have some missing, valid, possibly determinative ca certs, we re-run openssl: openssl verify -capath -untrusted cert results go in the ”transvalid” column select count(*) from valid_certs where transvalid="yes" → , tranvalid certs boolean valid = ( moz_valid == ”yes” or ms_valid == ”yes” or transvalid == ”yes”) validity in general more examples of the dataset at work... which root cas created the most subordinate cas? subordinatetracking.py for each root cert: select certid, subject, issuer, `subject key idenfier` from valid_certs where issuer = and locate(”true”, `x v basic constraints:ca`) and `x v authority key identifier:keyid` = (which may be null) (and recurse) results: top roots by ca proliferation . c=de, cn=deutsche telekom root ca sub-cas ( , leaves) . c=us, cn=gte cybertrust global root sub-cas ( , leaves) . c=se, cn=addtrust external ca root sub-cas ( , leaves) . c=be, cn=globalsign root ca sub-cas ( , leaves) . c=us, cn=entrust.net secure server certification authority sub-cas ( , leaves) . c=fr, o=pm/sgdn, ou=dcssi, cn=igc/a... sub-cas ( leaves) . ou=valicert class policy validation authority sub-cas ( , leaves) . o=verisign, inc, ou=class public primary certification authority sub-cas ( , leaves) great idea: certs become reliable again http://cabforum.org/ev_certificate_guidelines.pdf stricter rules like: owners exclusively own domains use relatively strong keys identifiable owners audits extended validation http://cabforum.org/ev_certificate_guidelines.pdf special oid per ca chromium source documents: ev_root_ca_metadata.cc extended validation ev's per ca oids ev hints via ugly where clause `x v authority key identifier` is null and (locate(" . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . . . . . . :",`x v certificate policies:policy`)or locate(" . . . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . . :", `x v certificate policies:policy`) or locate(" . . . . . . . . :", `x v certificate policies:policy`)) extended validation problems in general browser sop not super compatible same cas accountability & auditing? certificate policy statements , served over http not a violation finding ev problems with the observatory about , ev certs this time with issuers not all unique, not all really used. extended validation problems found by the observatory rfc- addreses unqualified names... localhost?!? weak keys long expiration ev crypto policy violations issuers signed valid, ev certs with bit rsa keys that expire after dec , but ”subscriber certificates whose validity period ends after dec ” must be bits finding ev problems with the observatory wildcard certs for *.domain.com are not allowed in ev certs. cybertrust certs: *.xlgroup.com *.amos.hosting.accenture.com ev certs for unqualified names still observe ev certs for: ”webmail”, ”zinc”, ”localhost” major class ev cas like verisign (revoked after defcon) ev certs for private ips globalsign signed an ev cert with a name for an rfc ip – i.e. . .x.x said they changed policy in & audited. last summer we found one they missed, and we just noticed another... ev certs for private ips... https://giftcard.ilcusys.com/ icul service corporation, "helping credit unions compete", illinois credit union league... with a "mcafee secure" badge. https://giftcard.ilcusys.com/ ev certs for private ips bit ev cert (!!!) https://suppliers.tnb.com thomas & betts corporation of memphis tn convinced a ca to give them a bit rsa cert in september... it expires in . finding the bit ev cert bit ev cert future work . release revised and neater datasets . a decentralised observatory decentralised observatory objectives . detect mitm attacks  even if only the victim gets the cert . protect user privacy  never know who looks at which site decentralised observatory design . user has tor running  but not currently in use . send raw certs to observatory  asynchronosly  via tor for anonymity . warn users about phishy ca signatures?  maybe not until a few seconds later :(  better late than never decentralised observatory the code is in progress conclusion join us eff.org/observatory questions: ssl-survey@eff.org slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide slide the software sustainability institute | software sustainability institute skip to main content pre header link rss twitter wakelet blog news events contact subscribe primary links about about us manifesto staff fellows advisory board funders partners brand guidelines how to cite us get involved data management plan contact programmes and events fellowship programme open call for projects research software healthcheck carpentry programmes research software engineers collaborations workshops research software camps past events resources get up to speed! online sustainability evaluation open evidence bank ref : guidance for software outputs guides top tips publications case studies t-shirts videos close primary links about about us manifesto staff fellows advisory board funders partners brand guidelines how to cite us get involved data management plan contact programmes and events fellowship programme open call for projects research software healthcheck carpentry programmes research software engineers collaborations workshops research software camps past events resources get up to speed! online sustainability evaluation open evidence bank ref : guidance for software outputs guides top tips publications case studies t-shirts videos other pages rss twitter wakelet blog news events contact subscribe search menu the software sustainability institute cultivates better, more sustainable, research software to enable world-class research. we help people build better software, and we work with researchers, developers, funders and infrastructure providers to identify key issues and best practice in scientific software. the software sustainability institute blog the institute's highlights from and upcoming opportunities we take a look back at some of the highlights at the institute from last year, and some of the exciting things happening in . read more a farewell to rses: reflecting on our progress and remaining barriers ssi deputy director simon hettrick reflects on the progress of research software engineers as he steps down as a trustee of the society of research software engineering. read more highlights of the fellowship programme launch webinar the ssi fellowship programme launch webinar took place on january and provided information about the institute, the fellowship programme, the application process and considerations with regard to covid- . read more see all news registration open for research software camp workshop: sustainable software development for researchers registration is open for the workshop 'sustainable software development for researchers' or 'bring your own script and make it ready for publication' by carina haupt, head of group software at the german aerospace centre. the workshop is part of our research software camp on research accessibility. read more future forum workshop on code security there will be another future forum for the secrious project, heriot-watt university, on saturday the february - pm. read more call for lightning talks at collaborations workshop collaborations workshop (cw ) will take place online from tuesday march to thursday april and the call for lightning talks is now open! read more see all events research software camp: research accessibility our inaugural online research software camp on research accessibility will take place from february to march . read more collaborations workshop collaborations workshop (cw ) takes place from tuesday, march to thursday, april online. the software sustainability institute’s collaborations workshop series brings together researchers, developers, innovators, managers, funders, publishers, policy makers, leaders and educators to explore best practices and the future of research software. read more collaborations workshop collaborations workshop (cw ) takes place from tuesday, st march to thursday, nd april online. the software sustainability institute’s collaborations workshop series brings together researchers, developers, innovators, managers, funders, publishers, leaders and educators to explore best practices and the future of research software. read more see all "it adds development time, doing things right, but it's worth it. our code is miles away from where it was. [the work has] really made a profound change to the way we do coding, even down to the way i write lines of code. we now understand the complexity involved in making things run smoothly – it's really been a very good learning experience" - prashant valluri, tpls subscribe every friday we send out a digest of the week's news and blog posts. email address ignore except where otherwise noted, content on this site is licensed under a creative commons attribution non-commercial . license. © - the university of edinburgh on behalf of the software sustainability institute. privacy policy website accessibility we use cookies on our website to support technical features that enhance your user experience. we also use analytics & advertising services. to opt-out click for more information. i've read it more information ai lam search this site ai lam home events working groups ai registry about ai lam home events working groups ai registry about more home events working groups ai registry about ai lam is an international, participatory community focused on advancing the use of artificial intelligence in, for and by libraries, archives and museums. next open community call is on february with lightning talks on named entity recognition corona: futurs fantastiques in paris is postponed until december - ! community calls ai lam holds open community calls on the third tuesday of each month at am pt | am et | pm uk | pm cet. join the ai lam slack or the google group for connection information. working group structure we're organizing! ai lam will start organizing working groups in the first quarter of . have a look at the "working groups" section to get more information. it is up to you now! ai tapas videos small plates of practical ai knowledge for lam practitioners. coming soon... relevant activities we have heard about ff: known ai projects in lams send an email to contact@ai lam.org if something is missing or wrong connect with the community! google group mailing list google drive slack (chat) view | join report abuse page details page updated google sites report abuse digital library assessment - dlf clir fellowships & grants dlf publications clir global join give clir programs digital library federation dlf forum digital library of the middle east digitizing hidden special collections and archives recordings at risk mellon fellowships for dissertation research leading change institute dlf eresearch network postdoctoral fellowship program dlf digital library federation toggle navigation about about dlf our staff our members governance dlf code of conduct events dlf year-round forum: online past forums dlf forum news social events checklist child care fund resources dlf organizers’ toolkit blog and news #dlfcontribute series digitizing special formats dlf cost calculator dlf jobs board groups dlf working groups clir/dlf affiliates dlf membership cohorts get involved with groups opportunities grants and fellowships authenticity project dlf community calendar data curation postdocs community/capacity awards post a job/find a job contact get in touch stay connected join dlf our members benefits dlf digital library federation toggle navigation about about dlf our staff our members governance dlf code of conduct events dlf year-round forum: online past forums dlf forum news social events checklist child care fund resources dlf organizers’ toolkit blog and news #dlfcontribute series digitizing special formats dlf cost calculator dlf jobs board groups dlf working groups clir/dlf affiliates dlf membership cohorts get involved with groups opportunities grants and fellowships authenticity project dlf community calendar data curation postdocs community/capacity awards post a job/find a job contact get in touch stay connected join dlf our members benefits digital library assessment home / dlf groups / digital library asse... dlf seeks to facilitate conversations and activities related to digital library assessment. to take part, join the digital library assessment interest group (dlf aig) discussion forum, which is open to anyone interested in learning about or collaborating on the improvement of this work! more background, working groups, and projects: research and cultural heritage institutions are, as a matter of course, providing online access to converted and born-digital scholarly and cultural content. as the amount of that content continues to grow, there is an increased need to standardize our assessment efforts in a more strategic way. this interest group is working on methods to measure the impact of digital collections; developing areas of commonality and benchmarks in how we measure collections across various platforms; understanding cost and benefit of digital collections; and exploring how can we best collect, analyze, communicate, and share such information effectively across our various stakeholders—from collection managers to scholars. the dlf aig was formed in as an informal interest group within the larger dlf community. the group meets during the dlf forum to share problems, ideas, and solutions. the group also has a dedicated google group, dlf-supported wiki, a slack workspace (sign up here), and project documentation available in the open science framework. the group is open to anyone interested in learning about or collaborating on the improvement of digital library assessment. current working groups are focusing on tools and best-practices documents to: support cost assessment develop guidelines and best practices in user and usability studies create a toolkit for assessing content reuse build metadata quality review recommendations and explore the cultural assessment of digital libraries. the assessment interest group is actively engaged in creating products and literature to assist with the assessment of digital libraries. here are white papers, annotated bibliographies, project documentation, and additional resources developed by dlf aig working groups: opportunities for feedback and contributions the digitization cost calculator project (contribute data) best practices for developing personas in digital libraries ( ) journey mapping ( ) metadata application profile clearinghouse ( ) (contribute here) annotated bibliography for cultural assessment of digital collections ( ) (contribute here) metadata assessment literature review & environmental scan ( ) (contribute here) assessment resources metadata assessment tools wiki ( ) setting a foundation for assessing content reuse: a white paper from the developing a framework for measuring reuse ( ) metadata assessment framework and guidance ( ) developing a framework for measuring reuse of digital objects ( ) web analytics annotated resource list ( - ) (contribute directly to the document) surveying the landscape: use and usability assessment of digital libraries ( ) dlf user studies in digital libraries bibliography ( ) best practices for google analytics in digital libraries ( ) altmetrics and analytics for digital special collections and institutional repositories ( ) guidelines for citing library-hosted, unique digital assets ( ) digitization projects: processes and definitions ( ) what's the dlf? networked member institutions and a robust community of practice—advancing research, learning, social justice, & the public good through the creative design and wise application of digital library technologies contact clir+dlf north union street suite -pmb alexandria, va e: info@diglib.org elsewhere twitter facebook linkedin youtube email rss community calendar geo lib camp february , – february , online https://www.eventbrite.com/e/geo libcamp- -tickets- nycdh week february , – february , https://nycdh.org/dhweek/ college art association annual conference february , – february , online https://www.collegeart.org/programs/conference/conference more events & links here. on twitter rt @clirnews: celebrating #archiveshashtagparty #archivesblackeducation and #blackhistorymonth with "our story: photographs and publication…yesterday have a spare minute? make sure to complete our question survey about attending fall events today! https://t.co/h mxokpxqz days ago rt @lellyjz: really looking forward to this! https://t.co/ o gkknzqq days ago unless otherwise indicated, content on this site is available for re-use under cc by-sa . license clir skip to content open toolbar accessibility tools increase text decrease text grayscale high contrast negative contrast light background links underline readable font reset think hazard toggle navigation thinkhazard! user feedback data map about faq share english english français español thinkhazard! identify natural hazards in your project areaand understand how to reduce their impact river flood urban flood coastal flood earthquake landslide tsunami volcano cyclone water scarcity extreme heat wildfire mayor dilkens pushing for windsor to join safety app iheartradio for you my stations live radio podcasts artist radio news features events videos contests iheartradio contact us logo am - °c facebook twitter instagram sms* user login edit profile log out shows news audio covid- updates newsletters contests events news poll mayor dilkens pushing for windsor to join safety app teresinha medeiros am file photo windsor's mayor is in talks to try and make the city the first in canada to use a new tool to improve public safety. the u.s company ring inc. which makes home camera system has a 'neighbours app' that allows people to join the ring network which is connected to police. if there is a crime in a certain neighbourhood, police can notify people with the cameras system that they require the footage. "they would then identify the area, push a button and it pushes out a request to everyone who is on the system saying police are requesting camera footage on this date, between this time, do you authorize the release of the camera footage," says mayor drew dilkens. speaking on am 's the lynn martin show, mayor dilkens says people can approve or reject the request. dilkens says it would save police time and resources instead of going door-to-door to get camera footage. "you can imagine how resource intensive it is for police to go business to business trying to get camera footage," he says. dilkens emphasizes the talks are in the preliminary stages, but he believes it is a tool worth exploring. just new cases of covid- reported sunday there are currently people in hospital being treated for the virus black history month marked with challenge to buy from black-owned businesses black business can will be spotlighting local black businesses in an effort to raise awareness more than $ k raised for alzheimers society of windsor-essex local business ground effects pledged to match donations up to $ , audio the afternoon news - canadian acting icon christopher plummer has dies the afternoon news - part shortage at windsor assembly load more load more upcoming events sending love with hospice dine yqg rotary club catch the ace changing lives together / in honour colours of change gala load more load more group element am footer contact us news shows contests news audio events logo am am contact information ouellette avenue windsor on n x l phones studio/contest lines - - news tips - - email newscentre@am cklw.com website facebook twitter instagram sms* user login edit profile log out just new cases of covid- … just new cases of covid- reported sunday group footer en iheartradio facebook twitter youtube instagram iheartradio for you my stations live radio podcasts news features events contests contact iheartradio queen street west toronto on m v z phone websites iheartradio terms and conditions iheartradio privacy policy contact us political ad registry get the apps google play apple store tweets to @realdonaldtrump; how many fucks are there to give? | nick ruest search nick ruest home c.v. posts presentations publications projects visualizations music contact tweets to @realdonaldtrump; how many fucks are there to give? jun , min read i’ve been collecting tweets to @realdonaldtrump since june . in my most recent time pulling together, and deduping the dataset i asked myself, “i wonder how many occurrences of ‘fuck’ are in the dataset.” or, how many fucks are there to give? well… the data is updated by running a query on the standard search api every five days. $ twarc search 'to:realdonaldtrump' --log donald_search_$date.log > donald_search_$date.jsonl which yields something like this every five days. ... donald_search_ _ _ .jsonl donald_search_ _ _ .log donald_search_ _ _ .jsonl donald_search_ _ _ .log donald_search_ _ _ .jsonl donald_search_ _ _ .log donald_search_ _ _ .jsonl donald_search_ _ _ .log donald_search_ _ _ .jsonl donald_search_ _ _ .log donald_search_ _ _ .jsonl donald_search_ _ _ .log donald_search_ _ _ .jsonl donald_search_ _ _ .log donald_search_ _ _ .jsonl donald_search_ _ _ .log donald_search_ _ _ .jsonl donald_search_ _ _ .log ... periodically, i cat all the jsonl files together, and then deduplicate them with deduplicate.py. so, this currently leaves us with , , tweets to work with. if you want to follow along, you can grab the most recent set of tweet ids from here. then “hydrate” them like so: $ gunzip to_realdonaldtrump_ _ids.txt.gz $ twarc hydrate to_realdonaldtrump_ _ids.txt > .jsonl this will probably take quiet a while since there are potentially , , tweets to hydrate. in the end, you’ll end up with a jsonl file around g. once we have our full dataset, first thing we’ll do is remove all of the retweets with noretweets.py, giving us just original tweets at @realdonaldtrump. $ noretweets.py .jsonl > _no_retweets.jsonl this brings us down to , , unique tweets. your number will probably be less if you’re working with a hydrated dataset because deleted tweets, suspended accounts, and protected accounts will not have tweets hydrated. $ wc -l _no_retweets.jsonl over the time of collecting, some of the twitter apis and fields changed slightly (extended tweets, and character tweets). for us, this means the “text” of our tweets can reside in two different attributes; text or full_text. so, we need to extract out the text. let’s use tweet_text. $ tweet_text.py _no_retweets.jsonl >| _tweet_text.txt now that we have just the text, we can count how many fucks there are with grep and wc! $ grep -i "fuck" _tweet_text.txt | wc -l there are , , fucks to give! that’s a fuck to tweet ratio of . %. … … … for some more fun, let’s take the last lines of the our new text file, and make an animated gif out of it. first, let’s get our text: $ grep -i "fuck" _tweet_text.txt > fucks.txt $ tail -n fucks.txt > _fucks.txt then let’s create a little bash script. #!/bin/bash index= cat /path/to/ _fucks.txt | while read line; do let "index++" pad=`printf "% d" $index` convert -size x -background black -weight -fill white -gravity center -font ubuntu caption:"$line" /path/to/images/$pad.png done cd /path/to/images convert -monitor -define registry:temporary-path=/tmp -limit memory gib -limit map gib -delay *.png -loop _fucks.gif give it a filename, then make it executable, and run it! in the end, you’ll end up with something like this: twitter donald trump fucks webarchiving art nick ruest associate librarian related twitter wordcloud pipeline the world is a beautiful and terrible place thumbnails in warclight twitter bots twitter datasets and derivative data cc-by · powered by the academic theme for hugo. cite × copy download wayback machine see what's new with book lending at the internet archive a line drawing of the internet archive headquarters building façade. an illustration of a magnifying glass. an illustration of a magnifying glass. an illustration of a horizontal line over an up pointing arrow. upload an illustration of a person's head and chest. sign up | log in an illustration of a computer application window wayback machine an illustration of an open book. books an illustration of two cells of a film strip. video an illustration of an audio speaker. audio an illustration of a . " floppy disk. software an illustration of two photographs. images an illustration of a heart shape donate an illustration of text ellipses. more an icon used to represent a menu that can be toggled by interacting with this icon. about blog projects help donate an illustration of a heart shape contact jobs volunteer people search metadata search text contents search tv news captions search archived websites advanced search sign up for free log in the wayback machine requires your browser to support javascript, please email info@archive.org if you have any questions about this. the wayback machine is an initiative of the internet archive, a (c)( ) non-profit, building a digital library of internet sites and other cultural artifacts in digital form. other projects include open library & archive-it.org. your use of the wayback machine is subject to the internet archive's terms of use. spatialchat we're sorry but spatialchat doesn't work properly without javascript enabled. please enable it to continue. sorry, this browser doesn't support spatialchat. please use the latest version of google chrome. wayback machine see what's new with book lending at the internet archive a line drawing of the internet archive headquarters building façade. an illustration of a magnifying glass. an illustration of a magnifying glass. an illustration of a horizontal line over an up pointing arrow. upload an illustration of a person's head and chest. sign up | log in an illustration of a computer application window wayback machine an illustration of an open book. books an illustration of two cells of a film strip. video an illustration of an audio speaker. audio an illustration of a . " floppy disk. software an illustration of two photographs. images an illustration of a heart shape donate an illustration of text ellipses. more an icon used to represent a menu that can be toggled by interacting with this icon. about blog projects help donate an illustration of a heart shape contact jobs volunteer people search metadata search text contents search tv news captions search archived websites advanced search sign up for free log in the wayback machine requires your browser to support javascript, please email info@archive.org if you have any questions about this. the wayback machine is an initiative of the internet archive, a (c)( ) non-profit, building a digital library of internet sites and other cultural artifacts in digital form. other projects include open library & archive-it.org. your use of the wayback machine is subject to the internet archive's terms of use. dlf dlf digital library federation metadata during covid this post was written by members of the metadata working group, a subgroup of dlf&# ;s assessment interest group. digital collections work has changed in a number of ways during the covid- pandemic. for many libraries and archives, this has meant working remotely and shifting toward tasks that can be done online. within the dlf aig metadata working group, members have discussed a number of ways that organizations have chosen to increase capacity for metadata, transcription, and other tasks related to digital collections as a way of providing work for employees who would normally work in public-serving positions. this post documents some of those projects and activities. university of north texas at the university of north texas, our digital collections use a web-based metadata editing interface and we can add as many users as needed. when the stay-at-home order went into effect right after spring break, many of our library staff members (including full-time librarians/staff and part-time student workers) were no longer able to do their regular jobs and we offered metadata as an alternative. we added about new editors to our system in march . additionally, we added some quickly-drafted documentation to steer people toward easy metadata projects and known issues that require clean-up (like fixing name formatting). to keep oversight manageable, new editors were still attached to their own departments (or assigned to one that needed help), with a central contact person for each department and a specific sub-set of projects. our team of developers rushed an overhaul of the event tracking system that documents who is editing and what records they are changing so that managers could more easily verify if workers were editing when they said they were working. tracking edits has also let us measure how significantly overall editing has increased. multiple times since this started, we have had at least one editor working during every hour of the day. having so many relatively-untrained editors has resulted in a large number of issues that will need to be reviewed, but we have tools built into our system to help identify those issues and have added them to our ongoing list of things to fix. overall, this was still an extremely positive experience since the increase in editors allowed significant progress or completion of work that would not have been done otherwise. &# ; hannah tarver university of utah marriott library at the university of utah, the covid- pandemic pivot to remote work prompted the launch of transcription projects, both with handwritten materials from special collections and newspaper ocr correction. this includes the transcription of , employee records by our digital operations student employees which resulted in the complete transcription of the kennecott miner records collection. we are also using omeka classic with the scripto plug-in as the platform for manuscript transcription projects and are able to find valuable work for people to engage in when they couldn’t physically be at the library. in addition, we developed a born-digital crowdsourced digital collection, the utah covid- digital collection designed to capture what is currently happening during this unusual time. we’ve gotten a great response from the university and larger utah communities, with over contributions so far available in the digital library. the covid- digital collection has enabled us to build new partnerships and provided the library with outreach opportunities. an article detailing the project is forthcoming in a special issue of the journal digital library perspectives. &# ; anna neatrour utah state archives after starting with from the page a few months earlier, moving staff and volunteers to transcription and indexing projects proved to be successful. contributors finished a historical court case (and now working on a second one) and a year’s worth of birth certificates in only a few months using the web-based interface that integrates with contentdm digital collections. with a built-in notes feature, questions can be asked and answered directly on a document’s page, which will then be exported along with the rest of the metadata. we are now preparing to open up the birth certificate indexing to the general public with additional training materials. in addition, new digital collections have been published, even with metadata developed remotely, using tools like google sheets for input and then converting to delimited text files for import. &# ; gina strack university of texas at austin at the start of march, the university of texas libraries collections portal, the public-facing search and discovery interface for our digital asset management system (dams), included approximately , items. shortly after, the ut-austin campus closed and many staff members from the libraries’ first-line customer service, acquisitions and cataloging units found their roles pivoting to create metadata remotely for our dams system. collection curators within ut libraries created large-scale digital projects to help ensure continued remote work and to utilize this unusual time to turn their focus to projects that had been placed on the back burner due to more pressing obligations. our digital asset management system coordinator and staff from our preservation and digital stewardship unit created flexible pathways to support these projects and to ensure successful ingests into the dams. staff at the architecture &# ; planning library and the alexander architectural archives, the nettie lee benson latin american collection, and the perry-castañeda library map collection dedicated themselves to ingesting and describing large amounts of digital items, increasing our total number of items available online to over , by september. digital objects newly available online as a result of this unprecedented, organization-wide collaborative effort include over , digitized architectural drawings and images, historic books from the benson rare book collection and primeros libros de las américas, and , scanned maps. the university of texas libraries documented the experience and provided a more detailed explanation of our dams in texlibris. &# ; mandy ryan colgate university colgate university’s special collections and university archives (scua) is documenting the colgate community’s experiences and stories of covid- . digital contributions can be submitted at any time via a google form and may be added to colgate’s digital collections portal. there read more the post metadata during covid appeared first on dlf. three new ndsa members since january , the ndsa coordinating committee unanimously voted to welcome three new members. each of these members bring a host of skills and experience to our group. please help us to welcome: arkivum: arkivum is recognized internationally for its expertise in the archiving and digital preservation of valuable data and digitized assets in large volumes and multiple formats. colorado state university libraries: colorado state university libraries’ digital preservation activities has focused on web archiving, targeted born-digital collecting, along with collection development and preservation guidelines for its digital repository. vassar college libraries: vassar college libraries are committed to supporting a framework of sustainable access to our digital collections and to participate locally, nationally, and globally with other cultural and professional organizations and institutions in efforts to preserve, augment, and disseminate our collective documentary heritage. each organization has participants in one or more of the various ndsa interest and working groups, so keep an eye out for them on your calls and be sure to give them a shout out. please join me in welcoming our new members. a complete list of ndsa members is on our website. in future, ndsa is moving to a quarterly process for reviewing membership applications. announcements for new members will be scheduled accordingly. ~ nathan tallman, vice chair of the ndsa coordinating committee the post three new ndsa members appeared first on dlf. virtual ndsa digital preservation recordings available online! session recordings from the virtual ndsa digital preservation conference are now available on ndsa’s youtube channel, as well as on aviary. the full program from digital preservation : get active with digital preservation, which took place online november , , is free and open to the public. ndsa is an affiliate of the digital library federation (dlf) and the council on library and information resources (clir). each year, ndsa’s annual digital preservation conference is held alongside the dlf forum and acts as a crucial venue for intellectual exchange, community-building, development of good practices, and national agenda-setting for digital stewardship. enjoy, tricia patterson; digipres vice-chair, chair the post virtual ndsa digital preservation recordings available online! appeared first on dlf. announcing finnish translations of the levels of preservation matrix and assessment tool the ndsa is pleased to announce that the levels of preservation documents have been translated into finnish by our colleagues from csc – it center for science and the finnish digital preservation collaboration group. translations for the assessment tool template and both versions of the levels of digital preservation matrix were completed. links to these documents are found on the levels of digital preservation osf site (https://osf.io/qgz /) as well as below. levels of digital preservation matrices and the assessment template: urn:nbn:fi-fe if you would be interested in translating the levels of digital preservation v . into another language please contact us at ndsa.digipres@gmail.com. &# ; suomenkieliset käännökset vuoden pitkäaikaissäilyttämisen tasot dokumentteihin ndsa:lla on ilo ilmoittaa, että csc – it center for science ja pas-yhteistyöryhmä ovat yhteistyössä kääntäneet pitkäaikaissäilyttämisen tasot dokumentit suomeksi. käännökset arviointityökaluun ja matriisin ovat valmiita. linkit dokumentteihin löytyvät osf verkkosivustolta (https://osf.io/qgz /) ja alta. urn:nbn:fi-fe &# ; the post announcing finnish translations of the levels of preservation matrix and assessment tool appeared first on dlf. clir now hiring: senior program officer, dlf clir is now seeking applicants for a senior program officer to lead the digital library federation (dlf) program. this position will play a pivotal role in the strategic direction of dlf and provide vital leadership and guidance both within clir and the wider community. in line with the program&# ;s mission, the senior program officer will serve as an advocate for research, learning, social justice, and the public good. at clir we believe a team with a wide variety of experiences, viewpoints, wisdom, and backgrounds makes for a healthier, stronger organization, and enables us to increase our impact as we collectively work toward more inclusive and useful information infrastructures. we encourage applications from all those who are qualified and interested, including black, indigenous, and other people of color; individuals with disabilities; veterans; and intersectional individuals. please find the full job description and learn how to apply here. the post clir now hiring: senior program officer, dlf appeared first on dlf. levels of digital preservation digital curation decision guide published it is with great pleasure to announce that the levels of digital preservation work continues to roll out. in these waning days of , the curatorial team of the levels has released its first published edition of the digital curation decision guide. this guide forms the basis of a series of decision points around collections and the implementation of a preservation strategy. it is not just for the preservation specialist but rather, it attempts to highlight the preservation implications of collections decisions. it is the group’s desire that the guide will facilitate dialogue between and among stakeholders across a given organization and inculcate a preservation mindset without necessarily having to be a deep expert. it is also not prescriptive. it is a non-linear exploration of multiple vectors that make up curatorial decision-making such as collection development, intellectual and security considerations, and technical capacity. the guide has both a visual and prose version to explore from different angles the basic factors in complex infrastructure and collections management decisions. the decision guide will be extended in the future, linking up with other efforts such as actively maintained working definitions, policy frameworks and examples, as well as the revision of the levels, themselves. i would like to thank the many people who gave us exceptional feedback along the way. this group started in late and worked for two years to craft something we hope will form a part of your organizational practice. in particular, i want to spotlight the work of the individuals who stuck it out to the very end: angela beking &# ; (co-chair) library and archives canada bradley daigle (co-chair) &# ; university of virginia / aptrust ian collins &# ; university of illinois chicago tawnya keller &# ; university of utah donald mennerich &# ; nyu libraries rosalyn metz &# ; emory university leah prescott &# ; georgetown university law library nathan tallman &# ; penn state university walker sampson &# ; university of colorado boulder david underdown &# ; the national archives (uk) simon wilson &# ; independent archivist lauren work &# ; university of virginia this group represents a broad spectrum of organizational perspective and creative talent. thanks to their collective effort, we have something that we hope will be of use to the community. that said, we also understand that the digital curation decision guide is just the beginning and that with increased use and refinement, we will need to update and improve on what we have started. we hope that you, the preservation community (broadly defined), use this guide to engage your colleagues and provide the clear pathways to preserving our cultural record. ~ bradley daigle, curation working group chair the post levels of digital preservation digital curation decision guide published appeared first on dlf. dlf forum community journalist reflection: lisa covington this post was written by lisa covington (@prof_cov), who was selected to be one of this year’s virtual dlf forum community journalists. lisa covington, ma is a phd candidate at the university of iowa studying sociology of education, digital humanities and african american studies. her dissertation work is “mediating black girlhood: a multi-level comparative analysis of narrative feature films.” this research identifies mechanisms in which media operates as an institution, (mis)informing individual and social ontological knowledge. in , lisa received the rev. dr. martin luther king, jr. award from the iowa department of human rights. she is the director of the ethnic studies leadership academy in iowa city, an educational leadership program for black youth, in middle school and high school, to learn african american advocacy through incorporating digital humanities and social sciences. lisa received her ma from san diego state university in women &# ; gender studies. as a youth development professional, lisa develops curriculum for weekly programming with girls of color, trains teachers on best practices for working with underrepresented youth, and directs programs in preschool through college settings in california, pennsylvania, iowa, new jersey, new york and washington, d.c. what is dlf? where does equity fit into conversations? is this a space where my scholarship is valued? these are the questions i asked myself as i planned to attend my first dlf conference. i was uncertain about the dlf community and what would be considered worthy of discussion in this space. these were the thoughts i had as i prepared for the opening plenary of the dlf conference. honestly, i was hopeful when i saw stacey patton was speaking. i am familiar with her work as a historian, professor, journalist and as an advocate on behalf of black children. hopeful, i listened and learned. one thing that patton said that remained with me throughout the conference: “galleries, libraries, archives &# ; museums (glams) reflect the ways in which white people colonize information but also serve as gatekeepers of information related to themselves. when archives were originally set up, black names weren’t important unless they were attached to the names of an important white person who owned slaves or was some kind of white savior to black people.” as i understood dr. patton’s statement during the dlf opening plenary, i was glad to hear those words for two reasons. it was refreshing to have a conference begin on the note of connecting modern-day oppressions to the profession and, secondly, the serious impact it has on black and brown people and the archives that reflect our identities. the uprisings and protests taking place in demanding the end of police brutality while asserting black lives matter across professions, walks of life and locale. the marches, the rallies and public outcry demand the end of police brutality while declaring black humanity is worthy of existence: a sign of the times. today, as we prepare to enter , we can no longer point to the s as winning the fight for civil rights. will the history books say that we were on the side of justice or oppression? we collectively bear witness to social unrest stemming from state sanctioned violence against black men, women and children in the united states. what followed was mass deployment of solidarity statements across universities, businesses and organizations. also, during this time, people across careers trajectories and higher education fields often went to twitter to talk about how their employer needs their opinion, the lack of inclusive curriculum or their department asks them to be on the diversity committee of one. it was the first time i had ever witnessed a collision between social injustice, academia and organizations. so why now? why was patton daring us to challenge oppression when it appears that releasing statements is enough? but then i thought about moving across country for graduate school. no university official checked on my well-being even though it was months after sandra bland was murdered in texas. when alfred olongo was murdered in california, no teacher or administrator at any school i ever attended asked how this impacted me as a student. but now: an abundance of statements addressing equity? our work with archives is no different. we can make inclusive claims or create just practices. during the dlf opening plenary patton’s acknowledgement to us of the daily violence—physically and disciplinarily—allowed us to honestly consider the necessary but difficult work of implementing just practices. she asked for and demanded accountability for the historical neglect of black archives across glams. patton further asked, “…is there a version of this kind of erasure happening in the galleries, libraries, archives, museums where there’s still baked-in inequality from a century ago?” the answer to this is clear in what is preserved with intention and what is ignored. whose history is preserved? what history is deserted? as curators, stewards and preservationists, it is our duty to develop intentional, concrete practices to ensure oppression does not continue in glams. the possibility of this rests in our daily and hourly commitment, or negligence, to just practices. it was hopeful to see several bipoc folks and allies model commitment to justice and equity throughout the digital humanities. in honoring an elder and first archivist at allen university—ms. wilhelmina broughton—carol bowers, courtney rounds &# ; sloane clark shared the curation of the archive. ms. broughton contributed much to south carolina communities during her years of teaching in public schools ( - ) and work as the first archivist at allen university for years ( -present). as with many of our institutions, when one person leaves their knowledge leaves with them, but we see other possibilities through what allen university created: honoring a living legend and incorporating ms. broughton’s institutional knowledge, with pastperfect software, to create the archive. another session brought a thoughtful approach to black people who were victims of state sanctioned violence. in her presentation on systems honoring these victims, gina nortonsmith of the civil rights and restorative justice project thoughtfully included trigger warnings. i have rarely heard read more the post dlf forum community journalist reflection: lisa covington appeared first on dlf. calls for volunteers for digital preservation conference the ndsa calls for volunteers to join our planning committee for the digital preservation conference. digital preservation (digipres) is the ndsa’s annual conference &# ; open to members and non-members alike &# ; focused on stewardship, curation, and preservation of digital information and cultural heritage. the meeting will take place on november - th in st. louis, missouri, just after the dlf forum. ndsa is an affiliate of the council on library and information resources (clir) and the digital library federation (dlf), and the digipres conference is held in concert with the annual dlf forum. clir continues to monitor the ongoing covid- pandemic, and after successfully pivoting to a virtual format for , will be making a call on this for by early spring . planning committee responsibilities include: defining a vision for the conference crafting and distributing a call for proposals reviewing and selecting proposals identifying a keynote speaker determining the conference schedule moderating sessions supporting membership through recruitment and mentorship efforts collaborating with the dlf forum planning committee on community events, equity and inclusion, and sponsorship opportunities we expect to have monthly group calls from january-november, and this year’s committee will have an exciting opportunity to creatively sustain some of the conveniences and benefits of our virtual platform as we negotiate meeting in person again. join us by completing this form by friday, january th, and please share widely. we look forward to working with you! tricia patterson, chair jes neal, vice chair/ chair the post calls for volunteers for digital preservation conference appeared first on dlf. dlf forum community journalist reflection: betsy yoon this post was written by betsy yoon (@betsyoon), who was selected to be one of this year’s virtual dlf forum community journalists. betsy yoon (she/they) is an adjunct assistant professor and oer/reference librarian at the college of staten island, cuny and earned her mlis in . she also has a master of international affairs. she lives in occupied lenapehoking and is a longtime member of nodutdol, a grassroots organization of diasporic koreans and comrades working to advance peace, decolonization, and self-determination on the korean peninsula and turtle island (north america). interests include critical approaches to oer and openness, the free/libre software movement, understanding and addressing root causes over symptom management, and the role that libraries and archives can play in our collective liberation. one advantage of attending a virtual forum is the fact that you no longer have to decide between two interesting panels that are happening at the same time. before i realized that the sessions would be pre-recorded and available for viewing any time, i pored over the schedule trying to decide which of the two sessions to participate in per time block. what complicated my decision-making process was the fact that i was attending with two different angles. as a librarian, i do reference and work related to open educational resources on my campus. but i am also part of nodutdol( ), a community organization with a -year history in need of archiving and preservation. while not totally distinct (outreach is relevant for both roles, for instance), my two roles occasionally had divergent needs. for example, the monday : pm session had both “where it’s at: using gis tools for engagement and outreach at an hbcu library,” which seemed a good fit for my position in the academic library, and “linked while apart: overcoming division with linked data,” which seemed more applicable to my work with nodutdol. so you can imagine my delight when i learned that not only would the sessions be available for asynchronous viewing, but that it would also be possible to engage in discussions about the panel on slack. the panel that ended up being the most informative for my specific needs was &# ;finding a good fit: scaling best practices for born-digital material to fit the needs of diverse communities.&# ; the presenters walked us through the process of setting up a small-scale digitization project and emphasized the iterative nature of the process. as a grassroots organization, we do not have the luxury of hiring digitization experts to guide us through the process, and it has been difficult to know how and where to get started. margo padilla&# ;s saying that &# ;good enough&# ; digital preservation practices (as opposed to best practices) stood out to me as particularly relevant to my organization&# ;s needs. the description of their organization&# ;s custom modular setup and the numerous resources that the slides linked out to were also very helpful in offering some solid starting points to embark on a “good enough” digital preservation process. i also found the learn@dlf sessions to be valuable in their specificity&# ;in particular, i found the tools introduced in &# ;wax and jekyll to build minimal digital projects&# ; and &# ;oral history as data: lightweight, static tool for publishing and analyzing transcripts&# ; to be accessible in that they did not necessarily require investing time in a comprehensive platform or software and instead had relatively low barriers to entry. wax, for example, describes itself as a &# ;minimal computing project for producing digital exhibitions focused on longevity, low costs, and flexibility.&# ; while not exactly the same, the spirit behind minimal computing reminded me of splots (what the acronym stands for is not yet fixed &# ; one interpretation is smallest portable open learning technology), which are intended to have low barriers to entry and &# ;support more accessible, sustainable, and user-friendly ways to get publicly-engaged learning happening on the open web.&# ; the question of platforms and sustainability is a topic that is directly relevant to my work in open educational resources and with nodutdol, and i always love to learn about technologies that provide access to knowledge creation mechanisms without locking you in to a specific system. though the fact that this year’s dlf forum was digital was due to the constraints of the pandemic, the thoughtful way in which the experience was designed was due to the efforts of the organizers. the asynchronous viewing options, the slack interface, the provision of presentation slides and transcripts all made it possible for organizations such as mine to benefit from the expertise of the dlf community. while an in-person dlf forum will no doubt have different considerations, i hope that some of the innovations of this year will be retained for future forums to ensure wide accessibility and participation from a wide variety of organizations and individuals. as a first-time dlf forum participant, i am grateful to have been able to participate in this year’s virtual forum and look forward to continuing to learn from the dlf community! ( ) nodutdol is a grassroots organization of diasporic koreans and comrades based in lenapehoking/new york city seeking to advance peace, decolonization, and self-determination on the korean peninsula and on turtle island/north america. we advance our mission through political education, collective action, and principled solidarity. the post dlf forum community journalist reflection: betsy yoon appeared first on dlf. dlf forum community journalist reflection: carolina hernandez this post was written by carolina hernandez (@carolina_hrndz), who was selected to be one of this year’s virtual dlf forum community journalists. carolina hernandez is currently an instruction librarian at the university of houston where she collaborates on creating inclusive learning environments for students. previously, she was the journalism librarian at the university of oregon, where she co-managed the oregon digital newspaper program. her mlis is from the university of wisconsin-madison. her current research interests are in critical information literacy, inclusive pedagogy, and most recently, the intersection of digital collections and pedagogy. i have been interested in attending the dlf forum for a few years now, but the timing was never quite right until this year. with the conference being both online and free, it was a no brainer for me to finally attend. considering i am an instruction librarian, though, it may seem like an odd choice for me. however, in part because of my previous experience with curating the oregon digital newspaper project at the university of oregon, i’ve been interested in exploring the many ways digital library technologies and digital collections themselves can be incorporated into information literacy instruction. with covid- entirely moving our instruction to the online realm, this interest has become an imperative. this conference has confirmed for me that there are many ways these areas intersect and could inform my instructional approach. while many of the sessions i watched did not directly address pedagogy, there was still so much i was able to glean from the presentations that i could take back to my realm. the main thing that popped out at me was the way so many presenters addressed accessibility in one way or another. of course, this stood out the most with the “creating accessible and inclusive content” combo session, which began with rebecca bayeck’s clarification of the difference between accessibility and inclusivity, two terms that are often used interchangeably. while accessibility is more about making sure that the final product is “usable by people with all abilities,” bayeck made the important distinction that inclusivity goes a step beyond that to also make sure individuals “feel comfortable/safe when using [it].” this is something i try to keep in mind when lesson planning, how it’s important to not only make sure that students are able to access the learning materials in whatever way works best for them, but that they also find the relevance of information literacy to their own lives. in another presentation from this session, daniella levy-pinto and mark weiler noted some of these issues, such as “unlabeled buttons or links,” which can be hard to identify properly for those using screen readers. in fact, several presenters and attendees emphasized the importance of testing platforms and content with screen readers. carli spina also spoke about the importance of including audio descriptions and transcripts for audio-video content and also mentioned specific tools, such as cadet, that can help create these necessary points of access. cadet, or caption and description editing tool, is free and allows you to create captions and timed scripts, but it can also be used to more easily add audio descriptions. it was helpful to see some of these accessibility best practices in action via the conference itself. because presentations were recorded in advance, they were able to include both closed captioning and transcripts for each one. conference coordinators encouraged attendees to make their postings in slack accessible as well by including image descriptions whenever a picture was included. this emphasized for me how it’s not only important to create accessible learning materials, but to foster a community that encourages others to follow suit. it is a helpful model for my instruction team as we move forward with helping our liaison colleagues with their own instruction. as i’ve been considering how to build lesson plans and activities around digital collections, the other session that stood out to me was the panel “us latino dh: recovering the past, creating the future.” the presenters gabriela baeza ventura, carolina villarroel, linda garcia merchant, and lorena gauthereau spoke about the us latino digital humanities program based at the university of houston, my current institution. this made their work immediately relevant to mine, as they are already working with part of the same community i teach. what stood out to me most, though, was their use of “omeka as pedagogy.” baeza ventura talked about her specific experience with teaching an undergraduate class wherein students used omeka to curate an exhibit, thus allowing them to “contribute to knowledge production.” this freirean approach to teaching is very much in line with our instruction team’s programmatic information literacy outcomes, which focus on encouraging students to see themselves as information creators. with a lot about the coming year still up in the air, my team and i plan to continue our efforts to strengthen both the synchronous and asynchronous online learning content we offer as it seems likely the demand for online teaching will certainly not go away. i am looking forward to bringing a lot of these ideas from the dlf community back to my department and finding ways to incorporate them into our pedagogy. the post dlf forum community journalist reflection: carolina hernandez appeared first on dlf. dlf forum community journalist reflection: rebecca bayeck this post was written by rebecca bayeck (@rybayeck), who was selected to be one of this year’s virtual dlf forum community journalists. rebecca y. bayeck is a dual-phd holder in learning design &# ; technology and comparative &# ; international education from the pennsylvania state university. currently a clir postdoctoral fellow at the schomburg center for research in black culture where she engages in digital research, data curation, and inclusive design. her interdisciplinary research is at the interface of several fields including the learning sciences, literacy studies, and game studies. at this intersection, she explores literacies and learning in games, particularly board games, the interaction of culture, space, and context on design, learning, research, literacies. the year is, without a doubt, complex, filled with multiple challenges as well as opportunities. from remote working/online learning environments to conferencing in a virtual space, everyone has tried and is still trying to adjust to the “new normal” or maybe “new abnormal”. among those adjusting to the world of social distancing is the digital library federation (dlf) forum. the dlf forum was my first introduction to and participation in the work completed by this community of librarians and library enthusiasts. the dlf forum was in a virtual format. though i would have loved to visit baltimore, interact in-person with my clir cohort, and experience the food culture of baltimore restaurants, the virtual space still provided a space for encounters and meaningful interactions. the conference organizers used two platforms: aviary for pre-recorded presentation viewing and slack for questions and dialogue with attendees. both platforms created a unique and complimentary online experience. for instance, each video had captions and a downloadable transcript, making the content more accessible. slack discussions created a sense of community, and gave me a sense of belonging through the interaction among participants. it was a common practice for participants through emojis to like, applaud, or reply to a post/comment (figure ). slack&# ;s direct message option further personalized my experience of the online conference. i directly exchanged with some attendees, and established personal rapport. however, my major takeaway from the dlf forum was inclusion, shown in the accessibility efforts deployed by the organizers, and in the diversity of topics covered by presenters. the conference brought to the forefront research/topics that have received less attention in the past, and should be discussed today across disciplines and fields. stacy reardon and michael lange from uc berkeley presentation on can we digitize this? should we? navigating ethics, law, and policy in bringing collections to digital life was so inspiring. in my view, it captured the essence of digitization and how it should be done. it is important for individuals engaged in digitization efforts to always ask whether “the value to cultural communities, researchers, or the public outweighs the potential for harm or exploitation of the people, resources, or knowledge.” it is about not harming or causing harm to the various stakeholders who will see or engage with our final product. this balancing principle can be applied to any research, design, development, or digitization endeavors. paraphrasing stacey patton, dlf forum plenary keynote speaker, “how the knowledge came about, [who it will harm, and who will claim ownership of it] is as important as the knowledge itself.&# ; juliet hardesty from indiana university&# ;s presentation on mitigating bias through controlled vocabularies gave a powerful insight into why it is important to incorporate community vocabulary to broaden access to knowledge/data, and fight biases. being a panelist in the session creating accessible and inclusive content and presenting on addressing issues of accessibility: urgency in a world of social distancing, added to my experience. my interest in issues of accessibility for blind/low vision, deaf/hard of hearing individuals in this era of social distancing facilitated conversation with other session presenters on the accessibility of this conference for screen reader users. from our conversation, it became obvious that the organizers needed to be commended for providing scripts and captions for the recorded videos. dee, who is blind and uses a screen reader to access contents, and micky, who is sighted, but did use a screen reader said about their experiences: dee: i do appreciate knowing that there is a caption and there is like, the scripts and everything. it does make you feel more welcome and it does make you realize yes, they&# ;re making an effort. applause should be given for, you know, for the effort and definitely you know even the fact there is like, an accessibility session or stream, that&# ;s important. micky: but if i could say one thing is, i think, i know people were making efforts into making it accessible. i think applause is very much deserved. and i think keep, we got to keep trying and we gotta keep finding innovative ways of working together and collaborating. so i would want people to feel encouraged that there&# ;s more work to be done. but let&# ;s pursue it as a team and then pursue it together because it&# ;s worth it. nevertheless, dee and micky did have some suggestions for creating accessible conference experiences: dee: the first thing i may say would be to include in the planning committee. kind of representation from all the groups that you would like to have at the conference, because they will help think about, in this case, about screen readers and which tool and will it be interactive and you know if we want to make it interactive, is there an option or how can we? you know, it&# ;s necessary for someone to think about those things&# ;ensure to have broad representation in the planning committee, or if not in the planning committee, make a point to reach out and consult. so, engage with people with lived experience of whatever the conference you know is trying to accomplish. micky: if i could make a suggestion that might just be to have a panel with like, just in this particular case, disabled leaders in the glam field. you know there&# ;s, i&# ;m sure, a ton out there read more the post dlf forum community journalist reflection: rebecca bayeck appeared first on dlf. dlf forum community journalist reflection: ana hilda figueroa de jesús this post was written by ana hilda figueroa de jesús, who was selected to be one of this year’s virtual dlf forum community journalists. ana hilda figueroa de jesús will be graduating next spring from the universidad de puerto rico in río piedras with a ba in history of art. her research interest focuses on education, accessibility and publicity of minority, revolutionary puerto rican art including topics such as race, gender and transnationalism. she has interned at visión doble: journal of criticism and history of art, and volunteered at meca international art fair and instituto nueva escuela. ana works as assistant for the curator and director of the museum of history, anthropology and art at upr. she is currently a katzenberger art history intern at smithsonian libraries. community, a repeated concept during the virtual dlf forum, transitioned from a word to a mantra. from my home in puerto rico, i listened and learned from peers who shared their professional and academic experiences. it was my first time attending this forum. lots of questions popped into my mind, particularly: why would staff insist on creating a virtual “community,” of four days duration, with an audience currently experiencing and being exposed to social injustice within their surroundings and media? what does “building a community” mean? i would like to thank the dlf staff’s commitment. when i received notice that i was selected as a community journalist, my first observation was my last name. the correct spelling and grammar of my name are much more than my cover letter, it represents and brings value to where i come from and those who came before me. i noticed the correct inclusion of the accent mark on “de jesús.” in that moment i knew my voice would be respected in the discourse. the physical distance between attendees was no obstacle. the forum implemented digital platforms such as slack and aviary. presentations included transcripts and, since they were previously recorded, i could pause and rewind the video when a new term was introduced. if anyone had a tech problem or just wasn&# ;t familiar with the apps, staff was available to help immediately. each speaker mentioned their preferred pronouns and acknowledged the indigenous homelands from where they spoke. being inclusive both in theory and practice was a priority. as a puerto rican woman of color with low resources, a first-generation college student and an undergraduate scholar, i saw benefit in this opportunity. the forum sessions reminded me of particular aspects of archives as spaces of data justice, cultural and social responsibility, knowledge production, and alternate historical narratives. in mitigating bias through controlled vocabularies, juliet hardesty explained the importance of being conscious of racial categories, first nation groups, non-binary people, and others when including metadata such as subjects, genres, and languages. as an example, she discussed linked data across institutions and the distinction between “exactmatch&# ; and “closematch.” through this conference, i sought to learn and carry out new interdisciplinary perspectives in my roles as a katzenberger art history intern at smithsonian institution libraries and as an assistant to the curator and director at the history, anthropology and art museum of the university of puerto rico. one of the most relevant presentations for me was curationist: designing a metadata standard and taxonomy for an open cultural ecosystem by sharon mizota. she explained the anti-colonialist, anti-racist, feminist, queer, accessible, and multilingual lens of data. also, she examined taxonomy guidelines that included sensitive subject areas. to illustrate this, she reflected on adopting the terms “latinx” instead of “latino” or “hispanic,” and “homeless people” instead of “homelessness” or “tramp.” i also attended discussions of human rights movements such as blm, community activism and digital archiving in the era of george floyd. other social matters were considered in the lightning talk titled pandemic pivot: how to take your event online, reach new audiences, and build even stronger communities, and the panel creating a virtual community for virtual reality: challenges and solutions for program building during a pandemic. another engaging talk was us latino digital humanities: recovering the past, creating the future. i would like to mention a crucial keynote by dr. linda garcia merchant when discussing chicana feminist scholar maria cotera, [the importance of understanding] “the archive as a living active experience of “encuentro” between the present and the past, with the potential to enact new strategies of allegiance and a new praxis”. i believe that this idea of encounter is linked with the dlf forum’s “building a community” proposal. we must review our reactions and interactions with each other, including those within academia. our involvement with the writing of histories makes us judges of how and what is told. something i learned from this experience is that beyond our professional responsibilities, we have a social contract and accountability. information must be accessible to traditionally marginalized public. stories, agents and terms that once were excluded from the official narrative must be taken into consideration. the virtual dlf forum included more than spaces for education. it was about feedback, mutual aid, being open to new perspectives, and building a community. &# ; &# ; the post dlf forum community journalist reflection: ana hilda figueroa de jesús appeared first on dlf. dlf forum community journalist reflection: arabeth balasko this post was written by arabeth balasko, who was selected to be one of this year’s virtual dlf forum community journalists. arabeth balasko (she/her) is an archivist and historian dedicated to public service and proactive stewardship. as a professional archivist, her overarching goals are to curate collections that follow a shared standardization practice, are user-centric, and are searchable and accessible to all via physical and digital platforms. she believes that an archive should be a welcoming place for all people and should be an inclusive environment which advocates to collect, preserve, and make accessible the stories and histories of diverse voices. by getting individuals involved in telling their story and making their history part of the ever-growing story of humanity, we all win! the dawn of the great archival shift over centuries, archives and archivists have been heralded as the keepers, the stewards of records, stories, and collective memory. however, at times this stewardship has come from a place of exclusion, centered heavily around white, english-speaking experiences. countless stories, memories, and events have been omitted from the larger historical narrative or have been rewritten from a skewed perspective. now (and unfortunately for centuries) racism and police brutality has permeated our country’s history. lack of racial equity has led to whitewashed and white supremacy-based collection policies that are geared towards uplifting and showcasing one-sided narratives, while often overlooking, overwriting, and suppressing contributions and accolades of black, indigenous, and people of color. so many times, it is easier for folks in the glam sectors to claim neutrality, to leave it to the next generation, to look the other way, and focus only on the past and occasionally, the present. from being overworked, under-supported, and oftentimes misunderstood, archivists have grown tired, and with tiredness comes apathy. other times it is simply not knowing what to do and/or not having the “authority” to make actual changes in an organization. this too leads to burnout, turnover, and once again, apathy. the humanities profession’s lack of diversity and equity has engrained a culture of dysfunction in several of the glam organizations across the country. during the dlf forum, it was apparent that i was not the only person who felt this way. several of the sessions focused on how to become a more proactive, mindful, and accountable steward, while also taking care of your own mental health and well-being. from practicing mindfulness and relaxation techniques, to reflecting upon language choices in metadata and/or finding aids, to reviewing your institution’s digitally available content for equity and inclusiveness; this conference truly spoke to my soul in many different ways. however, during the conference, one overarching question that i continued asking myself was – “how can i, as an archivist, ensure that all folks feel represented in an archive? is this possible?” upon a week’s reflection and allowing myself some time to digest the rich information and ideas posed during these diverse sessions, i came to reframe my question as, “what can we as archivists do to support the fight for archival equity?” in my opinion, one big step archival repositories and the archivists in those repositories can take is to not promote the idea of neutrality. oftentimes, archives can shy away from hard histories, hard conversations – they can minimize hurts, and maximize virtues, but i feel that is a misstep. history is ugly, sad, beautiful, heartwarming, heartbreaking, and real. it happened. you cannot change that. but you can work to showcase how it happened, why it happened, and help reshape it for today’s generation through modernized lenses. by uplifting the stories and voices of bipoc folks, which have traditionally been omitted from the collected narrative for centuries, and reinterpreting and reclaiming the stories of those lost, overwritten, and marginalized throughout history, archival repositories can truly become beacons of change throughout the glam sectors. many folks, especially those who identify as bipoc, feel they are not represented in an archive – or if they are, their stories and experiences have been retold without their voice, their input, or their permission. as an archivist it is so important to work to build relationships and connections with communities and foster and tend to those relationships over the years. so many times, archival organizations take on collections, sign deeds of gift, and then the relationship ends. i think this is a huge misstep for any archival repository. by investing in communities – communities will invest in you. with each new generation comes new opportunities to promote equity and accountability throughout archival repositories. each generation of archivists should be reflecting and reevaluating how stories are (and traditionally have been) collected, maintained, presented, and made (or not made) accessible. i cannot tell you how many times i have worked with patrons, students, and volunteers, who have expressed to me that they feel that they are not represented, they are not “seen” amongst the archival collections they are exploring. it truly breaks my heart, and it means that we, as archival professionals are falling short, and we need to do better and be better for all users. i feel that inclusivity is key to create a well-rounded narrative, where users/patrons/researchers/etc. can “see themselves” reflected in the archives and collections. it has been my experience that when someone feels invested in and has input to how they are being represented, there is a higher propensity for folks to champion for the survival and continuation of an archive. by getting community members and groups invested in telling their story, identifying themselves in their own way and own language/words, and by not leaving it up to the archivist to make assumptions, collections that are taken in become more authentic and personal. during this conference, i also reflected a lot on the right to be remembered and the right to be forgotten. nobody owes anyone their story. it is so important for archivists (including myself) to remember that. some stories are too painful for folks to share, some are not ready (and may never be), and some are read more the post dlf forum community journalist reflection: arabeth balasko appeared first on dlf. dlf forum community journalist reflection: jocelyn hurtado this post was written by jocelyn hurtado, who was selected to be one of this year’s virtual dlf forum community journalists. jocelyn hurtado is a native miamian who worked as an archivist at a community repository for four year. she is experienced in working with manuscript, art and artifact collections pertaining to a community of color whose history has often been overlooked. ms. hurtado, understands the responsibility and the significance of the work done by community archivists and has seen firsthand that this work not only affects the present-day community but that it will continue to have a deep-rooted impact on generations to come. ms. hurtado also has experience promoting collections through exhibits, presentations, instructional sessions, and other outreach activities which includes the development and execution of an informative historical web-series video podcast. ms. hurtado earned her associate degree in anthropology from miami-dade college and a bachelor of arts in anthropology from the university of florida. she also completed the georgia archives institute program. this year has been full of new experiences and we all have faced the challenges of adapting to the new professional realities of relying on technology to complete our work, promoting the goals of our organization all while staying connected with communities we serve virtually. our phones and laptops are now on the top of the list of tools we cannot function without and it’s arguably just as valuable as a pencil or finding aid to an archivist, at least from my own personal experience. as a first-time attendee and a community journalist, i was excited and unsure of how the dlf forum would operate on a virtual platform. like millions around the world i’ve been working remotely for months and learned to adapt but i was still hesitant on how attendees would be able to truly connect to the panelist, fellow attendees and with the subject of each talk remotely. it is no secret that librarians, archivists, historians or anyone in a related field have a tendency to be introverted and from my own personal experience starting a conversation, connecting with others and networking can be stressful. however, i was quickly positively surprised on how easy it was to start a conversation at the conference. i enjoyed the slack application in which attendees were able to share thoughts, ideas and pose questions about each session. i certainly viewed more opinions, concepts and panels virtually than i probably would have in person. i liked the fact that i could have access to the sessions anytime which is great for anyone who has a busy schedule as well as any problems accessing the videos due to the digital divide caused by finances or other factors such as remoteness or environmental factors such as hurricanes or storms. in the opening plenary i was delighted to hear the acknowledgement of the indigenous people and their lands in regards to the location of the original conference and area that was being discussed. the keynote speaker, dr. stacy patton, was simply incredible and asked us to grapple with a very important question: do black lives matter in galleries, libraries, archives and museums? i believe we all know and can say historically the answer is no, black lives have systematically been erased and unwelcome in these spaces. has become the year of reckoning for some institutions and for many in this field that have been part of this problem. thus, the question becomes what now? how can meaningful and genuine change come about? there is no one size fits all plan and up to in the field to do the work and realize there will never be a timeline or an exact moment where it will be marked as done. i had the opportunity to ask dr. patton a question and was also able to see other questions posed by fellow attendees. it created a hub for sharing experiences and problems encountered in our own institutions which was able to foster a connected moment and experience. dr. patton hit the nail on the head when reciting the claude mckay poem, “if we must die.” it was a couple of days before election day and oh how the words aptly describe the current era and the rawness of it all. during her speech i reflected on the work of schomberg and many other black intellectuals whose worked and made centers were black lives do matter and their stories were properly preserved. i also reflected on my experience working at a black community repository, in a space made for black lives to matter. i also recognize another important question: which black lives matter in these spaces? women, individuals overlooked due to their sexual orientation, and those from a lower socioeconomic status or position have had their stories overlooked. there is so much work to be done and this has encouraged and highlighted the importance of pushing the boundaries and the sharing of ideas. the recording restorative justice and accountability: the burnham-nobles digital archive presented by gina nortonsmith, raymond wilkes, amanda rust and drew facklam was inspiring. the work done by the team of telling the stories of victims and giving a voice is imperative. prior to this session i did not know of the civil rights and the restorative justice project and was glad to learn about the research being conducted along with support policy initiatives on racial hate crimes during the jim crow era and that justice is still being pursued for the victims and their families. the statement “i like to think of the investigator as the foundation for an archive, while the archivist is the architect and engineer, providing structure and organization in order to complete the building, i.e. the archive” by raymond wilkes beautifully explained the importance of the collaborative efforts and the relationship member of the team had to the task. i am looking forward to next year&# ;s dlf forum and hope/expect it to continue the focus on the community. the post dlf forum community journalist reflection: jocelyn hurtado appeared first on dlf. dlf forum community journalist reflection: melde rutledge this post was written by melde rutledge (@melderutledge), who was selected to be one of this year’s virtual dlf forum community journalists. melde rutledge is the digital collections librarian at wake forest university’s z. smith reynolds library. he is responsible for leading the library’s digitization services—primarily in support of zsr’s special collections and archives, as well as providing support for university departments. he earned his mlis from the university of north carolina at greensboro, and has served in librarianship for approximately years. his background also includes years of newspaper journalism, where he wrote news, sports, and feature articles for several locally published newspapers in north carolina. he currently lives in winston-salem, nc, with his wife and three sons. since , i’ve appreciated the opportunity to attend and participate in the annual dlf forum. i look forward to the great takeaways to share with my colleagues back home. let’s also not forget the wonderful venues where the forum takes place (las vegas, tampa, etc.). needless to say, a global pandemic emerged this year, resulting in the dlf forum to occur virtually. as i reflect on this year’s installment of the dlf forum, it’s difficult not to compare the forum’s first virtual event with the previous in-person gatherings—particularly in regards to size. the fact that the event had more than , registered participants is a testament to the popularity and value of dlf. being that it also surpassed the overall in-person attendance record (just over people) of the dlf forum in pittsburgh is also noteworthy. the segment that i look forward to the most from the dlf forums are the opening plenaries, because of its great keynote speakers. stacey patton provided an excellent talk highlighting the significance of preserving the black experience in galleries, libraries, archives, and museums (glams)—covering several cohesive themes during her one-hour-plus speech—including the covid- pandemic, hiring diverse staff in glams, america&# ;s ongoing issue with racism, and the social and racial parallels of then and now. one of my key takeaways of her talk is spotlighting the need for institutions to be “ready and equipped” during the pandemic to educate students. this was suggested to be accomplished by providing research materials remotely, but also asking, “what difference does this make that we’re digitizing things? how is this power to be used to protect documents when we may not know their importance? what about the digital divide and access to these materials?” these indeed are very important thoughts to me, as a key role of my work is providing digital access to materials tucked away within our special collections and archives. and it was great that presenters addressed patton’s questions throughout the dlf sessions. “‘can we digitize this? should we?’” navigating ethics, law, and policy in bringing collections to digital life” is a great example. this was presented by stacy reardon and michael lange from uc berkeley library. i was impressed with their digital lifecycle program, and the ethical considerations embedded within their workflow. i have interest in seeing how institutions confront the issue regarding the digitization of materials of underprivileged groups, as well as how they approach the handling of culturally sensitive materials, accessibility, and appropriate metadata creation. as julie hardesty pointed out in her presentation entitled, “mitigating bias through controlled vocabularies,” when working with metadata, you can become familiar with several widely used controlled vocabularies. however, working with large common vocabularies can “paint broad strokes that cover up more than they should, that generalize or simplify too much, and show the biases of dominant groups such as the white male viewpoint. additionally, the process to change and update terms can be slow to keep up.” when i listen to presentations on this theme, speakers routinely note the value of incorporating community engagement. a nice example of this was shared during the presentation, “curationist: designing a metadata standard and taxonomy for an open cultural ecosystem.” curationist.org is a site that finds and collects important cultural and historical resources that are within the public domain. as explained by presenter sharon mizota, community users will be able to include their own metadata to records on this site. overall, i salute all the organizers and presenters for producing an impressive dlf program. and kudos to the partnership between clir/dlf and the hbcu library alliance. as an hbcu graduate, i appreciate programming that covers how hbcus approach digitization, and the stories behind the unique materials that are digitally preserved, especially amid the covid- pandemic. covid- is indeed a big factor in how many of us in this profession are conducting decision making. the wealth of relevant content in this year’s dlf was very timely in this regard. the post dlf forum community journalist reflection: melde rutledge appeared first on dlf. dlf forum community journalist reflection: amanda guzman this post was written by amanda guzman, who was selected to be one of this year&# ;s virtual dlf forum community journalists. amanda guzman is an anthropological archaeologist with a phd in anthropology (archaeology) from the university of california, berkeley. she specializes in the field of museum anthropology with a research focus on the history of collecting and exhibiting puerto rico at the intersection of issues of intercultural representation and national identity formation. she applies her collections experience as well as her commitment to working with and for multiple publics to her object-based inquiry teaching practice that privileges a more equitable, co-production of knowledge in the classroom through accessible engagement in cultural work. amanda is currently the ann plato post-doctoral fellow in anthropology and american studies at trinity college in hartford, ct. on belonging: there is that decisive moment for me at every academic conference that i’ve ever attended – whether it is one that i frequent regularly (even annually) or one that i’m trying out for the first time like the dlf forum this year – where i’m sketching out my trajectory of movement and negotiating what my belonging might look like in the space. this moment exists in the scanning of the conference program and translating of different panel abstracts. this moment exists in those standstill seconds in the threshold of a panel room as you decide whether to enter or not, perhaps as you notice a familiar or friendly face. our current pandemic moment has transformed how we collectively gather in profound ways and brought into sharp relief the pre-existing structural social inequities of access. and yet, the decisive moment of my new belonging in the space of the dlf forum was from a distance, and yet was not solitary beginning with a wave of introductions among first-time attendees and offers by long-time forum-goers of support on slack and extending to the generosity of time, experience, and transparency offered to me by my dlf mentor, maggie mccready in our zoom conversations. the decisive moment ultimately resolved, as i left the metaphorical door threshold to take a seat, during dr. stacy patton’s keynote as she seamlessly moved between commentary on national news, archival text, pedagogical practice and her own powerful personal narrative of coming to belong in spaces not made for her experience and of coming to build new spaces of belonging. activating the archive by reframing history as practice: one of the most compelling interventions that dr. patton articulated in her keynote speech was a call for the dlf community to reframe their implicit understanding of history not only as a physical archive of a material past but also as an active departure point for our contemporary reorientation and empowerment. interweaving meaning and purpose across institutional case-studies, she referred to the concept of “historical touchstones” that present us with “context, guidance, and perspective” that have the analytical potential to ground our experience in the positionalities as a “keeper of knowledge” with a “traditional role of being…guide” to students. keepers of knowledge: this proposal of mobilizing history towards how we critically approach our present-day practice in the field echoed throughout the subsequent forum presentations and was especially materialized for me in the citational acts of emphasizing a theoretical focus on an ethics of care. in the “combo session: implementing responsible workflows and practices,” ethics of care was centered in an appreciation for “relationships with uneven power relations,” a methodological re-framing of both those actors who study and those communities who are studied as equal “independent rational subjects” and a researcher responsibility to identify the multi-faceted capacity of archival work to inflict harm. this work was discussed, for example, with the case-study of the archival process at the university of california, berkeley for selecting indigenous cultural material for digitization and if to be digitized, under what terms of public access. in other words, professional ways of working were recast beyond the technicalities of how archival material may be best processed and digitally preserved to include and more importantly, to privilege a recognition of academic histories of community extraction and an opportunity for academic futures of more collaborative, equitable workflows. student guides: building on this important reflection on institutional practices, the us latino dh panel entitled, “recovering the past, creating the future” brought the historically based practice conversation into the context of the undergraduate classroom. during their presentation, i was reminded of dr. patton’s earlier caution that digital work could not be the “end all be all” (even among undergraduates who are often thought to be “digital natives”) given how it is “alien to flow of time…nuances” and “abbreviates how we understand things”. presenters carolina villarroel, gabriela baeza ventura, lorena gauthereau and linda garcia merchant accepted the challenge and outlined a pedagogical design that built a student theoretical consciousness of the silences inherent in archival representations of the human experience and equipped students methodologically through programs like omeka to emerge as digital storytellers of new stories. moreover, the presenters destabilized the curatorial authority of collection-holding institutions by decolonizing where and how we locate archives with models such as post-custodial archives (describing archival management in which the community maintains physical custody of material records) and migrant archives. both panels therefore expanded the boundaries of what constitutes archival practice – in terms of how we keep existing knowledge and how we teach knowledge production – by expanding what we care for to who we care with. at the close: aliya reich, program manager for conferences and events at the digital library federation, remarked at the start of the forum that the goal of our gathering was “building community while apart”. joy banks, program officer at the council on library and information resources, responded on slack to a participant struggling with the digital conference platform that “there is no behind this year”. together, their words bring me – in concert with dr. patton’s keynote assertion of our roles as “guardians of the past and present” and “architects of read more the post dlf forum community journalist reflection: amanda guzman appeared first on dlf. dlf forum community journalist reflection: shelly black this post was written by shelly black (@shellyyblack), who was selected to be one of this year’s virtual dlf forum community journalists. shelly black is the cyma rubin library fellow at north carolina state university libraries where she supports digital preservation in the special collections research center. she also works on a strategic project involving immersive technology spaces and digital scholarship workflows. previously she was a marketing specialist at the university of arizona libraries and promoted library services and programs through social media, news stories, and newsletters. shelly was recently selected as a emerging leader by the american library association and is a provisional member of the academy of certified archivists. she received a mlis and a certificate in archival studies from the university of arizona where she was a knowledge river scholar. she also holds a bfa in photography and minor in japanese from the ua. the weekend protests began in response to george floyd’s murder, i was driving across the country for my first post-mlis job. i listened to the radio, scrolled through the news and felt the country in pain. reflecting on how volatile has been, i’m grateful that the dlf forum was freely open to all and held online. as a graduate student, my exposure to digital curation and preservation focused on theory more than practice. so i was eager to learn about current strategies and tools. considering the anti-racist commitments made recently by numerous organizations, i also looked forward to hearing about projects to improve discoverability of marginalized people in the historical record. many sessions covered computational methods used by librarians, archivists, and researchers to improve our understanding of, and increase access to, digitized materials. juan manuel garcía fernández and nickoal eichmann-kalwara presented on “digital el diario and archival justice in the digital humanities graduate classroom.” their work involved creating a corpus from a digitized s chicanx newspaper and showing students how to use text analysis tools, such as mallet and voyant, for the purpose of historical recovery. in “images as data with computer vision,” carol chiodo shared that harvard university library is using a python package to analyze and provide descriptive metadata at scale for photographic collections. this includes protest photography, so the project will also result in the creation of ethical guidelines for applying automation to sensitive materials. throughout the forum, a theme that resonated with me was the ethics of care. i learned how multiple presenters have adopted this feminist approach that emphasizes relationships and considers power imbalances. during “can we digitize this? should we? navigating ethics, law, and policy in bringing collections to digital life,” stacy reardon explained that she adopts an ethics of care when deciding whether to make materials available online. she noted how this framework urges us to consider the potential for harm not just to individuals but also communities. lorena gauthereau, one of the panel members of “us latino dh: recovering the past, creating the future,” imparted that community outreach should be approached with an ethics of care. she said we have a responsibility to make the people represented in archives feel valued, which can be achieved through post-custodial methods, consent, decolonial spaces, and trusting relationships. as a mexican japanese american, increasing representation and reclaiming the humanity of historically oppressed people has personal significance. i wholeheartedly agree with gauthereau who expressed: “by recovering the past, we can project toward our future. while working with recovered archives, we make space for healing by making visible not only painful histories, but also resistance, survival and joy, to acknowledge where we come from and where we are going.” these presentations reminded me to think critically about the interplay between people, archival collections, and technology. while i see promise in the application of computational methods for understanding and expanding access to stories beyond the dominant narrative, i’m also wary of the challenges. algorithms used for facial recognition, screening job applicants, and identifying high-risk patients continue to oppress communities of color. mixed race people like myself don’t fall neatly into metadata categories and likely aren’t seen by algorithms applied to textual or visual corpora. meanwhile, libraries have started using machine learning for appraisal, description, and other laborious tasks. many collections await being described—or re-described using anti-oppressive language—and made available online. algorithms offer efficiency, but when people create them with training data which centers whiteness, they further harm communities. another challenge is the layering of biases when working with digitized collections. we lose more than visual details and aesthetic qualities through reproductive technologies and migration of formats. there are racial consequences. we scan photographs made from color film stocks originally calibrated for light skin. art historian lyneise williams has also called attention to the erasure of black people through the high contrast process of microfilming. so what happens when we use biased machine learning models to process images that inherit white normativity? growing digitized collections make the adoption of machine learning compelling. at the same time, an ethics of care and diverse voices are needed when new tools are being designed. knowledge produced from analyzing collections at scale will only be as inclusive as the human beings who designed the algorithms and the digitized material’s source medium. as stacey patton reminded us in her keynote, digitization isn’t a be-all and end-all, particularly when there is still the digital divide. the dlf forum inspired me to think about the opportunities and issues ahead. i hope to attend future forums where discussions on using technology in ways which uplift communities of color continue. the post dlf forum community journalist reflection: shelly black appeared first on dlf. dlf forum community journalist reflection: hsiu-ann tom this post was written by hsiu-ann tom, who was selected to be one of this year&# ;s virtual dlf forum community journalists. hsiu-ann is the digital archivist at the amistad research center in new orleans, la where her work focuses on born digital collection development. she received her masters in library and information science with a concentration in archives management from simmons university in boston in . she is a graduate of columbia university (ba, sociology) and harvard university (ma, religion and politics), and is a member of the academy of certified archivists. prior to working in the archival field, hsiu-ann served in the united states army intelligence field as a cryptolinguistic analyst, attending the defense language institute in monterey, california. before coming to amistad, hsiu-ann worked on the archives staff of boston university’s howard gotlieb archival research center working with the military historical society of massachusetts collection. she recently obtained the society of american archivist digital archivist specialist certification and enjoys supporting students and new professionals in their educational development through her work as a member of saa’s graduate archival education committee. i am thankful for the opportunity to have helped cover this year&# ;s conference as one of community journalists. given all of the challenges of , i was looking forward to hearing how my colleagues were finding ways to navigate unfamiliar situations and overcome obstacles. as a new graduate in my first full-time position out of library school, it was both inspiring and encouraging to listen to so many in the field describe their innovations over the course of the conference. i am thankful to the conference organizers and attendees for assembling such a rich program for everybody to enjoy. as i gathered my thoughts about what to write, i was overwhelmed by the achievements and innovations of the dlf community. each session was packed with more information than i could absorb and it was a joy to hear such dedicated professionals talk about their work. yet, when i thought about the conference experience, what stuck out to me the most was the conference design itself and its alignment with dlf&# ;s mission and values. i considered the introductory comments of charles henry and the dlf mission: dlf: advancing research, learning, social justice, &# ; the public good through the creative design &# ; wise application of digital library technology the conference felt like a model for how to implement these community values. it was exciting to see ideas in action and to feel like i was included in that as a newcomer to this group. from before the start of the conference, i appreciated the efforts of conference staff to make conference content accessible to attendees of all abilities. in light of the pandemic and the turn to online communication platforms, transcription and closed captioning services for the hearing impaired are sometimes overlooked by conference organizers. as a us army veteran with hearing impairment who does not read lips, this was something i struggled with on my own until this year when online meetings meant i had to start asking for more accommodations. this is not always a comfortable or easy thing for anyone to do despite how accepting society says we are to helping those who need accommodations. dlf conference staff began reaching out by email prior to the conference to address these concerns and confirm the availability of captioning and transcription services for all sessions &# ; something that has not been my experience at other conferences. on the first day of the conference when i could not understand how to make the captioning features work, dlf had staff on hand via zoom link to jump on a screen share to show me how things worked on the aviary platform in real time. they emailed me links to sessions i missed, coached me through the features as i tested the captioning, showed me how transcriptions worked and even where i could have emailed transcripts of the conference sessions and slide notes to access at a later time. normally i would have to stop videos and play them back multiple times to get content for note taking. personally, these tools meant participating in the conference with fewer distractions and less stress. as the conference progressed, i thought more about the importance of these tools for our user community and how we can employ them to improve their experiences. dr. stacey patton&# ;s words during her opening address stayed with me throughout the conference as i considered the topic of accessibility in my daily work. she asked attendees to consider our role as archivists. as a new archivist trying to develop policies, procedures and workflows that encourage access and use for those coming to my archive, how can i perform my work more inclusively? are there communities being overlooked? what needs are going unmet and how do i address them? current discussions and work around accessibility are critical to ensuring that all patrons have access to the work we perform. the dlf forum was a great experience for me in that i was able to learn about new tools to help with accessibility like otter.ai and how to use it. i also learned about adjusting on the fly, accepting that sometimes technology will malfunction even with the best laid plans, having a backup plan to back up your plan is a great plan and finally, asking your community of peers for help may be your best plan. i am fortunate to have been able to attend dlf this year and to see the hard work of so many colleagues on display, learn new skills and connect with a community of professionals working in my field. there are many ways organizations demonstrate to their community of users &# ;you are welcome here. we want you here and you are part of this community.&# ; these simple steps taken by the dlf conference team through the accessibility tools helped me to feel part of the community this year. thank you to staff who provided assistance with read more the post dlf forum community journalist reflection: hsiu-ann tom appeared first on dlf. announcing a portuguese translation of the levels of digital preservation matrix portuguese translations of the levels of digital preservation matrix the ndsa is pleased to announce that version ( ) of the levels matrix has been translated into portuguese by laura vilela r. rezende. this document enriches the scientific studies on digital preservation and research data curation developed by the brazilian research group of which the researcher is part: the research network &# ; driade: digital preservation studies and practices links to these documents are found below as well as on the levels of digital preservation osf project page: https://osf.io/qgz / v . ( ) if you would be interested in translating the levels of digital preservation v . into another language please contact us at ndsa.digipres@gmail.com. tradução para o português da matriz dos níveis de preservação digital de a ndsa tem o prazer de anunciar que a versão . ( ) da matriz dos níveis de preservação digital foi traduzida para o português por laura vilela r. rezende. este documento enriquece os estudos científicos sobre preservação digital e curadoria de dados de pesquisa desenvolvidos pelo grupo de pesquisa brasileiro do qual a pesquisadora faz parte: rede de pesquisa driade &# ; estudos e práticas de preservação digital a seguir os links para acesso a este documentos. É possível acessar também pela página do projeto osf: https://osf.io/qgz / v . ( ) caso tenha interesse em traduzir os níveis de preservação digital v . em outro idioma, por favor entre em contato conosco pelo e-mail: ndsa.digipres@gmail.com the post announcing a portuguese translation of the levels of digital preservation matrix appeared first on dlf. additions to ndsa membership in summer and fall since the spring of , the ndsa leadership unanimously voted to welcome new members. each of these new members brings a host of skills and experience to our group. please help us welcome: arizona state university library: with many of their materials from local indigenous and latinx communities, the library is working with researchers from these communities to archive and preserve collections and artifacts unique to our region, making them accessible for generations to come. arkevist: a civil society that specializes in historical and genealogical research discoverygarden: for more than a decade, discoverygarden has been building trusted repositories and digital asset management systems for organizations around the world. global connexions: for two decades federick zarndt has provided consulting services to cultural heritage organizations and has contributed to ndsa, ala, ifla and alto. lyrasis: they are the non-profit organizational home of several open source projects that are focused on collecting, organizing, and ensuring long-term access to digital content including dspace, archivesspace, collectionspace, islandora, fedora repository, and duracloud. michigan digital preservation network: mdpn is an imls-grant funded initiative to build a member-run statewide distributed digital preservation network with members ranging from libraries, archives, museums, and historical societies with the primary purpose of preserving cultural heritage materials robert l. bogomolny library &# ; university of baltimore: robert l. bogomolny library is in the midst of a five year digital preservation implementation based upon results derived from conducting institutional readiness and digital preservation capability maturity model exercises. their special collections and archives hold sizable digital collection materials, including tbs of digitized local tv news. university of pennsylvania libraries: the penn libraries are working on many digital preservation activities, including but not limited to the ongoing development of a samvera repository, web archiving initiatives, conducting a pilot of two preservation storage systems, and developing governance for workflows and policies in order to have robust and programmatic digital preservation practices. university of victoria libraries: the uvic libraries are currently involved in a number of digital preservation-related infrastructure projects, including council of prairie and pacific university libraries (coppul) archivematica-as-a-service and westvault (a lockss-based preservation storage network), and serve as infrastructure hosts for the canadian government information preservation network (cgi-pn), the public knowledge project preservation network (pkp-pn), and perma.cc. university of wisconsin-milwaukee: over the past five years uwm has formed a digital preservation community of practice whose aim is to identify common digital preservation issues across departments and shared tools and workflows. uwm also co-founded the digital preservation expertise group (dpeg), a university of wisconsin system-wide group that shares digital preservation expertise, develops training, and investigates shared resources across all thirteen uw system libraries. each organization has participants in one or more of the various ndsa interest and working groups – so keep an eye out for them on your calls and be sure to give them a shout out. please join me in welcoming our new members. to review our list of members, you can see them here. ~ dan noonan, vice chair of the coordinating committee the post additions to ndsa membership in summer and fall appeared first on dlf. announcing spanish translations for the and levels matrix the ndsa is pleased to announce that both the original ( ) and version ( ) of the levels matrix have been translated into spanish by our colleagues from mexico and spain, dr. david leija (universidad autónoma de tamaulipas) and dr. miquel térmens (universitat de barcelona). drs. leija and térmens are academic researchers and founders of apredig (ibero-american association for digital preservation), a non-profit organization focused on spreading the importance of good practices of digital preservation for the spanish-speaking community. links to these documents are found below as well as on the levels of digital preservation osf project pages: (https://osf.io/qgz /) and (https://osf.io/ ya c/) as well as below. v . ( ) https://osf.io/wpdn &# ; http://www.apredig.org/npdndsa / v . ( ) https://osf.io/egjk &# ; http://www.apredig.org/npdndsa / in addition, miquel térmens and david leija have written a report analyzing and documenting the use of the ndsa levels in public and private organizations in spain, mexico, brazil and switzerland. the methodology of digital preservation audits with ndsa levels, can be found in spanish here and should be cited as found below. térmens, miquel; leija, david ( ). “methodology of digital preservation audits with ndsa levels”. el profesional de la información, v. , n. , pp. - . https://doi.org/ . /epi. .may. &# ; https://fima.ub.edu/pub/termens/docs/epi-v n .pdf if you would be interested in translating the levels of digital preservation v . into another language please contact us at ndsa.digipres@gmail.com. traducciones al español de la matriz de niveles de preservación digital y la ndsa se complace en anunciar que tanto la versión original como la versión de la matriz de niveles de preservación digital han sido traducidas al español por nuestros colegas investigadores de méxico y españa, el dr. david leija (universidad autónoma de tamaulipas) y el dr. miquel térmens (universitat de barcelona). térmens y leija son investigadores académicos fundadores de apredig (asociación iberoamericana de preservación digital), una organización sin ánimo de lucro enfocada en difundir la importancia de las buenas prácticas de preservación digital para la comunidad hispanohablante. los enlaces a estos documentos traducidos se encuentran a continuación, así como en las páginas del proyecto osf de niveles de preservación digital: ((https://osf.io/qgz /) y (https://osf.io/ ya c/). v . ( ) https://osf.io/wpdn &# ; http://www.apredig.org/npdndsa / v . ( ) https://osf.io/egjk &# ; http://www.apredig.org/npdndsa / adicionalmente, miquel térmens y david leija han escrito un reporte analizando y documentando el uso de los niveles ndsa en organizaciones públicas y privadas de españa, méxico, brasil y suiza. la auditoría de preservación digital con ndsa levels, se puede encontrar en español aquí y debe citarse como se encuentra a continuación. térmens, miquel; leija, david ( ). “auditoría de preservación digital con ndsa levels”. el profesional de la información, v. , n. , pp. - . https://doi.org/ . /epi. .may. &# ; https://fima.ub.edu/pub/termens/docs/epi-v n .pdf si está interesado en traducir los niveles de preservación digital v . en otros idiomas por favor póngase en contacto en ndsa.digipres@gmail.com. &# ; the post announcing spanish translations for the and levels matrix appeared first on dlf. ndsa announces winners of innovation awards the ndsa established its innovation awards in to recognize and encourage innovation in the field of digital stewardship. since then, it has honored exemplary educators, future stewards, individuals, institutions, and projects for their efforts in ensuring the ongoing viability and accessibility of our valuable digital heritage. the ndsa innovation awards are generously sponsored by digital bedrock. today, ndsa adds new awardees to that honor roll during the opening plenary ceremony of the ndsa digital preservation conference. these winners were selected from the largest pool of nominees so far in the awards’ history: nominations of nominees. while the pool size made the judging more difficult, the greater breadth, depth, and quality of the nominations is a positive sign for the preservation community, as it is indicative of the growing maturity and robustness of the field. this year’s awardees continue to reflect a recent trend towards an increasingly international perspective and recognition of the innovative contributions by and for historically underrepresented and marginalized communities. please help us congratulate these awardees! we encourage you to follow-up in learning more about their activities and the ways in which they have had a profound beneficial impact on our collective ability to protect and make accessible our valuable digital heritage. educators are recognized for innovative approaches and access to digital preservation through academic programs, partnerships, professional development opportunities, and curriculum development. this year’s awardees in the educators category are: library juice academy certificate in digital curation. this program, launched in , encompasses a six-course sequence for library, archives and museum practitioners wanting to learn more about and expand their skill sets for curating and maintaining unique digital assets. the curriculum offers comprehensive coverage of collection development and appraisal, description, rights and access, digital preservation, and professional ethics and responsible stewardship. the program’s affordability, flexible scheduling, and online pedagogy encouraging engaged collaborative learning provides a unique opportunity for professional development and continuing education. in particular, the emphasis placed on ethics and sustainability provides an appropriate counterpoint to other more technically-focused topics, drawing needed attention to critical issues of policy, finance, equity, and diversity. international council on archive (ica) africa programme digital records curation programme. the programme supports the professional development of new generations of digital archivists and records managers in africa, a geographic and cultural region historically marginalized and underrepresented in international digital stewardship discourse, practice, and education. the programme’s volunteer-taught study school uses open access readings and open source tools to minimize technical resource and financial impediments to participation, and to encourage creative repurposing of pedagogic materials in the participants’ local contexts. the programme also provides financial support for early-career practitioners and educators across the african continent to attend and learn, share their own teaching techniques and insights, and to build a professional research and teaching network. parallel instructional opportunities are offered for anglophone and francophone participants. with a focus on “training the trainers”, the digital records curation programme promotes the development of maturing cohorts of stewardship practitioners and the growing professionalism of digital preservation activities focused on long-term stewardship of africa’s vital digital heritage. &# ; future stewards are recognized as students and early-career professionals or academics taking a creative approach to advancing knowledge of digital preservation issues and practices. these year’s awardees in the future stewards category are: sawood alam. a phd candidate at old dominion university, sawood has been an active participant in the digital preservation community via the international internet preservation consortium, the acm/ieee joint conference on digital libraries, and other communities for years, presenting and reporting on the complex topics, like holdings of web archives, decentralized systems, archival fixity, web packaging, and more. as a developer and systems architect, sawood is a strong advocate for open-source and open-access tools, and has offered courses and lectures on various programming languages like linux, python, ruby on rails, and more. a mentor to new graduate students and researchers, sawood will join the internet archive after graduation, leveraging his engineering experience and his academic experience to perform outreach to research groups interested in making use of the wayback machine’s holdings. &# ; &# ; carolina quezada meneses. as an intern, carolina worked on a variety of projects that ranged from exploring new tools and software that help preserve, manage, and provide access to born-digital material, and helped develop a remote processing workflow that enabled university of california, irvine (uci) staff to work on the organization’s digital backlog while working from home during the coronavirus pandemic. however, it is meneses’s work with the christine tamblyn papers — which included numerous macintosh-formatted floppy disks and cd-roms — that deserves additional praise: faced with ample technical challenges to providing access, quezada created disk images of the floppy disks and cd-roms with specialized hardware, found a compatible emulator, and created screencast videos of the artwork, making the content accessible to a broader audience than traditional on-site access would typically allow. thanks to meneses’s innovative thinking, a collection that had no prior level of access for years is now accessible to researchers, and remains an example of her lasting dedication to providing access to born-digital formats. &# ; organizations are recognized for innovative approaches to providing support and guidance to the digital preservation community. this year’s awardee in the organizations category is: national archives and records administration (nara). nara has a notable history of providing records management guidance focusing on digital preservation and addressing key factors to the successful permanent preservation of digital content. this year, the panel is pleased to distinguish nara’s digital preservation framework. created after an extensive environmental scan of community digital preservation risk assessment and planning resources, this project recognizes that successful digital preservation requires both understanding the risks posed by file formats and identifying or developing processes for mitigating these risks. in response to this, the framework provides extensive risk and planning analysis for over formats in type categories. the framework can be applied across the lifecycle of digital content and is designed to enable a low-barrier read more the post ndsa announces winners of innovation awards appeared first on dlf. award winners: ndsa levels of digital preservation group this year’s world digital preservation day (#wdpd) was the biggest yet! with outpourings of research, achievements, practical advice, and fun it was hard to believe that there were also awards as part of that process. on november, the ndsa’s levels of digital preservation reboot was the recipient of one of the digital preservation coalition’s digital preservation award! we won in the ica-sponsored category for collaboration and cooperation &# ; the first time it has been awarded! this honor is collectively bestowed on the many of you who helped craft and refine the levels and we hope your continued ideas, and enthusiasm will keep the momentum going. thank you for all your hard work! for an overview, background, and charge for the levels, see my blog post that speaks to leveraging such a high level of collaborative energy. ~ bradley daigle, levels of digital preservation steering group lead the post award winners: ndsa levels of digital preservation group appeared first on dlf. meet the dlf forum community journalists the virtual dlf forum looks different from our typical event in almost every way imaginable. due to the fact that we aren’t convening in person and registration is free, we decided to offer a different kind of fellowship opportunity. because the guiding purpose of this year’s virtual dlf forum is building community while apart, through our re-envisioned fellowship program, we are highlighting new voices from “community journalists” in the field. we are providing $ stipends to a cohort of virtual dlf forum attendees from a variety of backgrounds and will feature their voices and experiences on the dlf blog after our events this fall. we are excited to announce this year’s dlf forum community journalists: arabeth balasko arabeth balasko (she/her) is an archivist and historian dedicated to public service and proactive stewardship. as a professional archivist, her overarching goals are to curate collections that follow a shared standardization practice, are user-centric, and are searchable and accessible to all via physical and digital platforms. she believes that an archive should be a welcoming place for all people and should be an inclusive environment which advocates to collect, preserve, and make accessible the stories and histories of diverse voices. by getting individuals involved in telling their story and making their history part of the ever-growing story of humanity, we all win! &# ; rebecca bayeck @rybayeck rebecca y. bayeck is a dual-phd holder in learning design &# ; technology and comparative &# ; international education from the pennsylvania state university. currently a clir postdoctoral fellow at the schomburg center for research in black culture where she engages in digital research, data curation, and inclusive design. her interdisciplinary research is at the interface of several fields including the learning sciences, literacy studies, and game studies. at this intersection, she explores literacies and learning in games, particularly board games, the interaction of culture, space, and context on design, learning, research, literacies. &# ; shelly black @shellyyblack shelly black is the cyma rubin library fellow at north carolina state university libraries where she supports digital preservation in the special collections research center. she also works on a strategic project involving immersive technology spaces and digital scholarship workflows. previously she was a marketing specialist at the university of arizona libraries and promoted library services and programs through social media, news stories, and newsletters. shelly was recently selected as a emerging leader by the american library association and is a provisional member of the academy of certified archivists. she received a mlis and a certificate in archival studies from the university of arizona where she was a knowledge river scholar. she also holds a bfa in photography and minor in japanese from the ua. &# ; lisa covington @prof_cov lisa covington, ma is a phd candidate at the university of iowa studying sociology of education, digital humanities and african american studies. her dissertation work is “mediating black girlhood: a multi-level comparative analysis of narrative feature films.” this research identifies mechanisms in which media operates as an institution, (mis)informing individual and social ontological knowledge. in , lisa received the rev. dr. martin luther king, jr. award from the iowa department of human rights. she is the director of the ethnic studies leadership academy in iowa city, an educational leadership program for black youth, in middle school and high school, to learn african american advocacy through incorporating digital humanities and social sciences. lisa received her ma from san diego state university in women &# ; gender studies. as a youth development professional, lisa develops curriculum for weekly programming with girls of color, trains teachers on best practices for working with underrepresented youth, and directs programs in preschool through college settings in california, pennsylvania, iowa, new jersey, new york and washington, d.c. ana hilda figueroa de jesús ana hilda figueroa de jesús will be graduating next spring from the universidad de puerto rico in río piedras with a ba in history of art. her research interest focuses on education, accessibility and publicity of minority, revolutionary puerto rican art including topics such as race, gender and transnationalism. she has interned at visión doble: journal of criticism and history of art, and volunteered at meca international art fair and instituto nueva escuela. ana works as assistant for the curator and director of the museum of history, anthropology and art at upr. she is currently a katzenberger art history intern at smithsonian libraries. amanda guzman amanda guzman is an anthropological archaeologist with a phd in anthropology (archaeology) from the university of california, berkeley. she specializes in the field of museum anthropology with a research focus on the history of collecting and exhibiting puerto rico at the intersection of issues of intercultural representation and national identity formation. she applies her collections experience as well as her commitment to working with and for multiple publics to her object-based inquiry teaching practice that privileges a more equitable, co-production of knowledge in the classroom through accessible engagement in cultural work. amanda is currently the ann plato post-doctoral fellow in anthropology and american studies at trinity college in hartford, ct. &# ; carolina hernandez @carolina_hrndz carolina hernandez is currently an instruction librarian at the university of houston where she collaborates on creating inclusive learning environments for students. previously, she was the journalism librarian at the university of oregon, where she co-managed the oregon digital newspaper program. her mlis is from the university of wisconsin-madison. her current research interests are in critical information literacy, inclusive pedagogy, and most recently, the intersection of digital collections and pedagogy. &# ; jocelyn hurtado jocelyn hurtado is a native miamian who worked as an archivist at a community repository for four year. she is experienced in working with manuscript, art and artifact collections pertaining to a community of color whose history has often been overlooked. ms. hurtado, understands the responsibility and the significance of the work done by community archivists and has seen firsthand that this work not only affects the present-day community but that it will continue to have a deep-rooted impact on generations read more the post meet the dlf forum community journalists appeared first on dlf. dlf forum: building community with dlf’s digital library pedagogy working group though dlf is best known for our signature event, the annual dlf forum, our working groups collaborate year round. long before covid- introduced the concept of “zoom fatigue” into our lives, dlf’s working groups organized across institutional and geographical boundaries, building community while apart, to get work done. made possible through the support of our institutional members, working groups are the efforts of a committed community of practitioners, using dlf as a framework for action, engaged in problem-solving in a variety of digital library subfields from project management and assessment to labor and accessibility. once we decided that the dlf forum and affiliated events would be held in a virtual format, it meant that our working groups wouldn’t have the opportunity to meet in person for their typical working meals that take place throughout the forum; however, this year’s virtual format means that we’ll have more new dlf forum attendees than ever before. because dlf’s working groups are open to all, regardless of whether you’re affiliated with a dlf member institution or not, we asked leaders of the dlf working groups to introduce their groups and the work they do to the new and returning members of the #dlfvillage in a series of blogs and videos. we’ll share these working group updates in the days leading to this year’s dlf forum. who are we? the dlf digital library pedagogy working group, commonly referred to as #dlfteach (also our twitter hashtag), was founded in and is focused on building a community of practice for those interested in using digital library collections and technology in the classroom. the group is open for anyone to join regardless of your position, academic discipline, or dlf institutional affiliation. here is what #dlfteach does and the ways you can join us: twitter chats one of the best ways to get involved with #dlfteach is to participate in a twitter chat. our twitter chats offer a chance to chat with colleagues from all over on different subjects each chat. every chat has a host or two who plan the topic and write questions that will be tweeted at intervals over the course of one hour. participants can follow the questions tweeted from the @clirdlf handle and respond from their own twitter account. hosts will monitor the chat and also tweet frequently. to see all the tweets as they happen, the hashtag #dlfteach is included with every tweet, and participants should likewise add it to their tweets. people can participate as much or as little as possible, ranging from lurking to tweeting answers and replying to others’ tweets. twitter chats usually take place at - pm est / am &# ; noon pst on the third tuesday of every other month. once or twice a year, the chat will take place at another time for those who cannot make the regular time. you can see previous chats on the group’s wiki. interested in hosting a chat? want to suggest a topic? get in touch with the outreach coordinators of the dlf digital library pedagogy group! past projects #dlfteach is a uniquely project-based working group, and we are usually working on a couple of projects at any given time of year. typically, members propose or are made aware of projects that would benefit from the expertise and dedication of group members working to implement them. if you are interested in our group’s focus and are looking to get involved, you are welcome to propose a project. if you do not have a specific project in mind but still want to get involved, that’s great, too, since these projects offer many opportunities to contribute to the community and the profession. you may be wondering: what projects does #dlfteach work on? in september , we released #dlfteach toolkit . , an openly available, peer reviewed collection of lesson plans and concrete instructional strategies edited by erin pappas and liz rodrigues and featuring the work of many #dlfteach members and affiliates. check it out to get ideas of how to incorporate digital library collections and technologies into the classroom in structured, reproducible ways. another resource developed by #dlfteach is the teaching with digital primary sources white paper, by brianna gormly, maura seale, hannah alpert-abrams, andi gustavson, angie kemp, thea lindquist, and alexis logsdon, which outlines literacies and considers issues associated with finding, evaluating, and citing digital primary resources. if you are considering using digital primary sources in the classroom, this is an excellent resource to accompany your work with these materials. additionally, #dlfteach has developed and facilitated workshops at the dlf forum and learn@dlf pre-conferences in , , and . current projects following the success of the first version released last year, we have issued a call for participation for the #dlfteach toolkit . , which will focus on instructional strategies using immersive technology. we are looking for both contributors and volunteers to assist with reviewing submissions and producing the toolkit. additionally, we are currently working on two blog series! one is focused on ethical issues for multimodal scholarship and pedagogy, and the other, practitioner perspectives: developing, adapting, and contextualizing the #dlfteach toolkit, is collecting interviews from practitioners (via google form) who have used or adapted #dlfteach toolkit lesson plans. look for these to be published in the coming months as well as calls to participate. how can you get involved? anyone is welcome to join and participate in the digital library pedagogy group and help grow the community of practitioners around teaching with digital library collections and tools. our next twitter chat will be on december at : pm est and will be focused on ways #dlfteach can help build community and support each other with the projects and ongoing initiatives we work on. have you used or adapted lesson plans from the #dlfteach toolkit . ? add your voice to practitioner perspectives: developing, adapting, and contextualizing the #dlfteach toolkit, a forthcoming blog series! just answer our questions on this google form. additionally, please consider joining our google group and read more the post dlf forum: building community with dlf’s digital library pedagogy working group appeared first on dlf. louisa kwasigroch appointed interim dlf senior program officer the council on library and information resources (clir) is pleased to announce the appointment of louisa kwasigroch as interim digital library federation (dlf) senior program officer. kwasigroch, who currently serves as clir’s director of outreach and engagement and has worked extensively with the dlf community, will serve as the primary point of contact for dlf member institutions and individuals until a permanent senior program officer has been appointed. during the interim period, she will also continue to serve as director of outreach and engagement. “i’m delighted louisa has accepted this interim appointment,” said clir president charles henry. “with her knowledge of dlf’s engaged and active community, she will bring an empathetic and insightful continuity that will position us strategically for the next phase of dlf’s evolution.” kwasigroch has been in the library field for more than years, working with public, museum, and academic libraries. she has her ba in photography from columbia college chicago, and both an mslis and mba from the university of illinois, urbana-champaign. she began her career with clir in as a dlf program associate and was promoted to director of development and outreach in and director of outreach and engagement in . “it has been a great joy to serve the dlf community these past seven years in my roles at clir,” said kwasigroch. “i look forward to continuing to support our members, working groups, and constituents while collaborating even more closely with clir and dlf staff, who have been doing an amazing job moving things forward.” clir will resume its search for a permanent senior program officer in january . the post louisa kwasigroch appointed interim dlf senior program officer appeared first on dlf. dlf forum: building community with dlf’s data and digital scholarship working group though dlf is best known for our signature event, the annual dlf forum, our working groups collaborate year round. long before covid- introduced the concept of “zoom fatigue” into our lives, dlf’s working groups organized across institutional and geographical boundaries, building community while apart, to get work done. made possible through the support of our institutional members, working groups are the efforts of a committed community of practitioners, using dlf as a framework for action, engaged in problem-solving in a variety of digital library subfields from project management and assessment to labor and accessibility. once we decided that the dlf forum and affiliated events would be held in a virtual format, it meant that our working groups wouldn’t have the opportunity to meet in person for their typical working meals that take place throughout the forum; however, this year’s virtual format means that we’ll have more new dlf forum attendees than ever before. because dlf’s working groups are open to all, regardless of whether you’re affiliated with a dlf member institution or not, we asked leaders of the dlf working groups to introduce their groups and the work they do to the new and returning members of the #dlfvillage in a series of blogs and videos. we’ll share these working group updates in the days leading to this year’s dlf forum. what is the dlf digital scholarship and data services working group? the dlf data and digital scholarship working group (dlfdds) is a continuation of two dlf groups: the eresearch network and the digital scholarship working group. the current version of the group uses a mutual aid model to offer peer leaders and the group the ability to create topics of interest for the community. it is an evolution of the eresearch network that dlf ran for many years. sara mannheimer, data librarian, montana state university and jason clark, lead for research informatics, montana state university will be facilitating the working group this year. our charge notes that we are “a community of practice focused on implementing research data and digital scholarship services. the group focuses on shared skill development, peer mentorship, networking, and collaboration. dlfdds aims to create a self-reliant, mutually supportive community: a network of institutions and individuals engaged in continuous learning about research data management, digital scholarship, and research support.” learn more about us: our digital scholarship and data services wiki our digital scholarship and data services google group (listserv) what are we working on? we meet quarterly for discussion and activities based on dlf dds community interest and ideas. past topics have included: advocacy and promotion of data services and digital scholarship, data discovery/metadata and reusability, collections as data, assessment (metrics for success with data services and digital scholarship), etc. last month, we met to talk about roadmapping in a session led by shilpa rele scholarly communication &# ; data curation librarian, rowan university. view slides and video these sessions have a flexible focus between rdm and ds. these minutes each quarter are structured around a particular topic and usually involve: a short visit from invited speaker on topic an in-session discussion and activity we are basing this structure on the former eresearch network (ern) cohort model which had a more of a course-based mode. an example eresearch network syllabus is linked here to give you some more perspective on the history of that group. our new goal is to bring the best parts of ern into this revitalized working group. we also connect folks in the working group around consultation ideas. consultations are working sessions that give consultees a chance to work through an in-depth, peer conversation to solve a local data services or digital scholarship question. consultants are peers and associated experts (e.g. fellow dlfdds members, former eresearch network participants, practitioners from other dlf member institutions). consultations are flexible and customized according to consultee needs. how to contribute or get involved? as we are working to instill a mutual aid model for our community, we are interested in your ideas. we have opened a survey to pull together these interests and welcome your thoughts. take our dlf dds interest and ideas survey: https://bit.ly/dlf-dds-survey beyond the survey, please feel free to join our google group as announcements and opportunities related to the wg and digital scholarship/data services in general will be available there. our next scheduled meeting will be in december . we hope to see you there! the post dlf forum: building community with dlf’s data and digital scholarship working group appeared first on dlf. dlf forum: building community with dlf’s metadata support group though dlf is best known for our signature event, the annual dlf forum, our working groups collaborate year round. long before covid- introduced the concept of “zoom fatigue” into our lives, dlf’s working groups organized across institutional and geographical boundaries, building community while apart, to get work done. made possible through the support of our institutional members, working groups are the efforts of a committed community of practitioners, using dlf as a framework for action, engaged in problem-solving in a variety of digital library subfields from project management and assessment to labor and accessibility. once we decided that the dlf forum and affiliated events would be held in a virtual format, it meant that our working groups wouldn’t have the opportunity to meet in person for their typical working meals that take place throughout the forum; however, this year’s virtual format means that we’ll have more new dlf forum attendees than ever before. because dlf’s working groups are open to all, regardless of whether you’re affiliated with a dlf member institution or not, we asked leaders of the dlf working groups to introduce their groups and the work they do to the new and returning members of the #dlfvillage in a series of blogs and videos. we’ll share these working group updates in the days leading to this year’s dlf forum. the metadata support group is excited to be part of the dlf forum virtual community. we would like to share an open invitation for anyone who works with metadata to join our community. the metadata support group was founded in on slack in order to provide colleagues from the glam (galleries, archives, libraries, museum) community with a space to ask questions, get answers, and develop a network of colleagues from institutions across the country. the metadata support group has over members and channels dedicated to topics such as general questions, migrations, conferences, archivesspace, tools, workflows, and much more. anyone is welcome to join our community, just agree to our code of conduct when you fill out this form. if you have any questions about the group, feel free to reach out to the co-founders and facilitators: julie hardesty, jlhardes@iu.edu liz woolcott, liz.woolcott@usu.edu anna neatrour, anna.neatrour@utah.edu bria parker, brialparker@gmail.com the post dlf forum: building community with dlf’s metadata support group appeared first on dlf. dlf forum: building community with dlf’s labor working group though dlf is best known for our signature event, the annual dlf forum, our working groups collaborate year round. long before covid- introduced the concept of “zoom fatigue” into our lives, dlf’s working groups organized across institutional and geographical boundaries, building community while apart, to get work done. made possible through the support of our institutional members, working groups are the efforts of a committed community of practitioners, using dlf as a framework for action, engaged in problem-solving in a variety of digital library subfields from project management and assessment to labor and accessibility. once we decided that the dlf forum and affiliated events would be held in a virtual format, it meant that our working groups wouldn’t have the opportunity to meet in person for their typical working meals that take place throughout the forum; however, this year’s virtual format means that we’ll have more new dlf forum attendees than ever before. because dlf’s working groups are open to all, regardless of whether you’re affiliated with a dlf member institution or not, we asked leaders of the dlf working groups to introduce their groups and the work they do to the new and returning members of the #dlfvillage in a series of blogs and videos. we’ll share these working group updates in the days leading to this year’s dlf forum. tools for the lam labor situation the covid- pandemic has only intensified precarity in our field, but we don’t have to face it alone. over the last years, the labor working group (labor wg) has been building tools to improve library, archives, and museum (lam) labor conditions. nor are we the only ones: similar efforts are underway in many other groups. take a tour 🚴 of our resources below and join us as we work to build collective power to take on collective problems. collective responsibility labor advocacy toolkit in , members of labor wg organized and participated in the imls-funded collective responsibility: national forum on labor practices for grant-funded digital positions. one idea that surfaced at the forum was to collect the project’s outcomes along with other resources for documenting and changing the landscape of contingent labor in lam. we created the collective responsibility labor advocacy toolkit to publish these materials. during , a group of forum and labor wg members developed guiding principles, naming the concepts which have shaped our work so far and concretizing these into principles for our future work. another group turned working notes on “how to talk about collective responsibility at work” into a set of scripts and scenarios for discussing contingency and precarity in your actual workplace. we collaborated with members of the archival workers emergency fund organizing committee to develop these scenarios. in response to the pandemic, we assembled a page of emergency resources for contingent or suddenly-precarious lam workers. valuing labor in digital libraries in we published a research agenda identifying areas for investigation and action in lam labor: recognizing labor as being undervalued, unacknowledged, and erased has long been framed as an individual concern in the field of digital libraries, archives, and museums (lam); but organized and collective action is required to address labor conditions at structural and organizational scales. such action requires information, from empirical evidence to testimonies to guidance and best practices. this document lays out a research agenda for valuing labor, collaboratively developed by members of the digital library federation working group on labor in digital libraries, archives, and museums (labor wg). we intend for research building upon this agenda to also be collaborative: by and for the dlf community. each section of the agenda describes a key research area and why it matters, offering questions and research designs that bear investigating. in early , a few members of labor wg set out to research these questions: to what extent and how are digital lam workers organized? what organizing and advocacy tactics can be effective while minimizing risk for workers? what are creative and effective ways to share data about digital lam labor? we got totally derailed (thanks, covid) but those questions are still out there and still important, probably more than ever. if you build something from the research agenda, we’d love to hear about it. do better, love us in january , we published a document that has been underway since the working group’s formation. “do better” -love(,) us is built around a simple principle: when funding a term position, fund a good position. if you’re writing a grant, it means asking for what you need for such a position, not lowballing on labor in order to win a grant. for funders, it means approving those applications, even if it means funding fewer projects. the cost of these projects should not fall on the workers least equipped to bear it. our work on this document inspired the collective responsibility project. while it sparked conversations at the forums and shaped some of the project’s outcomes, and our revision used ideas, data, and new collaborations developed during the forum, we kept this document within labor wg. it was important to us to retain this document’s aspirational nature, rather than reducing it to the most practical next steps available at this time. connect with lam labor organizers this work is much too big for just one group. members of labor wg didn’t want to build one-time solutions or reinvent the wheel, so we went looking for others advocating and organizing to improve the situation of lam workers. we built a list of organizations and campaigns (see it as a table or a list). we also built up a bibliography on lam labor, from scholarly articles to local news to blog posts to twitter threads. lately we&# ;ve been using news alerts to attend to what&# ;s happening to lam workers around the world during the pandemic and what they&# ;re doing about it. we know that people are losing their jobs, being furloughed, being replaced, losing pay and benefits, taking on new work outside of their job read more the post dlf forum: building community with dlf’s labor working group appeared first on dlf. dlf forum: building community with dlf’s working group on privacy and ethics in technology though dlf is best known for our signature event, the annual dlf forum, our working groups collaborate year round. long before covid- introduced the concept of “zoom fatigue” into our lives, dlf’s working groups organized across institutional and geographical boundaries, building community while apart, to get work done. made possible through the support of our institutional members, working groups are the efforts of a committed community of practitioners, using dlf as a framework for action, engaged in problem-solving in a variety of digital library subfields from project management and assessment to labor and accessibility. once we decided that the dlf forum and affiliated events would be held in a virtual format, it meant that our working groups wouldn’t have the opportunity to meet in person for their typical working meals that take place throughout the forum; however, this year’s virtual format means that we’ll have more new dlf forum attendees than ever before. because dlf’s working groups are open to all, regardless of whether you’re affiliated with a dlf member institution or not, we asked leaders of the dlf working groups to introduce their groups and the work they do to the new and returning members of the #dlfvillage in a series of blogs and videos. we’ll share these working group updates in the days leading to this year’s dlf forum. in this blog post, we briefly describe the mission and activities of the dlf privacy and ethics in technology working group. overview of the working group the dlf privacy and ethics in technology working group is based on the recognition that libraries are increasingly investing in systems that can collect, store, analyze, and potentially leak data related to user activities. the work of the group focuses on challenging and complicating our relationship with data collection technologies. we then seek to create tools and resources to help practitioners critically engage with these data collection technologies, with the goal of aligning our practice with privacy-oriented principles—for the ultimate betterment of our profession, our user communities, and society. resources produced by the working group members we have an active and thriving group membership. over the past few years, our members have collaborated to co-produce new resources to improve practice in support of privacy and responsible use of data-collecting technologies: a practical guide to performing a library user data risk assessment in library-built systems — https://osf.io/v c m/ vendor privacy policy analysis project — https://osf.io/ svz/ advocacy action plan — https://osf.io/ smrf/ digital privacy instruction curriculum — https://osf.io/sebhf/ ethics in research use of library patron data: glossary and explainer https://osf.io/bygj / joining the working group our working group is flexible and adaptable according to our members’ interests. if you would like to contribute to the production of resources like those listed above, or if you want to help build a new resource that addresses a different topic in this area, consider joining our group. we have a wiki available through the dlf website that provides more information: https://wiki.diglib.org/privacy_and_ethics_in_technology. future directions moving ahead, we as co-conveners want to emphasize greater collaboration and visibility with other groups and organizations that focus on privacy, ethics, and the responsible use of technology. through collaboration and cross-learning with other related organizations—including those in adjacent fields—we hope to strengthen our work in pursuit of shared goals. we&# ;re especially emphasizing building deeper relationships with public librarians and librarians of diverse backgrounds. we hope that you&# ;re doing well wherever you are, and we hope that we’ll be talking to you soon about privacy and ethics in technology. — scott w. h. young, ux &# ; assessment librarian, montana state university — michelle gibeault, scholarly engagement librarian for the humanities, tulane university the post dlf forum: building community with dlf’s working group on privacy and ethics in technology appeared first on dlf. dlf forum: building community with dlf’s digital accessibility working group though dlf is best known for our signature event, the annual dlf forum, our working groups collaborate year round. long before covid- introduced the concept of “zoom fatigue” into our lives, dlf’s working groups organized across institutional and geographical boundaries, building community while apart, to get work done. made possible through the support of our institutional members, working groups are the efforts of a committed community of practitioners, using dlf as a framework for action, engaged in problem-solving in a variety of digital library subfields from project management and assessment to labor and accessibility. once we decided that the dlf forum and affiliated events would be held in a virtual format, it meant that our working groups wouldn’t have the opportunity to meet in person for their typical working meals that take place throughout the forum; however, this year’s virtual format means that we’ll have more new dlf forum attendees than ever before. because dlf’s working groups are open to all, regardless of whether you’re affiliated with a dlf member institution or not, we asked leaders of the dlf working groups to introduce their groups and the work they do to the new and returning members of the #dlfvillage in a series of blogs and videos. we’ll share these working group updates in the days leading to this year’s dlf forum. the dlf digital accessibility working group (dawg) is a group dedicated to exploring issues around ensuring efficient access for disabled people in information organizations. within dawg, there are three subgroups: advocacy and education, it and development, and policies and workflows. these three areas were selected because they were identified as essential to ensuring a holistic approach to the adoption, implementation, and maintenance of inclusive practices for disabled users, staff, and other information organization stakeholders. dawg subscribes to the idea that in order for technology, or “the digital library,” to be accessible, it is not enough that the software, hardware, and accompanying resources be accessible. instead, there is a need within organizations and outside vendors for: cultural change regarding awareness of the disability rights efforts of the past and present via continuing education and advocacy work. adaptable policies and workflows that are inclusive of disabled people and enforced on an institutional level via allocation of financial, legal, and human resources. emphasis on inclusive design and development practices to ensure that technologies are designed to be accessible, as well as to be maintained sustainably with accessibility in mind. though only founded within the last year, dawg has managed to initiate meaningful dialogue around these issues while acting as a source of community for information professionals looking to navigate the challenges posed, both by the field in general, as well as the more contemporary challenges posed by covid- . it and development the it and development subgroup focuses on specific software, hardware, and development practices associated with information organizations. from a list of suggested software, we’ve already developed a deep-dive into the accessibility of zoom, and are gathering information on many other software applications. ultimately, we’d like to develop a system for the glam community to easily ask questions or share information about the accessibility of software, building off the existing networks and centers for information for at professionals. some of the other projects we’ve done include the accessible documentation guidelines and a collection of accessibility auditing resources. for more information, contact debbie krahmer dkrahmer@colgate.edu advocacy and education libraries and information organizations have a responsibility to proactively build features into our products and services that recognizes the value and rights of people with disabilities. this requires continual learning as well as creative and collaborative advocacy. as a new working group, the advocacy and education subgroup seeks contributors who welcome complexity and responsibility. this means, when possible, fixing a regular, monthly meeting time in our calendars, as well as reserving time throughout the month to do a bit of work. most recently, this sub-group hosted a kick-off webinar, inclusive design and accessible exhibits: some guidance for libraries, galleries, and museums by sina bahram. advocacy and continuing education wiki page for information, contact mark weiler (mweiler@wlu.ca) policies and workflows glam organizations incorporate a wide variety of policies and workflows, with varying levels of success. this subgroup is collecting best practices, implementation processes, and educational materials to help individuals in glam organizations authentically implement accessibility policies and workflows in digital libraries. some of the work we have embarked on thus far has included defining the scope of digital library policies and workflows, conducting an environmental scan of policies and workflows, and creating a toolkit to be used with digital libraries. the toolkit will contain a reflection of many of the conversations the subgroup has had revolving around the successful implementation of policies and workflows &# ; and not only the suggested content glam institutions will want to adopt. the individuals working together represent a wide variety of types of institutions, locations, and experience. each individual has had responsibilities in their home organization to implement accessible policies and workflows in regards to digital libraries. for scholars and practitioners alike, the work being conducted to create recommendations represents the practical knowledge others need. to learn more about the work, completed resources are available on the dlf wiki subpage for policies and workflows. to join in the conversation, please email the co-chairs, gabriel galson at galson@temple.edu and amy vecchione at amyvecchione@boisestate.edu. getting involved if you want to learn more about the work dawg is doing, check out the digital accessibility group wiki. you can also join the listserv (google group), attend meetings (updates about meeting times are shared via the listserv, and join the slack channel. for additional questions regarding the group, feel free to email jasmine clark at jasmine.l.clark@temple.edu. the post dlf forum: building community with dlf’s digital accessibility working group appeared first on dlf. dlf forum: building community with dlf’s project managers group though dlf is best known for our signature event, the annual dlf forum, our working groups collaborate year round. long before covid- introduced the concept of “zoom fatigue” into our lives, dlf’s working groups organized across institutional and geographical boundaries, building community while apart, to get work done. made possible through the support of our institutional members, working groups are the efforts of a committed community of practitioners, using dlf as a framework for action, engaged in problem-solving in a variety of digital library subfields from project management and assessment to labor and accessibility. once we decided that the dlf forum and affiliated events would be held in a virtual format, it meant that our working groups wouldn’t have the opportunity to meet in person for their typical working meals that take place throughout the forum; however, this year’s virtual format means that we’ll have more new dlf forum attendees than ever before. because dlf’s working groups are open to all, regardless of whether you’re affiliated with a dlf member institution or not, we asked leaders of the dlf working groups to introduce their groups and the work they do to the new and returning members of the #dlfvillage in a series of blogs and videos. we’ll share these working group updates in the days leading to this year’s dlf forum. all of our events are free of charge. register by november . join dlf&# ;s project managers group during their forum working session on november . what is the project managers group? the dlf project managers group (or dlf pmg) was formed in to acknowledge the intersection between project management and library technology. we provide a place to share project management methodologies and tools, alongside broader discussions that consider issues such as portfolio management and cross-organizational communication, and have branched into discussions of personnel management as well. we look to keep pace with the digital library landscape by bringing new and evolving project management practices to the attention and mutual benefit of our colleagues. this has been a big focus this year as many of us have made the transition to managing teams remotely. you can find more information about our group on the dlf pmg wiki. this year the project managers group decided to take on a new mentoring initiative as well as expand a webinar program series that we’ve offered on a limited basis to help connect and support project managers. mentoring: in june , the project managers group launched its pilot of a project management mentoring program. this was in response to feedback the steering committee has heard at meetings and past forums for several years. the committee wanted to provide an opportunity to connect with other project managers in our field to ask advice, share ideas, and learn from someone at a different organization. the goals of the program are to cultivate relationships and communication between project managers who have different experiences and areas of interest in digital libraries throughout the year. the committee envisioned this program as geared toward anyone in the field&# ;new professionals or experienced project managers&# ;who are looking to gain new perspectives. our first cohort of mentors and mentees are currently in their third month of the program and by all accounts, everyone is finding this program insightful and worthwhile. we recently sent out our first quarter survey for participants and got positive feedback from everyone on the experience so far. especially right now, as our professional lives are so upended, mentors and mentees are appreciative of someone to bounce ideas off and commiserate with – no matter their prior experience. based on the success of this pilot program, the steering committee will continue to offer this mentoring program and hope to call for a second cohort in spring . webinars: the project managers group provides webinars to keep pace with the dynamic digital library landscape, by bringing new and evolving project management, service design, user experience, and assessment practices to the dlf community. the webinars are recorded and uploaded to the dlf youtube channel and then linked to the pmg wiki so that they can be shared with a wider audience. this year, speakers presented on new technologies and practices in project management. melissa wisner, it project and portfolio management librarian at north carolina state university libraries, spoke about before action review, a method used by teams at the onset of a project to talk through anticipated changes, assumptions, and risks by drawing on the lessons learned from past experiences. jenn nolte, user experience (ux) librarian at yale university library, spoke about user experience and service design. the talk focused on the importance of putting the end user’s needs at the center when planning for new services or enhancing current services. casey davis kaufman, associate director of the gbh media library and archives, spoke about leading teams and projects while working remotely during the ongoing global pandemic crisis. the presentation covered a range of topics relevant to the present remote-working environment&# ;effective communication and support, managing expectations, measuring productivity and impact, providing resources, and &# ;holding space&# ; for team members who may be experiencing incredibly challenging and emotionally difficult situations within and/or outside of work. these virtual sessions provide an opportunity for project managers to learn and share new information with others in the field. we are always looking for feedback from our audience. if you have any suggestions for future webinars or if you would like to present at one of the sessions, please reach out to us via our listserv. toolkit: we’ve also continued to enhance our collective documentation and aides for all, including the dlf project managers toolkit. the toolkit is a collaborative project, based on contributions by members of the dlf project managers group over the years. the toolkit offers crowdsourced information, tips, techniques, and tools for project managers working in or with digital libraries. we include information on project management basics, documentation and templates shared by members, and evaluations of some of the popular project management read more the post dlf forum: building community with dlf’s project managers group appeared first on dlf. update on dlf’s senior program officer search dear dlf community, following our recent news about what is shaping up to be an exciting forum, we wanted to also update you on the dlf senior program officer position that was posted earlier this year. the search was notable for its strong, diverse pool of candidates and the many perspectives each could bring to dlf. the process was conducted with professional diligence and benefited significantly from the expertise and insight of the search committee, culminating in a lively final round of interviews. unexpectedly, we were not able to make a successful offer. while we are fortunate to still have a very strong candidate pool, we feel it is important to pause and reflect upon the recent search and wider context and conditions that may have influenced its outcome. we have had conversations with search firms lately that corroborate a significant rise in candidates declining job offers nationally across disciplines and professions. the reasons for this are being explored, but it is believed that many of the positions—like the one for dlf—were posted pre-covid and that the increased unpredictability of the academic workplace and the understandable avoidance of risk-taking could be significant factors. given the importance of the dlf leadership position, we plan to start a second search in early january. clir has begun an internal evaluation of the first round search process. this review is taking into account the changes in our academic environment stemming from the disruptions of, and responses to, the pandemic, including a profound rethinking of approaches to teaching, research, information sharing, and community building, as well as the implications for institutional budgets near and longer-term. as always, your support and engagement are deeply appreciated and essential to dlf’s continued success. charles j. henry president the post update on dlf&# ;s senior program officer search appeared first on dlf. dlf forum: building community with dlf’s assessment interest group though dlf is best known for our signature event, the annual dlf forum, our working groups collaborate year round. long before covid- introduced the concept of “zoom fatigue” into our lives, dlf’s working groups organized across institutional and geographical boundaries, building community while apart, to get work done. made possible through the support of our institutional members, working groups are the efforts of a committed community of practitioners, using dlf as a framework for action, engaged in problem-solving in a variety of digital library subfields from project management and assessment to labor and accessibility. once we decided that the dlf forum and affiliated events would be held in a virtual format, it meant that our working groups wouldn’t have the opportunity to meet in person for their typical working meals that take place throughout the forum; however, this year’s virtual format means that we’ll have more new dlf forum attendees than ever before. because dlf’s working groups are open to all, regardless of whether you’re affiliated with a dlf member institution or not, we asked leaders of the dlf working groups to introduce their groups and the work they do to the new and returning members of the #dlfvillage in a series of blogs and videos. we’ll share these working group updates in the days leading to this year’s dlf forum. all of our events are free of charge. register by november . updates from the dlf assessment interest group (aig) the dlf assessment interest group (aig) seeks to engage the community in developing best practices and guidelines for various kinds of digital library assessment. as the amount of digitized/born-digital content continues to grow at our research and cultural heritage institutions, there is an ever-increasing need to strategically standardize our assessment efforts. the only requirement for participation in a dlf aig working group is a willingness to dig in and devote a small part of your time contributing to the tools, methods, and body of knowledge on digital library assessment. we encourage you to join a meeting or reach out to the contacts listed below. (dlf membership is not a requirement for participation.) we look forward to extending an invitation for you to join us at our upcoming aig virtual event in early , which will showcase working group achievements and help us brainstorm for the year ahead. stay tuned for event details! in the meantime, here’s an overview of the five aig working groups and along with some recent highlights. the groups represented in this post are: the content reuse working group, the cost assessment working group, the cultural assessment working group, the metadata assessment working group, and the user experience assessment working group. aig content reuse working group the content reuse working group (#digreuse) focuses on developing standards, strategies, and workflows for assessing the use and reuse of digital objects across galleries, libraries, archives, museums, and repositories (glamr) content reuse working group in the content reuse working group is developing the digital content reuse assessment framework toolkit (d-craft). the d-craft project, generously funded by a national leadership grants for libraries (lg- - - - ) award from the institute of museum and library services, started in july . working group members are building the toolkit in two phases of work. the first phase includes work on ethical guidelines and recommended practices. ethical guidelines. the guidelines are intended for practitioners assessing use and reuse of digital cultural heritage artifacts, research outputs and scholarship, and data. these guidelines are meant both to inform practitioners in their decision-making, and to model for users what they can expect from those who steward digital collections. integral to the creation of this code are user privacy considerations, and a particular focus on concerns and ideas of black, indigenous, people of color, people with disabilities, working class, and poor communities. the working group is currently finalizing an ethical guidelines for assessing reuse draft for public review and comment. recommended practices. recommended practices will document tools, resources, and existing strategies for assessing various facets of digital object reuse. along the way, the working group will develop draft content with d-craft consultants, solicit feedback from their advisory group and glamr practitioners, and launch the toolkit. how can you participate? we encourage those interested in knowing more about the project to visit the d-craft project site as well as to review a dlf blog post introducing d-craft. those who have questions or comments should feel free to reach out to santi thompson. aig cost assessment working group we dig into the cost (both time and money) of digitization. cost assessment working group in building on the development of the digitization cost calculator, the aig cost group is shifting directions to create a more expansive toolkit that will serve a wider range of digitization labs. survey. we are seeking information from a variety of digitization stakeholders from across the dlf to inform two new initiatives. the data collected will be used to help create resources for calculating and assessing institutional commitments necessary for successful digitization projects or starting a digitization program. resource library. we are looking to assemble a resource library similar to the dlf aig metadata application profile clearinghouse project. the survey data will help us gather some preliminary information about the various organizations successfully using cost assessment tools in their local context. if possible, we will link to or provide samples which may provide a template for other organizations. cost worksheets. we seek more data on those interested in cost assessment practices that will give us better insight into how to further develop templates for “cost worksheets” which might be applied at the institutional level and modified for local use. unlike the cost calculator, these worksheets would be generated from local information to give end-users more contextualized information. digitization costs can be calculated in terms of either/both money and time, which themselves exist in tension with one another. for the purposes of this work, digitization is defined as the practices and procedures associated with reformatting physical read more the post dlf forum: building community with dlf’s assessment interest group appeared first on dlf. dlf forum: building community with dlf’s born-digital access working group though dlf is best known for our signature event, the annual dlf forum, our working groups collaborate year round. long before covid- introduced the concept of “zoom fatigue” into our lives, dlf’s working groups organized across institutional and geographical boundaries, building community while apart, to get work done. made possible through the support of our institutional members, working groups are the efforts of a committed community of practitioners, using dlf as a framework for action, engaged in problem-solving in a variety of digital library subfields from project management and assessment to labor and accessibility. once we decided that the dlf forum and affiliated events would be held in a virtual format, it meant that our working groups wouldn’t have the opportunity to meet in person for their typical working meals that take place throughout the forum; however, this year’s virtual format means that we’ll have more new dlf forum attendees than ever before. because dlf’s working groups are open to all, regardless of whether you’re affiliated with a dlf member institution or not, we asked leaders of the dlf working groups to introduce their groups and the work they do to the new and returning members of the #dlfvillage in a series of blogs and videos. we’ll share these working group updates in the days leading to this year’s dlf forum. all of our events are free of charge. register by november . join the born-digital access working group during their forum working session on november . who are we? the dlf born-digital access working group, or what is affectionately known as bdawg, is currently led by co-coordinators karla irwin and jessika drmacich. bdawg is the only born-digital group focused entirely on access. bdawg is very easy to join and the group is open to everyone! our google group is our main mode of communication, so please sign up there if you would like to receive updates and more information. bdawg meets bimonthly and there are many different opportunities for participation based on area of interest and availability. if you are curious about what members have published so far you can also check out dlf’s osf site for our bdawg. at around members, bdawg garners participation from a wide gamut of archivists, librarians, and cultural heritage workers coming from higher education institutions, museums, government archives, and even some students. what have we been working on? in august of , bdawg held our first ever virtual colloquium. this was an opportunity for bdawg members to share some work related to born-digital access at their respective institutions. we had a fantastic turnout and there are hopes to hold this event again next year. also this year, bdawg formed five sub-working groups focused around researching different born-digital access topics. these subgroups are typically formed once a year and are discussed, decided, and voted on by our members. updates on our sub-working groups! the access values subgroup completed a values statement for bdawg on the topic of how to approach work around born-digital access. you can read the access values here. special thanks to jessica farrell, brain dietz, and alison clemens for their great work on this! the documenting access methods subgroup is exploring what elements institutions could include in documentation for providing access. documenting access methods include two *sub-subgroups*: a framework group and a pointer group. the framework group is creating a framework or template for documentation and the pointer group will categorize and point to existing documentation. the documenting access methods subgroup plans to wrap up their work by the end of this year so be on the lookout for that! the donor relations subgroup has been developing a resource for managing remote donor relations and acquisitions in the form of an annotated bibliography that will be available very soon. once that is wrapped up, future work by a new subgroup will be looking at remote on-site visits and born-digital acquisitions. there will be some upcoming forums on this topic, so be on the lookout for information (or you can reach out to us)! the legal due diligence subgroup is creating a resource that helps practitioners understand what steps are needed to provide low-risk access with a close look at attorney client privilege, culturally sensitive materials, donor restrictions, ferpa and hipaa, gdpr, institutionally mandated confidentiality, pii, privacy, and terms of service. they plan on releasing a skeletal set of guidelines with a workflow once they wrap up their research. the ideal access systems subgroup is working to define what &# ;ideal&# ; access means in order to provide recommendations on how to create and implement ideal access systems in different environments. they are currently working on a literature review of relevant articles and case-studies and finalizing a functional definition of ideal in order to move into their next stage which involves building a project plan for an ideal access system. the subgroup hopes to release their documentation by the beginning of next year. get in touch! please reach out to karla and/or jessika with questions! jessika, williams college, (jgd @williams.edu) karla, university of nevada, las vegas (karla.irwin@unlv.edu) keep these links handy: bdawg google group: https://tinyurl.com/bdawggg dlf osf website: https://osf.io/hqmy / we hope to see you at a future bdawg meeting! the post dlf forum: building community with dlf’s born-digital access working group appeared first on dlf. announcing incoming ndsa coordinating committee members for - please join me in welcoming the two newly elected coordinating committee members elizabeth england and jessica neal, and one re-elected member, linda tadic. their terms begin january , and run through december , . elizabeth england is a digital preservation specialist at the u.s. national archives and records administration, where she participates in strategic and operational initiatives and services for the preservation of born-digital and digitized records. she previously was the digital archivist and a national digital stewardship resident at johns hopkins university. elizabeth currently serves on the ndsa communications and publications group and the digipres planning committee. jessica neal, was recently named the sterling a. brown archivist at williams college, having previously been the college archivist at hampshire college. additionally, jes is a workshop facilitator with docnow, and a member of ndsa’s digipres planning committee. linda tadic has served on the coordinating committee for the past two years. as an educator, she incorporates ndsa reports and projects into her courses in the ucla information studies department. additionally, linda brings her diverse experience working in non-profit and educational archives, managing digital asset management systems, and founding digital bedrock, a managed digital preservation service provider. we are also grateful to the very talented, qualified individuals who participated in this election. we are indebted to our outgoing coordinating committee members, karen cariani, bradley daigle (chair), sibyl schaefer, and paige walker, for their service and many contributions. to sustain a vibrant, robust community of practice, we rely on and deeply value the contributions of all members, including those who took part in voting. the post announcing incoming ndsa coordinating committee members for - appeared first on dlf. digital library federation - dlf clir fellowships & grants dlf publications clir global join give clir programs digital library federation dlf forum digital library of the middle east digitizing hidden special collections and archives recordings at risk mellon fellowships for dissertation research leading change institute dlf eresearch network postdoctoral fellowship program dlf digital library federation toggle navigation about about dlf our staff our members governance dlf code of conduct events dlf year-round forum: online past forums dlf forum news social events checklist child care fund resources dlf organizers’ toolkit blog and news #dlfcontribute series digitizing special formats dlf cost calculator dlf jobs board groups dlf working groups clir/dlf affiliates dlf membership cohorts get involved with groups opportunities grants and fellowships authenticity project dlf community calendar data curation postdocs community/capacity awards post a job/find a job contact get in touch stay connected join dlf our members benefits dlf digital library federation toggle navigation about about dlf our staff our members governance dlf code of conduct events dlf year-round forum: online past forums dlf forum news social events checklist child care fund resources dlf organizers’ toolkit blog and news #dlfcontribute series digitizing special formats dlf cost calculator dlf jobs board groups dlf working groups clir/dlf affiliates dlf membership cohorts get involved with groups opportunities grants and fellowships authenticity project dlf community calendar data curation postdocs community/capacity awards post a job/find a job contact get in touch stay connected join dlf our members benefits thanks for joining us! learn more and apply miss a great talk? we've got you covered. dlf forum plenary recordings forum child care fund you can make a difference. please give. more than just a conference. find your people all year long. join the dlf today meet the authenticity project participants! volunteers and fellows have been matched in yearlong partnerships. learn more what’s the dlf? networked member institutions and a robust community of practice—advancing research, learning, social justice, & the public good through the creative design and wise application of digital library technologies dlf as community find your people, year-round. grassroots, pragmatic, and mission-driven, dlf is a space where ideas are road-tested and shared strategies and visions emerge. dlf as platform get things done. we foster active, open, and welcoming working groups dedicated to building better libraries, museums, and archives for the digital age. dlf as crossroads meet up. our annual dlf forum serves as meeting place, marketplace, and congress for diglib practitioners from member institutions and the community at large. read more selected posts what’s new on the dlf blog cfps are here! dlf forum and affiliated events dlf forum community journalist reflection: lisa covington three questions on irus-usa dlf forum and fall events move online metadata during covid february , gayle assessment, community this post was written by members of the metadata working group, a subgroup of dlf’s assessment interest group. digital collections work has changed in a number of ways during the covid- pandemic. for many libraries and archives, this has meant... more three new ndsa members january , nathan tallman ndsa since january , the ndsa coordinating committee unanimously voted to welcome three new members. each of these members bring a host of skills and experience to our group. please help us to welcome: arkivum: arkivum is recognized internationally for... more virtual ndsa digital preservation recordings available online! january , kussmann ndsa session recordings from the virtual ndsa digital preservation conference are now available on ndsa’s youtube channel, as well as on aviary. the full program from digital preservation : get active with digital preservation, which took place... more more posts jobs working groups ndsa organizers' toolkit community calendar dlf contribute dlf events grants & fellowships digitization cost calculator contact clir+dlf north union street suite -pmb alexandria, va e: info@diglib.org elsewhere twitter facebook linkedin youtube email rss community calendar geo lib camp february , – february , online https://www.eventbrite.com/e/geo libcamp- -tickets- nycdh week february , – february , https://nycdh.org/dhweek/ college art association annual conference february , – february , online https://www.collegeart.org/programs/conference/conference more events & links here. on twitter rt @clirnews: celebrating #archiveshashtagparty #archivesblackeducation and #blackhistorymonth with "our story: photographs and publication…yesterday have a spare minute? make sure to complete our question survey about attending fall events today! https://t.co/h mxokpxqz days ago rt @lellyjz: really looking forward to this! https://t.co/ o gkknzqq days ago unless otherwise indicated, content on this site is available for re-use under cc by-sa . license clir skip to content open toolbar accessibility tools increase text decrease text grayscale high contrast negative contrast light background links underline readable font reset samvera - a vibrant and welcoming community developing repository software tools skip to content about samvera samvera is community sourced software for repository solutions philosophy governance annual report (pdf download) all annual reports interest and working groups faq samvera privacy policy licensing what is samvera? samvera is an open source repository framework samvera community overview (pdf download) technology white paper (pdf download) applications & demos technology stack why use samvera? samvera is flexible and extensible the samvera community community support sustainability who uses samvera? samvera partners partner prospectus (pdf download) partner contribution model (pdf download) samvera adopters community framework samvera user profiles case study: emory university case study: avalon at the university of houston getting started general documentation installation from github communication service providers news & events news & events samvera events diary upcoming meetings samvera twitter page search samvera.org close search for: a vibrant and welcoming community samvera vision statement “samvera™ is a vibrant and welcoming community of information and technology professionals who share challenges, build expertise, and create sustainable, best-in-class solutions, making the world’s digital collections accessible now and into the future.” samvera’s suite of repository software tools offers flexible and rich user interfaces tailored to distinct content types on top of a robust back end – giving adopters the best of both worlds. '; benefits of samvera we believe that no single system can provide the full range of repository-based solutions for a given institution’s needs and that no single institution can resource the development of a full range of solutions on its own. working together, the samvera community creates sustainable solutions using a common infrastructure within which there is the flexibility to tailor solutions to local demands and workflows. samvera software is free and open source, available under an apache license. how it works samvera maintains a set of ruby on rails components (ruby gems) that, together, can be used to build flexible and extensible digital repository solutions. hyrax combines a number of these components into a toolkit (a rails engine) for building repository applications to meet a wide range of repository requirements, whilst hyku is an out-of-the-box repository application with multi-tenant capability built on hyrax. samvera does not work in isolation and relies on a number of external open source components, including: fedora – a durable repository layer for persisting and managing digital objects. apache solr – a fast and performant search platform blacklight – a discovery platform built on solr applications institutional repositories samvera is being used as a base for a number of institutional repositories (irs) each of which contains a range of content types. many of the samvera partners have developed ir with hyrax. for instance, george washington university, unc chapel hill, the university of hull, uk cultural heritage organisations, and many others have an ir containing electronic dissertations and theses (etds), past examination papers, learning materials, journal articles, small datasets and more. for more information visit samvera wiki implementations information page. media collections avalon, an access platform for online audio and video was developed by indiana university and northwestern university using the samvera stack. amongst others, wgbh, a public broadcaster in boston, the university of virginia, the university of houston and washington university are utilizing avalon and samvera to manage their digital media content. solution bundles the avalon media system is a collaborative samvera-based project for managing and providing online access to digital video and audio. it is now available as a samvera “solution bundle”. hyku is the result of a collaboration to extend the existing samvera project codebase to build, bundle, and promote a feature-rich, robust, flexible digital repository that is easy to install, configure, and maintain. hyku is a solution bundle that can be installed locally or run in the cloud. it is based on hyrax, a community-developed ruby gem that allows users to design and build their own, customized installation of our software. data and preservation the samvera software is being used as the basis for data repositories, for instance “deep blue data” at the university of michigan and “imago” at indiana university. the digital repository of ireland is “a national repository for ireland’s humanities, social sciences and cultural heritage data.” a number of samvera partners are investigating the use of our software for dealing with the long-term preservation of research data. in the uk, the universities of york and hull have been integrating the open-source preservation system archivematica into their samvera workflows. archives and special collections samvera is being used in conjunction with archives and special collections. the university of york in the uk has used it as the basis for their archbishops’ registers site, providing access to more than , pages of early manuscripts. princeton university has used samvera to create “figgy”, a workflow tool for digitizing a wide range of formats including archival materials, ephemera, maps, audio, and coins. publishing fulcrum is a community-based, open source publishing platform based on samvera that helps publishers present their authors’ research outputs in a durable, discoverable, accessible and flexible form. it is hosted on the university of michigan library infrastructure, specifically designed to curate digital objects. interoperable with other publishing tools and integrated into the information supply chain, fulcrum ensures that content is discovered by readers and impact is tracked. fulcrum aims to implement accessible systems and features and effect change by sharing and maintaining a high standard of accessibility. previous slide next slide the samvera community samvera is not (and has never been) grant funded. it is distributed, robust and open. the samvera community was conceived and executed, under its original name “hydra”, as a collaborative, open source effort from its very beginning in . samvera has grown into a vibrant, highly active community including more than partners who formally support our work and development. samvera is designed so that adopters can each have their own mix of features; variation is part of the plan. for adopters who do not have the resourcing to create their own variant, the samvera community has developed rather more “off-the-shelf” application bundles. getting started samvera news news & events season’s greetings from the samvera community save the date for samvera virtual connect developer resources: bug hunting in hyrax; adding blacklight advanced search to hyku fedora alpha release available for download and testing samvera tech : a beginner-friendly overview of samvera view all news if you want to go far, go together. contact us samvera partners boston public library columbia university cornell university cosector, university of london data curation experts digital repository of ireland duke university emory university indiana university lafayette college northwestern university notch oregon state university penn state university princeton university library stanford university tufts university ubiquity press university of california, santa barbara university of california, san diego university of cincinnati university of houston university of hull university of michigan university of notre dame university of oregon university of utah university of virginia university of york washington university in st louis wgbh boston yale university this work is licensed under a creative commons attribution . international license. © samvera. samvera twitter samvera github samvera wiki tfnsw open data hub and developer portal skip to main content log inregister historical car park data for selected park&ride and metro station car parks is now available. get it here. search log inregister toggle navigation browse data developers get started documentation developer information marketing status resources innovation innovation challenges open data day maas data specification transport for nsw endorsed apps forum blog open data available today we supply real-time data to apps with over million unique customer downloads in total. the open data program will make these datasets, along with other transport data, more broadly available. this data will be capable of supporting apps and a whole lot more. learn more get started! we have a range of resources to help you get started using our data. read through our user guide, api basics or documentation for information or visit the troubleshooting page if you are stuck. learn more transport for nsw innovation transport for nsw is committed to fostering innovation by providing open access to our data. future transport holds a number of events and challenges to uncover new ways of using our data. learn more ‹ › data catalogue browse our extensive data catalogue and access resources more general info learn more about the open data hub and developer portal more developers find everything you need to get started and access our data more useful links a list of useful links that will assist you in developing great apps more product showcase anytrip anytrip overview anytrip lets you track public transport vehicles around you in real-time using a live map. it will also show you upcoming departures from your favourite stops and stations. check which service you are currently travelling on check real-time departure information get an at-a-glance view of all public transport across nsw access anytrip web app through a web browser. datasets trip planner api public transport - timetables - for realtime public transport - location facilities and operators traffic visit website google play store url apple store url boatable boatable overview log your trip with boatable and enjoy the waterways safely with tips before you head off, and alerts while out and about. boatable is a boating assistance app that helps recreational boaters navigate the seas. it includes features such as a range of maps, boat detail records, weather, logging of trips and safety information for both on and off the water. boatable is endorsed by transport for nsw following the boating companion innovation challenge in . datasets aidtonavigation: channel markers and other selected aids to navigation boatramp: boat ramps with attributes as displayed on the boat-ramp locator on the rms website coastalbar: indicative locations of coastal bars as described in the marine safety regulation notowing: areas where towing of persons is prohibited as designated by signs along the waterway nowash: areas where the generation of wash is prohibited as designated by signs along the waterway publicmooring: courtesy and emergency moorings restrictedzone: areas of restricted waters for port security or naval purposes speed: areas of speed limits (in knots) as designated by signs along the waterway shallowwater: areas of assumed shallow water (depth of less than approximately metres in tidal waters and the shallower water in inland waters at full supply) speed: areas of speed restrictions (in knots) as designated by signs along the waterway webcamera: locations of bar cameras, including urls to webpages on the rms internet site for viewing of the live feed visit website google play store url apple store url bustle bustle overview bustle is an australian first implementation of digital advertising screens together with real-time public transport information. combining advertising and public transport information creates a unique service to naturally capture attention of customers and provide a community service. screens are customised with local train / metro / light-rail and bus real-time information. in a world where brands are fighting to be noticed and people's attention is short - bustle creates a unique experience for your customers to remember you. datasets timetables complete gtfs public transport - timetables - for realtime visit website citymapper citymapper overview citymapper is a multi-modal trip planning app, making cities easier to use. the app allows for checking of nearby departures in real-time and has the ability to find the fastest route combining bus, train, ferry, light rail, taxi, car share, bike share and walking. commuters can decide which transport option based on time, the amount of calories burned, and also receive alerts for route disruptions and more. citymapper is endorsed by transport for nsw following the travel choices innovation challenge in . datasets public transport scheduled and real-time information visit website google play store url apple store url deckee deckee overview deckee is a social app that helps boaters explore the waterways safely, share fishing and anchorage reports, find local marine services and much more. deckee is a boating assistance app that helps recreational boaters explore the waterways through a community-based contributions. it also has safety alerts, weather conditions, helps with boat insurance finding all in one app. deckee is endorsed by transport for nsw following the boating companion innovation challenge in . datasets aidtonavigation: channel markers and other selected aids to navigation boatramp: boat ramps with attributes as displayed on the boat-ramp locator on the rms website coastalbar: indicative locations of coastal bars as described in the marine safety regulation notowing: areas where towing of persons is prohibited as designated by signs along the waterway nowash: areas where the generation of wash is prohibited as designated by signs along the waterway publicmooring: courtesy and emergency moorings restrictedzone: areas of restricted waters for port security or naval purposes speed: areas of speed limits (in knots) as designated by signs along the waterway shallowwater: areas of assumed shallow water (depth of less than approximately metres in tidal waters and the shallower water in inland waters at full supply) speed: areas of speed restrictions (in knots) as designated by signs along the waterway webcamera: locations of bar cameras, including urls to webpages on the rms internet site for viewing of the live feed visit website google play store url apple store url easydrop easydrop overview easydrop is an interactive app for commercial delivery drivers. the app helps drivers manage their daily delivery schedule, find the fastest route and provides up to date traffic conditions. drivers can also stay in touch with their customers through the app; with one click delivering becomes delivered! developed by ozpoint. view the developer discussing their relationship with tfnsw here datasets roads real-time information visit website apple store url embark embark overview embark not only lets you plan trips around greater sydney, but works in hundreds of cities around australia and the world - now there's no need to download a new transport app when you're travelling. embark shows you live arrival times for buses, trains, light rail, ferries, uber, and more. additionally, embark also gives you insight into calorie burn for walking routes, carbon emissions comparisons for public transport routes, and wheelchair accessibility information for each service. accessible travel provides wheelchair accessibility information for supported services supports voiceover for iphone visit website google play store url apple store url finderful.com finderful.com overview finderful.com is a website designed to help you decide where to buy or rent in sydney, new south wales, and australia. you can search for homes or suburbs based on a wide range of customisable search preferences you set, including number of bedrooms and bathrooms, safer suburbs, or cheaper homes. finderful.com uses transport for nsw public transport datasets in its searches to calculate travel times from every home to your workplace or any other point of interest. it even lets you select your preferred transport mode! finderful.com is endorsed by transport for nsw following the travel choices innovation challenge in . finderful.com can help you find homes and suburbs that best suit you. visit website google maps google maps overview navigate your world faster and easier with google maps. over countries and territories mapped and hundreds of millions of businesses and places on the map. get real-time gps navigation, traffic, and transit info, and explore local neighbourhoods by knowing where to eat, drink and go - no matter what part of the world you’re in. accessible travel provides accessibility information for travelling with a wheelchair. datasets timetables complete gtfs public transport - timetables - for realtime public transport - realtime alerts public transport - realtime vehicle positions public transport - realtime trip update public transport - location facilities and operators visit website google play store url apple store url l p l p overview l p app is a digital logbook designed for the latest generation of learner drivers. its intuitive user interface features a countdown to motivate learners, real time tracking and recording of driving sessions plus an online educational platform with video clips for demonstrations. visit website google play store url apple store url licence ready licence ready overview with digital driving instruction, personalised training and nsw log book, this is a fantastic app for learner drivers. download across devices and utilise with multiple supervisors for flexible learning. visit website google play store url apple store url live traffic nsw live traffic nsw overview live traffic nsw provides to-the-minute updates about incidents and conditions that may affect a user’s journey. users can save their favourite routes to receive scheduled alerts and can check images from live traffic cameras that update every seconds. the app also enables driving mode so users can receive audio alerts of nearby incidents while on their trip. live traffic nsw can be used in both sydney and regional nsw. datasets live traffic information visit website google play store url apple store url metarove metarove overview metarove was developed to make public transport trip planning easier for customers with limited mobility. the app provides real-time information about departure times, route recalculation, trip plan updates and options if services are delayed or cancelled. the app is highly customisable and users can set personal walking speeds, maximum physical travel distances or display accessible journeys only. developed by metarove. view the developer discussing their relationship with tfnsw here or see how metarove works for customers with limited mobility here datasets public transport scheduled and real-time information visit website google play store url moovit moovit overview moovit provides users with real-time information and fastest routes for the public transport network. users can send live reports about their travel experience, such as cleanliness and seat availability, through the app. this live information helps improve route plans and provides other users with more accurate travel time estimates. developed by moovit. datasets public transport scheduled and real-time information visit website microsoft store url google play store url apple store url my house geek my house geek overview my house geek allows users to get to know a neighbourhood before they move in. a map allows users to search for properties, as well as discover nearby schools, public transport services and other places of interest such as shopping centres and child care facilities. using transport data, the map shows users how far places of interest are from their selected property. users can even save and share all searches! visit website next station next station overview next station provides trip planning and timetables, real-time vehicle position and service alerts in english, simplified and traditional chinese covering sydney’s public transport network, nsw trainlink and nsw regional buses. 【next station 下一站】提供大服务，包括 . 行程规划， . 实时时间表， . 实时公交位置， . 实时中文服务信息， . 路线地图， .车厢空间拥挤度。【next station 下一站】涵盖火车，城轨，公交，轻轨及渡船。【next station 下一站】提供全中文界面及实时翻译所有公共交通服务信息。【next station 下一站】提供大服務，包括 . 行程規劃， . 實時時間表， . 實時公交位置， . 實時中文服務信息， . 路線地圖， .車廂空間擁擠度。【next station 下一站】涵蓋火車，城軌，巴士，輕軌及渡船。【next station 下一站】提供全中文界面及實時翻譯所有公共交通服務信息。 datasets public transport schedules and real-time information visit website google play store url apple store url nextthere nextthere overview nextthere provides users with real-time service information. the app tracks a user’s location and shows when the next trains, buses, ferries or light rail are due to depart from that location. an easy-to- read map tracks services and users can receive alerts about disruptions before their journey. developed by appjourney pty ltd. view the developer discussing their relationship with tfnsw here datasets public transport scheduled and real-time information visit website apple store url opal travel opal travel overview opal travel allows users to plan their trip and check their opal fare estimate for train, bus, ferry and light rail services. adult, child/youth and senior/pensioner fares are displayed as well as opal card retailers. nfc android users can also scan their opal card to check their balance, journey count and last tap details. details of a trip plan can be displayed as a map or as text-based instructions, and user can receive service alerts. users can also save regular trips and locations as favourites. trip planning is enabled for both sydney and regional nsw. datasets public transport scheduled and real-time information visit website google play store url apple store url rome rio rome rio overview rome rio is a global travel planning website and app that allows users to quickly find their way from a to b using any combination of transport. simply type in two addresses anywhere in the world and you’ll be offered a choice of journey options that seamlessly stitch together flights, transfers, trains, ferries, and buses, to get you from door to door – all with prices, estimated durations, and booking details at lightning-fast speed. rome rio covers million locations around the world and includes routing from more than , transportation operators. datasets timetables complete gtfs visit website google play store url apple store url roundtrip roundtrip overview roundtrip is an app for nsw learner drivers that makes it super easy to record supervised driving practice. tap the record button, enter your odometer and you’re off! roundtrip will track your time, start and end location, weather and more, so you focus on learning to drive. you can also use roundtrip to view and unlock learning goals, see your total practice times and submit your digital logbook to the rms. no more paper logbook needed! visit website apple store url snarl snarl overview snarl provides up-to-date information about accidents and congestion across the nsw, qld and vic road networks. users can check traffic conditions before their journey, or on the go using driving mode. in driving mode, auto driving detection will warn users of current conditions, accidents and incidents while they are on the move. developed by snarl. view the developer discussing access to open data here datasets roads real-time information visit website google play store url apple store url stop announcer stop announcer overview stop announcer is a route guidance app that provides audio notifications of stops made along a trip. the app will announce bus stops, train stations, ferry wharves and light rail stops. it will also alert users when their selected stop has been reached. view the developer discussing access to open data here datasets public transport scheduled and real-time information visit website google play store url apple store url transit transit overview transit app provides real-time trip planning including departure times, timetables and route maps all in big text and bright colours. users can set reminders, get notifications about disruptions and view visual stop notifications on a map. the app also allows users to request an uber. developed by transitapp, inc. datasets public transport scheduled and real-time information visit website google play store url apple store url tripchecker tripchecker overview tripchecker is the world's free, take-anywhere transit companion. users can get live vehicle departures and key travel info in sydney, london and new york - and other major cities around the world, including every town in great britain. next train or bus arrival is instantly shown to users, as well as live journey planning, instant disruptions info and traffic reports. tripchecker will even wake users when they arrive at their stop! datasets public transport scheduled and real-time information google play store url apple store url tripgo tripgo overview tripgo allows users to compare public and private transport options. it provides information including estimated costs , fastest modes and routes based on real-time data. users can manually search for trip options, or tripgo can use their calendar to provide options automatically. the app can also provide users with door-to-door directions. developed by skedgo. view the developer discuss their relationship with tfnsw here datasets public transport scheduled and roads real-time information visit website google play store url apple store url triptastic triptastic overview triptastic shows users the next available service from their current location, service disruption information and vehicle tracking. from within the app users can search for suburbs, routes, stops and businesses or explore a range of interactive, detailed maps of routes, stops and service frequencies. developed by appjourney pty ltd. datasets public transport scheduled and real-time information visit website apple store url tripview tripview overview tripview displays train, bus, ferry and light rail timetables for sydney. users can view a summary of next services or a full timetable. alarms can also be set for upcoming trips. developed by grofsoft. datasets public transport scheduled and real-time information visit website microsoft store url google play store url apple store url waverley transport waverley transport overview our hyperlocal maas solution provides multimodal journey options for customers to have a seamless travel experience. it is customised for the waverley council area. the main objectives are easing traffic congestion and reducing parking demand. developed by smart cities transport (sct) datasets public transport - real-time trip update public transport - timetables - for realtime other datasets including waverley council datasets visit website google play store url apple store url media browse media releases, promotional videos, information for third party apps and more. more performance and analytics transport performance and analytics (tpa) operates as a centre of excellence, providing objective and credible transport data, advice and analysis. tpa combines the bureau of transport statistics and bureau of freight statistics and provides the evidence base that helps drive strategic decision making in support of an effective transport system. more since the launch of the open data hub... , registered users , registered applications , , , api hits open data what is open data? general information contact us general enquiry @datatfnsw data request media developers developer information data catalogue get started documentation api explorer marketing widget support forum support faqs troubleshooting useful links legal terms privacy policy acceptable use policy data licence follow us back to top transport for nsw acknowledges the traditional owners and custodians of the land, and respects elders past, present and future. metadata during covid - dlf clir fellowships & grants dlf publications clir global join give clir programs digital library federation dlf forum digital library of the middle east digitizing hidden special collections and archives recordings at risk mellon fellowships for dissertation research leading change institute dlf eresearch network postdoctoral fellowship program dlf digital library federation toggle navigation about about dlf our staff our members governance dlf code of conduct events dlf year-round forum: online past forums dlf forum news social events checklist child care fund resources dlf organizers’ toolkit blog and news #dlfcontribute series digitizing special formats dlf cost calculator dlf jobs board groups dlf working groups clir/dlf affiliates dlf membership cohorts get involved with groups opportunities grants and fellowships authenticity project dlf community calendar data curation postdocs community/capacity awards post a job/find a job contact get in touch stay connected join dlf our members benefits dlf digital library federation toggle navigation about about dlf our staff our members governance dlf code of conduct events dlf year-round forum: online past forums dlf forum news social events checklist child care fund resources dlf organizers’ toolkit blog and news #dlfcontribute series digitizing special formats dlf cost calculator dlf jobs board groups dlf working groups clir/dlf affiliates dlf membership cohorts get involved with groups opportunities grants and fellowships authenticity project dlf community calendar data curation postdocs community/capacity awards post a job/find a job contact get in touch stay connected join dlf our members benefits metadata during covid february , gayle assessment, community this post was written by members of the metadata working group, a subgroup of dlf’s assessment interest group. digital collections work has changed in a number of ways during the covid- pandemic. for many libraries and archives, this has meant working remotely and shifting toward tasks that can be done online. within the dlf aig metadata working group, members have discussed a number of ways that organizations have chosen to increase capacity for metadata, transcription, and other tasks related to digital collections as a way of providing work for employees who would normally work in public-serving positions. this post documents some of those projects and activities. university of north texas at the university of north texas, our digital collections use a web-based metadata editing interface and we can add as many users as needed. when the stay-at-home order went into effect right after spring break, many of our library staff members (including full-time librarians/staff and part-time student workers) were no longer able to do their regular jobs and we offered metadata as an alternative. we added about new editors to our system in march . additionally, we added some quickly-drafted documentation to steer people toward easy metadata projects and known issues that require clean-up (like fixing name formatting). to keep oversight manageable, new editors were still attached to their own departments (or assigned to one that needed help), with a central contact person for each department and a specific sub-set of projects. our team of developers rushed an overhaul of the event tracking system that documents who is editing and what records they are changing so that managers could more easily verify if workers were editing when they said they were working. tracking edits has also let us measure how significantly overall editing has increased. multiple times since this started, we have had at least one editor working during every hour of the day. having so many relatively-untrained editors has resulted in a large number of issues that will need to be reviewed, but we have tools built into our system to help identify those issues and have added them to our ongoing list of things to fix. overall, this was still an extremely positive experience since the increase in editors allowed significant progress or completion of work that would not have been done otherwise. – hannah tarver university of utah marriott library at the university of utah, the covid- pandemic pivot to remote work prompted the launch of transcription projects, both with handwritten materials from special collections and newspaper ocr correction. this includes the transcription of , employee records by our digital operations student employees which resulted in the complete transcription of the kennecott miner records collection. we are also using omeka classic with the scripto plug-in as the platform for manuscript transcription projects and are able to find valuable work for people to engage in when they couldn’t physically be at the library. in addition, we developed a born-digital crowdsourced digital collection, the utah covid- digital collection designed to capture what is currently happening during this unusual time. we’ve gotten a great response from the university and larger utah communities, with over contributions so far available in the digital library. the covid- digital collection has enabled us to build new partnerships and provided the library with outreach opportunities. an article detailing the project is forthcoming in a special issue of the journal digital library perspectives. – anna neatrour utah state archives after starting with from the page a few months earlier, moving staff and volunteers to transcription and indexing projects proved to be successful. contributors finished a historical court case (and now working on a second one) and a year’s worth of birth certificates in only a few months using the web-based interface that integrates with contentdm digital collections. with a built-in notes feature, questions can be asked and answered directly on a document’s page, which will then be exported along with the rest of the metadata. we are now preparing to open up the birth certificate indexing to the general public with additional training materials. in addition, new digital collections have been published, even with metadata developed remotely, using tools like google sheets for input and then converting to delimited text files for import. – gina strack university of texas at austin at the start of march, the university of texas libraries collections portal, the public-facing search and discovery interface for our digital asset management system (dams), included approximately , items. shortly after, the ut-austin campus closed and many staff members from the libraries’ first-line customer service, acquisitions and cataloging units found their roles pivoting to create metadata remotely for our dams system. collection curators within ut libraries created large-scale digital projects to help ensure continued remote work and to utilize this unusual time to turn their focus to projects that had been placed on the back burner due to more pressing obligations. our digital asset management system coordinator and staff from our preservation and digital stewardship unit created flexible pathways to support these projects and to ensure successful ingests into the dams. staff at the architecture & planning library and the alexander architectural archives, the nettie lee benson latin american collection, and the perry-castañeda library map collection dedicated themselves to ingesting and describing large amounts of digital items, increasing our total number of items available online to over , by september. digital objects newly available online as a result of this unprecedented, organization-wide collaborative effort include over , digitized architectural drawings and images, historic books from the benson rare book collection and primeros libros de las américas, and , scanned maps. the university of texas libraries documented the experience and provided a more detailed explanation of our dams in texlibris. – mandy ryan colgate university colgate university’s special collections and university archives (scua) is documenting the colgate community’s experiences and stories of covid- . digital contributions can be submitted at any time via a google form and may be added to colgate’s digital collections portal. there have been direct submissions as of october . physical donations of covid- related materials will be accepted once staff return to the library building. colgate’s metadata and cataloging (m&c) staff have been working with scua’s digital collections at home for the first time, describing the work of the university’s longest-serving official photographer, edward h. stone. stone documented life at colgate from the s to the s, and also photographed the people, places, businesses, and industry of the village of hamilton and madison county, new york. m&c staff are creating and editing metadata for more than glass plate negatives scanned by scua staff and students. we anticipate this will be a successful collaboration between the two departments that will serve as a model for other metadata-based remote work projects on campus. m&c staff have also worked with a born-digital lgbtq oral history project curated by students in the explorations in lgbtq studies class. oral history interviews with colgate graduates active in the struggle for lgbtq rights on campus from the s to the s is now available on the digital collections site – rachel white digital library of georgia most of our staff were able to continue doing most of our work from home, though some imaging projects shifted from actively imaging work (which would have had to be done in the office with our cameras) to working on image editing and curation work. we also had to postpone a meeting for our digitization partners. some metadata projects that were waiting on new imaging work were shifted to complete later; metadata staff worked on metadata remediation and metadata harvesting projects. one colleague who works on newspaper imaging was shifted over to a project describing moving image footage for the parade of quartets collection. we set up a student transcription project to keep students teleworking while they had to remain off-campus due to covid- . their transcription work was incorporated into our full-text accessibility feature for some smaller collections. students are now working in the office and from home on newspaper collation and metadata work, and our imaging staff have worked out a schedule to work while social distancing. our full staff meetings shifted from weekly meetings (in person) to daily meetings (via zoom). unit and supervisor meetings continue with the same frequency as they were held pre-covid. our quarter - newsletter and our quarter newsletter both provide more details of what we have worked on throughout the year. – mandy mastrovita university of florida since the middle of march , the digital support services (dss) at the libraries has shifted the focus of its imaging assistant crew. collaborating with the metadata staff, this crew has carried out site-wide metadata cleanup projects for the university of florida digital collection (ufdc) using ufdc’s online metadata edit form. these tasks can all be done at home using a computer connected to the internet with minimum instructions. the projects include adding missing system id numbers, unifying the spelling of language terms, correcting diacritic displays, updating rights statements, transcribing hand-written content, merging genre terms of different spelling variations to selected ones. so far, dss staff has modified over , rights statements and transcribed over , words. these projects improve the overall metadata quality dramatically. for instance, the genre terms in use will then be cut down to about from the original terms gathered from all data contributors over the years. to maintain this smaller selection of genre terms, the dss will also implement steps to assure all incoming content uses terms from the controlled genre list. – xiaoli ma the ohio state university libraries the onset of the covid- pandemic necessitated a shift to telework for university libraries’ employees. in collaboration with metadata initiatives and preservation & digitization, staff and student employees in other units and needing remote work to do were given the opportunity to do metadata telework. these entailed review and description of content for digital collections, a digital repository for digitized and born-digital special collections and archival materials. catalogers worked on remediation of legacy metadata records, particularly audio and image resources. staff and student employees with no prior metadata experience assisted with review and description of digitized audio and video content in the backlog. this group also contributed to metadata gathering and quality review for a large migration of digitized student newspapers. virtual collaboration was conducted with zoom, e-mail, and the university’s instance of box, a cloud-based content management system. this work has made a significant impact on the backlog for dc. however, metadata initiatives and applicable stakeholders are still reviewing the work that was done before making updates to records and ingesting the newly processed content. – annamarie klose related read more &media="> read more "> previous what's the dlf? networked member institutions and a robust community of practice—advancing research, learning, social justice, & the public good through the creative design and wise application of digital library technologies contact clir+dlf north union street suite -pmb alexandria, va e: info@diglib.org elsewhere twitter facebook linkedin youtube email rss community calendar geo lib camp february , – february , online https://www.eventbrite.com/e/geo libcamp- -tickets- nycdh week february , – february , https://nycdh.org/dhweek/ college art association annual conference february , – february , online https://www.collegeart.org/programs/conference/conference more events & links here. on twitter rt @clirnews: celebrating #archiveshashtagparty #archivesblackeducation and #blackhistorymonth with "our story: photographs and publication…yesterday have a spare minute? make sure to complete our question survey about attending fall events today! https://t.co/h mxokpxqz days ago rt @lellyjz: really looking forward to this! https://t.co/ o gkknzqq days ago unless otherwise indicated, content on this site is available for re-use under cc by-sa . license clir skip to content open toolbar accessibility tools increase text decrease text grayscale high contrast negative contrast light background links underline readable font reset google online security blog: know, prevent, fix: a framework for shifting the discussion around vulnerabilities in open source security blog the latest news and insights from google on security and safety on the internet know, prevent, fix: a framework for shifting the discussion around vulnerabilities in open source february , posted by eric brewer, rob pike, abhishek arya, anne bertucio and kim lewandowski executive summary: the security of open source software has rightfully garnered the industry’s attention, but solutions require consensus about the challenges and cooperation in the execution. the problem is complex and there are many facets to cover: supply chain, dependency management, identity, and build pipelines. solutions come faster when the problem is well-framed; we propose a framework (“know, prevent, fix”) for how the industry can think about vulnerabilities in open source and concrete areas to address first, including: consensus on metadata and identity standards: we need consensus on fundamentals to tackle these complex problems as an industry. agreements on metadata details and identities will enable automation, reduce the effort required to update software, and minimize the impact of vulnerabilities. increased transparency and review for critical software: for software that is critical to security, we need to agree on development processes that ensure sufficient review, avoid unilateral changes, and transparently lead to well-defined, verifiable official versions. the following framework and goals are proposed with the intention of sparking industry-wide discussion and progress on the security of open source software. due to recent events, the software world gained a deeper understanding about the real risk of supply-chain attacks. open source software should be less risky on the security front, as all of the code and dependencies are in the open and available for inspection and verification. and while that is generally true, it assumes people are actually looking. with so many dependencies, it is impractical to monitor them all, and many open source packages are not well maintained. it is common for a program to depend, directly or indirectly, on thousands of packages and libraries. for example, kubernetes now depends on about , packages. open source likely makes more use of dependencies than closed source, and from a wider range of suppliers; the number of distinct entities that need to be trusted can be very high. this makes it extremely difficult to understand how open source is used in products and what vulnerabilities might be relevant. there is also no assurance that what is built matches the source code. taking a step back, although supply-chain attacks are a risk, the vast majority of vulnerabilities are mundane and unintentional—honest errors made by well-intentioned developers. furthermore, bad actors are more likely to exploit known vulnerabilities than to find their own: it’s just easier. as such, we must focus on making fundamental changes to address the majority of vulnerabilities, as doing so will move the entire industry far along in addressing the complex cases as well, including supply-chain attacks. few organizations can verify all of the packages they use, let alone all of the updates to those packages. in the current landscape, tracking these packages takes a non-trivial amount of infrastructure, and significant manual effort. at google, we have those resources and go to extraordinary lengths to manage the open source packages we use—including keeping a private repo of all open source packages we use internally—and it is still challenging to track all of the updates. the sheer flow of updates is daunting. a core part of any solution will be more automation, and this will be a key theme for our open source security work in and beyond. because this is a complex problem that needs industry cooperation, our purpose here is to focus the conversation around concrete goals. google co-founded the openssf to be a focal point for this collaboration, but to make progress, we need participation across the industry, and agreement on what the problems are and how we might address them. to get the discussion started, we present one way to frame this problem, and a set of concrete goals that we hope will accelerate industry-wide solutions. we suggest framing the challenge as three largely independent problem areas, each with concrete objectives: know about the vulnerabilities in your software prevent the addition of new vulnerabilities, and fix or remove vulnerabilities. a related but separate problem, which is critical to securing the supply chain, is improving the security of the development process. we’ve outlined the challenges of this problem and proposed goals in the fourth section, prevention for critical software. know your vulnerabilities knowing your vulnerabilities is harder than expected for many reasons. although there are mechanisms for reporting vulnerabilities, it is hard to know if they actually affect the specific versions of software you are using. goal: precise vulnerability data first, it is crucial to capture precise vulnerability metadata from all available data sources. for example, knowing which version introduced a vulnerability helps determine if one's software is affected, and knowing when it was fixed results in accurate and timely patching (and a reduced window for potential exploitation). ideally, this triaging workflow should be automated. second, most vulnerabilities are in your dependencies, rather than the code you write or control directly. thus, even when your code is not changing, there can be a constant churn in your vulnerabilities: some get fixed and others get added. goal: standard schema for vulnerability databases infrastructure and industry standards are needed to track and maintain open source vulnerabilities, understand their consequences, and manage their mitigations. a standard vulnerability schema would allow common tools to work across multiple vulnerability databases and simplify the task of tracking, especially when vulnerabilities touch multiple languages or subsystems. goal: accurate tracking of dependencies better tooling is needed to understand quickly what software is affected by a newly discovered vulnerability, a problem made harder by the scale and dynamic nature of large dependency trees. current practices also often make it difficult to predict exactly what versions are used without actually doing an installation, as the software for version resolution is only available through the installer. prevent new vulnerabilities it would be ideal to prevent vulnerabilities from ever being created, and although testing and analysis tools can help, prevention will always be a hard problem. here we focus on two specific aspects: understanding risks when deciding on a new dependency improving development processes for critical software goal: understand the risks for new dependencies the first category is essentially knowing about vulnerabilities at the time you decide to use a package. taking on a new dependency has inherent risk and it needs to be an informed decision. once you have a dependency, it generally becomes harder to remove over time. knowing about vulnerabilities is a great start, but there is more that we can do. many vulnerabilities arise from lack of adherence to security best practices in software development processes. are all contributors using two-factor authentication ( fa)? does the project have continuous integration set up and running tests? is fuzzing integrated? these are the types of security checks that would help consumers understand the risks they’re taking on with new dependencies. packages with a low “score” warrant a closer review, and a plan for remediation. the recently announced security scorecards project from openssf attempts to generate these data points in a fully automated way. using scorecards can also help defend against prevalent typosquatting attacks (malevolent packages with names similar to popular packages), since they would score much lower and fail many security checks. improving the development processes for critical software is related to vulnerability prevention, but deserves its own discussion further down in our post. fix or remove vulnerabilities the general problem of fixing vulnerabilities is beyond our scope, but there is much we can do for the specific problem of managing vulnerabilities in software dependencies. today there is little help on this front, but as we improve precision it becomes worthwhile to invest in new processes and tooling. one option of course is to fix the vulnerability directly. if you can do this in a backwards-compatible way, then the fix is available for everyone. but a challenge is that you are unlikely to have expertise on the problem, nor the direct ability to make changes. fixing a vulnerability also assumes the software maintainers are aware of the issue, and have the knowledge and resources for vulnerability disclosure. conversely, if you simply remove the dependency that contains the vulnerability, then it is fixed for you and those that import or use your software, but not for anyone else. this is a change that is under your direct control. these scenarios represent the two ends of the chain of dependencies between your software and the vulnerability, but in practice there can be many intervening packages. the general hope is that someone along that dependency chain will fix it. unfortunately, fixing a link is not enough: every link of the dependency chain between you and the vulnerability needs to be updated before your software will be fixed. each link must include the fixed version of the thing below it to purge the vulnerability. thus, the updates need to be done from the bottom up, unless you can eliminate the dependency altogether, which may require similar heroics and is rarely possible—but is the best solution when it is. goal: understand your options to remove vulnerabilities today, we lack clarity on this process: what progress has been made by others and what upgrades should be applied at what level? and where is the process stuck? who is responsible for fixing the vulnerability itself? who is responsible for propagating the fix? goal: notifications to speed repairs eventually, your dependencies will be fixed and you can locally upgrade to the new versions. knowing when this happens is an important goal as it accelerates reducing the exposure to vulnerabilities. we also need a notification system for the actual discovery of vulnerabilities; often new vulnerabilities represent latent problems that are newly discovered even though the actual code has not changed (such as this -year old vulnerability in the unix utility sudo). for large projects, most such issues will arise in the indirect dependencies. today, we lack the precision required to do notification well, but as we improve vulnerability precision and metadata (as above), we should also drive notification. so far, we have only described the easy case: a sequence of upgrades that are all backwards compatible, implying that the behavior is the same except for the absence of the vulnerability. in practice, an upgrade is often not backward compatible, or is blocked by restrictive version requirements. these issues mean that updating a package deep in the dependency tree must cause some churn, or at least requirement updates, in the things above it. the situation often arises when the fix is made to the latest version, say . , but your software or intervening packages request . . we see this situation often, and it remains a big challenge that is made even harder by the difficulty of getting owners to update intervening packages. moreover, if you use a package in a thousand places, which is not crazy for a big enterprise, you might need to go through the update process a thousand times. goal: fix the widely used versions it’s also important to fix the vulnerability in the older versions, especially those in heavy use. such repair is common practice for the subset of software that has long-term support, but ideally all widely used versions should be fixed, especially for security risks. automation could help: given a fix for one version, perhaps we can generate good candidate fixes for other versions. this process is sometimes done by hand today, but if we can make it significantly easier, more versions will actually get patched, and there will be less work to do higher in the chain. to summarize, we need ways to make fixing vulnerabilities, especially in dependencies, both easier and more timely. we need to increase the chance that there is a fix for widely used versions and not just for the latest version, which is often hard to adopt due to the other changes it includes. finally, there are many other options on the “fixing” front, including various kinds of mitigations, such as avoiding certain methods, or limiting risk through sandboxes or access controls. these are important practical options that need more discussion and support. prevention for critical software the framing above applies broadly to vulnerabilities, regardless of whether they are due to bad actors or are merely innocent mistakes. although the suggested goals cover most vulnerabilities, they are not sufficient to prevent malicious behavior. to have a meaningful impact on prevention for bad actors, including supply-chain attacks, we need to improve the processes used for development. this is a big task, and currently unrealistic for the majority of open source. part of the beauty of open source is its lack of constraints on the process, which encourages a wide range of contributors. however, that flexibility can hinder security considerations. we want contributors, but we cannot expect everyone to be equally focused on security. instead, we must identify critical packages and protect them. such critical packages must be held to a range of higher development standards, even though that might add developer friction. goal: define criteria for “critical” open source projects that merit higher standards it is important to identify the “critical” packages that we all depend upon and whose compromise would endanger critical infrastructure or user privacy. these packages need to be held to higher standards, some of which we outline below. it is not obvious how to define “critical” and the definition will likely expand over time. beyond obvious software, such as openssl or key cryptographic libraries, there are widely used packages where their sheer reach makes them worth protecting. we started the criticality score project to brainstorm this problem with the community, as well collaborating with harvard on the open source census efforts. goal: no unilateral changes to critical software one principle that we follow across google is that changes should not be unilateral—that is, every change involves at least an author and a reviewer or approver. the goal is to limit what an adversary can do on their own—we need to make sure someone is actually looking at the changes. to do this well for open source is actually quite a bit harder than just within a single company, which can have strong authentication and enforce code reviews and other checks. avoiding unilateral changes can be broken down into two sub-goals: goal: require code review for critical software besides being a great process for improving code, reviews ensure that at least one person other than the author is looking at every change. code reviews are a standard practice for all changes within google. goal: changes to critical software require approval by two independent parties to really achieve the “someone is looking” goal, we need the reviewer to be independent from the contributor. and for critical changes, we probably want more than one independent review. we need to sort out what counts as “independent” review, of course, but the idea of independence is fundamental to reviews in most industries. goal: authentication for participants in critical software any notion of independence also implies that you know the actors—an anonymous actor cannot be assumed to be independent or trustworthy. today, we essentially have pseudonyms: the same person uses an identity repeatedly and thus can have a reputation, but we don’t always know the individual’s trustworthiness. this leads to a range of subgoals: goal: for critical software, owners and maintainers cannot be anonymous attackers like to have anonymity. there have been past supply-chain attacks where attackers capitalized on anonymity and worked their way through package communities to become maintainers, without anyone realizing this “new maintainer” had malicious intent (compromising source code was eventually injected upstream). to mitigate this risk, our view is that owners and maintainers of critical software must not be anonymous. it is conceivable that contributors, unlike owners and maintainers, could be anonymous, but only if their code has passed multiple reviews by trusted parties. it is also conceivable that we could have “verified” identities, in which a trusted entity knows the real identity, but for privacy reasons the public does not. this would enable decisions about independence as well as prosecution for illegal behavior. goal: strong authentication for contributors of critical software malicious actors look for easy attack vectors, so phishing attacks and other forms of theft related to credentials are common. one obvious improvement would be the required use of two-factor authentication, especially for owners and maintainers. goal: a federated model for identities to continue the inclusive nature of open source, we need to be able to trust a wide range of identities, but still with verified integrity. this implies a federated model for identities, perhaps similar to how we support federated ssl certificates today—a range of groups can generate valid certificates, but with strong auditing and mutual oversight. discussions on this topic are starting to take place in the openssf’s digital identity attestation working group. goal: notification for changes in risk we should extend notifications to cover changes in risk. the most obvious is ownership changes, which can be a prelude to new attacks (such as the recent npm event-stream compromise). other examples include discovery of stolen credentials, collusion, or other bad actor behavior. goal: transparency for artifacts it is common to use secure hashes to detect if an artifact has arrived intact, and digital signatures to prove authenticity. adding “transparency” means that these attestations are logged publicly and thus document what was intended. in turn, external parties can monitor the logs for fake versions even if users are unaware. going a step further, when credentials are stolen, we can know what artifacts were signed using those credentials and work to remove them. this kind of transparency, including the durable public logs and the third-party monitoring, has been used to great success for ssl certificates, and we have proposed one way to do this for package managers. knowing you have the right package or binary is similar to knowing you are visiting the real version of a web site. goal: trust the build process ken thompson's turing award lecture famously demonstrated in that authentic source code alone is not enough, and recent events have shown this attack is a real threat. how do you trust your build system? all the components of it must be trusted and verified through a continuous process of building trust. reproducible builds help—there is a deterministic outcome for the build and we can thus verify that we got it right—but are harder to achieve due to ephemeral data (such as timestamps) ending up in the release artifact. and safe reproducible builds require verification tools, which in turn must be built verifiably and reproducibly, and so on. we must construct a network of trusted tools and build products. trust in both the artifacts and the tools can be established via “delegation”, through a variant of the transparency process described above called binary authorization. internally, the google build system signs all artifacts and produces a manifest that ties it to the source code. for open source, one or more trusted agents could run the build as a service, signing the artifact to prove that they are accountable for its integrity. this kind of ecosystem should exist and mostly needs awareness and some agreements on the format of attestations, so that we can automate the processes securely. the actions in this section are great for software in general, and are essentially in use today within google, but they are heavier weight than usual for open source. our hope is that by focusing on the subset of software that is critical, we can achieve these goals at least for that set. as the tooling and automation get better, these goals will become easier to adopt more widely. summary the nature of open source requires that we solve problems through consensus and collaboration. for complex topics such as vulnerabilities, this implies focused discussion around the key issues. we presented one way to frame this discussion, and defined a set of goals that we hope will accelerate industry-wide discourse and the ultimate solutions. the first set of goals apply broadly to vulnerabilities and are really about enabling automation and reducing risk and toil. however, these goals are not enough in the presence of adversaries or to prevent “supply chain” attacks. thus we propose a second set of goals for critical software. the second set is more onerous and therefore will meet some resistance, but we believe the extra constraints are fundamental for security. the intention is to define collectively the set of “critical” software packages, and apply these higher standards only to this set. although we have various opinions on how to meet both sets of goals, we are but one voice in a space where consensus and sustainable solutions matter most of all. we look forward to this discussion, to promoting the best ideas, and eventually to solutions that both strengthen and streamline the security of open source that we all depend on. notes ideally, depended-upon versions should be stable absent an explicit upgrade, but behavior varies depending on the packaging system. two that aim for stability rather than fast upgrades are go modules and nuget, both of which by default install upgrades only when the requirements are updated; the dependencies might be wrong, but they only change with explicit updates. ↩ google no comments : post a comment    labels  android android security android tr app security big data biometrics blackhat chrome chrome security federated learning gboard google play google play protect pha family highlights privacy security spyware targeted spyware vulnerabilities  archive    feb jan   dec nov oct sep aug jul jun may apr mar feb jan   dec nov oct sep aug jul jun may apr mar feb jan   dec nov oct sep aug jul jun may apr mar feb jan   dec nov oct sep jul jun may apr mar feb jan   dec nov oct sep aug jul jun may apr mar feb jan   dec nov oct sep aug jul jun may apr mar feb jan   dec nov oct sep aug jul jun apr mar feb jan   dec nov oct aug jun may apr mar feb jan   dec sep aug jun may apr mar feb jan   dec nov oct sep aug jul jun may apr mar feb   nov oct sep aug jul may apr mar   nov oct aug jul jun mar   dec nov oct aug jul may feb   nov oct sep jul jun may feed follow @google follow give us feedback in our product forums. google privacy terms wayback machine see what's new with book lending at the internet archive a line drawing of the internet archive headquarters building façade. an illustration of a magnifying glass. an illustration of a magnifying glass. an illustration of a horizontal line over an up pointing arrow. upload an illustration of a person's head and chest. sign up | log in an illustration of a computer application window wayback machine an illustration of an open book. books an illustration of two cells of a film strip. video an illustration of an audio speaker. audio an illustration of a . " floppy disk. software an illustration of two photographs. images an illustration of a heart shape donate an illustration of text ellipses. more an icon used to represent a menu that can be toggled by interacting with this icon. about blog projects help donate an illustration of a heart shape contact jobs volunteer people search metadata search text contents search tv news captions search archived websites advanced search sign up for free log in the wayback machine requires your browser to support javascript, please email info@archive.org if you have any questions about this. the wayback machine is an initiative of the internet archive, a (c)( ) non-profit, building a digital library of internet sites and other cultural artifacts in digital form. other projects include open library & archive-it.org. your use of the wayback machine is subject to the internet archive's terms of use. home - digital preservation coalition menu about back about credits and acknowledgements about the digital preservation coalition join us back join us apply for full or associate membership benefits of membership criteria and fees prospectus (en) عربى (ar) prospekt (de) programa (es) prospectus (fr) our members back our members allied organisations our supporters back our supporters programme governance back governance annual reports articles of association policies contact us news back news job vacancies new members of the dpc digital preservation back digital preservation what is digital preservation? discover good practice back discover good practice get help with your digital preservation dedicated support for members digital preservation handbook technology watch publications back technology watch publications dpc style guide older or superseded reports working groups and task forces curation costs exchange case studies translations implement digital preservation back implement digital preservation rapid assessment model back rapid assessment model who can use dpc ram? how long will it take? how to use dpc ram how often should it be used? what to do after dpc ram grille d'évaluation rapide (fr) デジタル保存連合ラピッド・アセスメントモデル (jp) acknowledgements policy toolkit back policy toolkit introduction what makes a good digital preservation policy? step-by-step-guide to building a preservation policy template for building a preservation policy university of bristol case study further resources executive guide back executive guide introduction organization type organizational motivators what are the risks? what are the opportunities? what does digital preservation need? facts and figures templates le guide pour la préservation numérique (fr) guía ejecutiva para la preservación digital (es) acknowledgements business case toolkit procurement toolkit back procurement toolkit introduction lessons learned in digital preservation procurement common requirements for repository procurement dp futures webinars with dpc supporters further resources champion digital preservation back champion digital preservation bit list back bit list interpreting and using the bitlist lower risk vulnerable endangered critically endangered practically extinct concern the bit list jury digital preservation awards world digital preservation day engaging with public policy train your staff back train your staff novice to know-how career development fund back career development fund career development fund application guidelines getting started and making progress training modules collaborative projects back collaborative projects copim novice to know-how reliable, robust and resilient digital infrastructure for nuclear decommissioning completed projects events back events upcoming events past events back past events webinars digital preservation awards back digital preservation awards overview the finalists eligibility and faqs timetable judges roll of honour sponsors digital preservation awards digital preservation awards digital preservation awards digital preservation awards digital preservation award digital preservation award digital preservation award world digital preservation day blog login login about news digital preservation events blog community engagement we enable a growing number of agencies and individuals in all sectors and in all countries to participate in a dynamic and mutually supportive digital preservation community. about us advocacy we campaign for a political and institutional climate more responsive and better informed about the digital preservation challenge; raising awareness about the new opportunities that resilient digital assets create. find out more workforce development we provide opportunities for our members to acquire, develop and retain competent and responsive workforces that are ready to address the challenges of digital preservation. learn more capacity building we support and assure our members in the delivery and maintenance of high quality and sustainable digital preservation services through knowledge exchange, technology watch, research and development. learn more good practice and standards we identify and develop good practice and standards that make digital preservation achievable, supporting efforts to ensure services are tightly matched to shifting requirements. find out more the 'bit list' of digitally endangered species the dpc has published an update to the bitlist on world digital preservation day see the list digital preservation awards the search for the very best work in digital preservation across all sectors reached its exciting culmination on world digital preservation day! see the winners management and governance we ensure the dpc is a sustainable, competent organisation focused on member needs, providing a robust and trusted platform for collaboration within and beyond the coalition. join us we are building: a dynamic and mutually supportive digital preservation community. a political and institutional climate more responsive and better informed about the digital preservation challenge. workforces that are ready to address the challenges of digital preservation. high quality and sustainable digital preservation services. good practice and standards that make digital preservation achievable. a sustainable, competent organization focussed on member needs. join us digital preservation handbook the handbook identifies good practice in creating, managing and preserving digital materials. by providing a strategic overview of the key issues, discussion and guidance on strategies and activities, and pointers to key projects and reports, the handbook provides guidance for institutions and individuals and a range of tools to help them identify and take appropriate actions. view handbook two new webinar series: transforming archives and using digital archives for academic research february the digital repository of ireland is running two new webinar series this february, march and april read article more, more, more: introducing novice to know-how . sharon mcmeekin last updated on february sharon introduces a new phase of the "novice to know-how" online training project, developing content on providing access to preserved digital content. go to blog do (preservation) actions speak louder than words?: a briefing day on preservation planning and technology watch | upcoming events february | : - : gmt at the dpc’s ‘connecting the bits’ unconference in june this was an area that was flagged up as a challenge. member’s reported a substantial gap between where they are now and where they would like to be. view event dpc chatter @dpc_chat rt @williamkilbride: a dreadful reminder of the price paid for truth and the trust we place in records. paging archivists and records mana… view tweet feb dpc on instagram see all privacy policy © digital preservation coalition. unless otherwise stated, content is shared under cc-by-nc licence login remember me log in forgot login? sign up × scroll to top success fail oct nov dec capture nov about this capture collected by organization: archive team formed in , the archive team (not to be confused with the archive.org archive-it team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. the group is % composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history. history is littered with hundreds of conflicts over the future of a community, group, location or business that were "resolved" when one of the parties stepped ahead and destroyed what was there. with the original point of contention destroyed, the debates would fall to the wayside. archive team believes that by duplicated condemned data, the conversation and debate can continue, as well as the richness and insight gained by keeping the materials. our projects have ranged in size from a single volunteer downloading the data to a small-but-critical site, to over volunteers stepping forward to acquire terabytes of user-created data to save for future generations. the main site for archive team is at archiveteam.org and contains up to the date information on various projects, manifestos, plans and walkthroughs. this collection contains the output of many archive team projects, both ongoing and completed. thanks to the generous providing of disk space by the internet archive, multi-terabyte datasets can be made available, as well as in use by the wayback machine, providing a path back to lost websites and work. our collection has grown to the point of having sub-collections for the type of data we acquire. if you are seeking to browse the contents of these collections, the wayback machine is the best first stop. otherwise, you are free to dig into the stacks to see what you may find. the archive team panic downloads are full pulldowns of currently extant websites, meant to serve as emergency backups for needed sites that are in danger of closing, or which will be missed dearly if suddenly lost due to hard drive crashes or server failures. collection: archive team: urls timestamps we've detected that javascript is disabled in your browser. would you like to proceed to legacy twitter? yes something went wrong, but don’t fret — let’s give it another shot. about | cs mesh eosc about partners contribution to eosc & fair technology projects science mesh data services join our science mesh community users & developers news & events news events media articles & papers media kit presentations contact about partners contribution to eosc & fair technology projects science mesh data services join our science mesh community users & developers news & events media articles & papers media kit presentations follow us about breadcrumb home why cs mesh eosc? cloud services for synchronization and sharing (cs ) have been widely deployed in the research and educational space, mostly by e-infrastructure providers, nrens (national research & education networks) and major research institutions. these services, usually in daily workflows for hundreds of thousands of users (including researchers, students, scientists and engineers) remain largely disconnected, and they are developed and deployed in isolation from each other. combining these various systems and services would boost open-science by presenting a joint, coordinated service to the users in the research and education space at a pan-european level. cs mesh eosc focuses on this major technical, but especially societal challenge. cs mesh eosc - interactive and agile sharing mesh of storage, data and applications for eosc - aims to create an interoperable federation of data and higher-level services to enable friction-free collaboration between european researchers. cs mesh eosc will connect locally and individually provided services, and scale them up at the european level and beyond, with the promise of reaching critical mass and brand recognition among european scientists that are not usually engaged with specialist einfrastructures. science mesh - cs mesh eosc main asset cs mesh eosc will provide a pan-european natively fair and gdpr compliant data storage and sharing fabric. science mesh, the main asset of the project, provides an interoperable platform to easily sync & share, and deploy applications and software components within the full cs community to extend functionality of the service. science mesh allows the users to retain control over their remote or domestic datasets, while at the same time becoming fair compatible and integrated with the european open science cloud (eosc). users will also be able to directly access the service provided by sciencemesh from easy-to-use interfaces and discover the different functionalities. these services are independent from the specific field of work of the researchers, thus enabling them to reach a wide audience of stakeholders from both the academia and the research industry interested in collaborating to increase scientific knowledge. the added-value for the stakeholders of cs mesh eosc is constituted by the possibility for them to contribute in developing application plugins or easily set-up federations in their own scope and develop their own activities on top of them. cs mesh eosc key facts starting date: january , duration: months partners from member states who will benefit from cs mesh eosc? initiatives that can benefit from sciencemesh by gaining access to a vast amount of data for free and improving scientific knowledge: join the movement and pre-register to our free services! follow cs mesh eosc on twitter: https://twitter.com/cs org connect with us on linkedin: https://www.linkedin.com/company/cs mesh eosc/ search search latest news the science mesh workshop: unlocking open science and digital sovereignty in europe january cs mesh eosc – making data sharing in europe child’s play: an interview with the coordinator jakub moscicki december the initial definition of science mesh protocols and application programming interfaces is ready! november partners cs mesh eosc - interactive and agile/responsive sharing mesh of storage, data and applications for eosc, has received funding from the european union’s horizon research and innovation programme under grant agreement no. . © copyright - cs mesh eosc footer menu contact login privacy policy terms of use wayback machine see what's new with book lending at the internet archive a line drawing of the internet archive headquarters building façade. an illustration of a magnifying glass. an illustration of a magnifying glass. an illustration of a horizontal line over an up pointing arrow. upload an illustration of a person's head and chest. sign up | log in an illustration of a computer application window wayback machine an illustration of an open book. books an illustration of two cells of a film strip. video an illustration of an audio speaker. audio an illustration of a . " floppy disk. software an illustration of two photographs. images an illustration of a heart shape donate an illustration of text ellipses. more an icon used to represent a menu that can be toggled by interacting with this icon. about blog projects help donate an illustration of a heart shape contact jobs volunteer people search metadata search text contents search tv news captions search archived websites advanced search sign up for free log in the wayback machine requires your browser to support javascript, please email info@archive.org if you have any questions about this. the wayback machine is an initiative of the internet archive, a (c)( ) non-profit, building a digital library of internet sites and other cultural artifacts in digital form. other projects include open library & archive-it.org. your use of the wayback machine is subject to the internet archive's terms of use. trump’s tweets. tldr: trump’s tweets are gone from… | by ed summers | jan, | documenting docnow sign in coverage of trump’s twitter account at the internet archive trump’s tweets ed summersfollow jan · min read trump’s tweets are gone from twitter.com but still exist spectrally in various states all over the web. after profiting off of their distribution twitter now has a responsibility to provide meaningful access to the trump tweets as a read only archive. so trump’s twitter account is gone. finally. it’s strange to have had to wait until the waning days of his presidency to achieve this very small and simple act of holding him accountable to twitter’s community guidelines…just like any other user of the platform. better late than never, especially since his misinformation and lies can continue to spread after he has left office. but isn’t it painful to imagine what the last four years (or more) could have looked like if twitter and the media at large had recognized their responsibility and acted sooner? when twitter suspended trump’s account they didn’t simply freeze it and prevent him from sending more hateful messages. they flipped a switch that made all the tweets he has ever sent disappear from the web. these are tweets that had real material consequences in the world. as despicable as trump’s utterances have been, a complete and authentic record of them having existed is important for the history books, and for holding him to account. where indeed? one hopes that they will end up in the national archives (more on that in a moment). but depending on how you look at it, trump’s tweets are everywhere. twitter removed trump’s tweets from public view at twitter.com. but fortunately, as shawn jones notes, embedded tweets like the one above persist the tweet text into the html document itself. when a tweet is deleted from twitter.com the text stays behind elsewhere on the web like a residue, as evidence (that can be faked) of what was said and when. it’s difficult to say whether this graceful degradation was an intentional design decision to make their content more resilient, or it was simply a function of twitter wanting their content to begin rendering before their javascript had loaded and had a chance to emboss the page. but design intent isn’t really what matters here. what does matter is the way this form of social media content degrades in the web commons. kari kraus calls this process “spectral wear”, where digital media “help mitigate privacy and surveillance concerns through figurative rather than quantitative displays, reflect and document patterns of use, and promote an ethics of care.” this spectral wear is a direct result of tweet embed practices that twitter itself promulgates while simultaneously forbidding it developer terms of service: if twitter content is deleted, gains protected status, or is otherwise suspended, withheld, modified, or removed from the twitter applications (including removal of location information), you will make all reasonable efforts to delete or modify such twitter content (as applicable) as soon as possible. fortunately for history there has probably never been a more heavily copied social media content than donald trump’s tweets. we aren’t immediately dependent on twitter.com to make this content available because of the other other places on the web where it exists. what does this copying activity look like? i intentionally used copied instead of archived above because the various representations of trump’s tweets vary in terms of their coverage, and how they are being cared for. given their complicity in bringing trump’s messages of division and hatred to a worldwide audience, while profiting off of them, twitter now have a responsibility to provide as best a representation of this record for the public, and for history. we know that the trump administration have been collecting the @realdonaldtrump twitter account, and plan to make it available on the web as part of their responsibilities under the presidential records act: the national archives will receive, preserve, and provide public access to all official trump administration social media content, including deleted posts from @realdonaldtrump and @potus. the white house has been using an archiving tool that captures and preserves all content, in accordance with the presidential records act and in consultation with national archives officials. these records will be turned over to the national archives beginning on january , , and the president’s accounts will then be made available online at nara’s newly established trumplibrary.gov website. nara is the logical place for these records to go. but it is unclear what shape these archival records will take. sure the library of congress has (or had) its twitter archive. it’s not at all clear if they are still adding to it. but even if they are lc probably hasn’t felt obligated to collect the records of an official from the executive branch, since they are firmly lodged in the legislative. then again they collect animated gifs so, maybe? reading between the lines it appears that a third party service is being used to collect the social media content: possibly one of the several e-discovery tools like archivesocial or hanzo. it also looks like the trump administration themselves have entered into this contract, and at the end of its term (i.e. now) will extract their data and deliver it to nara. given their past behavior it’s not difficult to imagine the trump administration not living up to this agreement in substantial ways. this current process is a slight departure from the approach taken by the obama administration. obama initiated a process where platforms [migrate official accounts] to new accounts that were then managed going forward by nara (acker & kriesberg, ). we can see that this practice being used again on january , when biden became president. but what is different is that barack obama retained ownership of his personal account @barackobama, which he continues to use. arguably this is good because trump used his twitter account in a very different manner than obama. this process is complicated further by the fact that trump’s account has been removed from the web. nevertheless, nara has announced that they will be archiving trump’s now deleted (or hidden) personal account. a number of trump administration officials, including president trump, used personal accounts when conducting government business. the national archives will make the social media content from those designated accounts publicly available as soon as possible. the question remains, what representation should be used, and what is twitter’s role in providing it? meanwhile there are online collections like the trump twitter archive, the new york times’ complete list of trump’s twitter insults, propublica’s politwoops and countless github repositories of data which have collected trump’s tweets. these tweets are used in a multitude of ways including things as absurd as a source for conducting trades on the stock market. but seeing these tweets as they appeared in the browser, with associated metrics and comments is important. of course you can go view the account in the wayback machine and browse around. but what if we wanted a list of all the trump tweets? how many times were these tweets actually archived? how complete is the list? after some experiments with the internet archive’s api it’s possible to get a picture of how the tweets from the @realdonaldtrump account have been archived there. there are a few wrinkles because a given tweet can have many different url forms (e.g. tracking parameters in the url query string). in addition just because there was a request to archive a url for something that looks like a realdonaldtrump tweet url doesn’t mean it resulted in a successful response. success here means a ok from twitter.com when resolving the url. factoring these issues into the analysis it appears the wayback machine contains (at least) , , snapshots of trump’s tweets: https://twittter.com/realdonaldtrump/status/{tweet-id} of these millions of snapshots there appear to be , unique tweets. this roughly correlates with the k total tweets suggested by the last profile snapshots of the account. the maximum number of times in one day that his tweets were archived was , times on february , . here’s what the archive snapshots of trump’s tweets look like over time (snapshots per week). it is relatively easy to use the csv export from the trump twitter archive project to see what tweets they know about that the internet archive does not and vice-versa (for the details see the jupyter notebook and sqlite database here). it looks like there are tweet ids in the trump twitter archive that are missing from the internet archive. but further examination shows that many of these are retweets, which in twitter’s web interface, have sometimes redirected back to the original tweet (a found instead of a ok). after removing these retweets to specifically look at trump’s own tweets there are only tweets in the trump twitter archive that are missing from the internet archive. of these are in fact retweets that have been misclassified as tweets by the trump twitter archive. one of the three remaining is this one, which is identified in the trump twitter archive as deleted, and wasn’t collected quick enough by the internet archive before it was deleted: roger stone was targeted by an illegal witch hunt tha never should have taken place. it is the other side that are criminals, including the fact that biden and obama illegally spied on my campaign — and got caught!” sure enough, over at the politwoops project you can see that this tweet was deleted seconds after it was sent: deleted tweet from politwoopsturning the table it’s also possible to look at what tweets are in the internet archive but not in the trump twitter archive. it turns out that there are , tweet identifiers in the wayback machine for trump’s tweets which do not appear in the trump twitter archive. looking a bit closer we can see that some are clearly wrong, because the id itself is too small a number, or too large. and then looking at some of the snapshots it appears that they often don’t resolve, and simply display a “something went wrong” message: yes, something definitely went wrong (in more ways than one). just spot checking a few there also appear to be some legit tweets in the wayback that are not in the trump twitter archive like this one: notice how the media will not play? these tweets appear to be in the internet archive, but it’s not at all certain what their state is. it would take some heavy manual curation work to sort through these tweet ids to see which ones are legit, and which ones aren’t. but if you are interested here’s an editable google sheet of the tweets in wayback that don’t appear in the trump archive. the situation gets even more complicated because there appear to be deleted realdonaldtrump tweets in politwoops that are not in the internet archive. finally, here is a list of the top ten archived (at internet archive) tweets. the counts here reflect all the variations for a given tweet url. so they will very likely not match the count you see in the wayback machine, which is for the specific url (no query paramters). thank you alabama! #trump #supertuesday , make america great again! , thank you georgia!#supertuesday #trump , such a beautiful and important evening! the forgotten man and woman will never be forgotten again. we will all come together as never before , watching the returns at : pm. #electionnight #maga🇺🇸 https://t.co/hfujerzbod , #electionday https://t.co/mxraxyntjy https://t.co/fzhoncih , i will be watching the election results from trump tower in manhattan with my family and friends. very exciting! , i love you north carolina- thank you for your amazing support! get out and https://t.co/hfihperfgz tomorrow!watch:… https://t.co/jzzfquznyh , still time to #votetrump! #ivoted #electionnight https://t.co/uztyay ba , watching my beautiful wife, melania, speak about our love of country and family. we will make you all very proud…. https://t.co/dikmsntlc the point of this rambling data spelunking, if you’ve made it this far, is to highlight the degree to which trump’s tweets have been archived (or collected), and how the completeness and quality of those representations is very fluid and difficult to ascertain. hopefully twitter is working with nara to provide as complete a picture as possible of what trump said on twitter. as much as we would like to forget, we must not. references acker, a., & kriesberg, a. ( ). tweets may be archived: civic engagement, digital preservation and obama white house social media data. proceedings of the association for information science and technology, ( ), – . kraus, k. ( ). the care of enchanted things. in m. k. gold & l. f. klein (eds.), debates in the digital humanities . retrieved from https://www.jstor.org/stable/ . /j.ctvg hk. thanks to jess ogden, shawn walker, adam kriesberg and derek willis who read versions of this post and provided feedback/ideas. all the mistakes are my own. originally published at https://inkdroid.org. documenting docnow news from the documenting the now project follow some rights reserved archive social media politics clap clap written by ed summers follow i’m a software developer at @umd_mith & study archives on/of the web at @ischoolumd follow documenting docnow follow news from the documenting the now project follow written by ed summers follow i’m a software developer at @umd_mith & study archives on/of the web at @ischoolumd documenting docnow follow news from the documenting the now project more from medium streams ed summers in documenting docnow remembering bassem masri ed summers in documenting docnow disinformation metadata ed summers in documenting docnow the important emotional labor of librarians most people never think about oleg kagan in everylibrary the kings of recycling are fighting over scraps adam popescu in onezero anton wilhelm amo, a black philosopher in th century europe temitope ajileye #influenceforsale: venezuela’s twitter propaganda mill @dfrlab in dfrlab “the task lamp” by bruce sterling ( ) bruce sterling learn more. medium is an open platform where million readers come to find insightful and dynamic thinking. here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. learn more make medium yours. follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. explore share your thinking. if you have a story to tell, knowledge to share, or a perspective to offer — welcome home. it’s easy and free to post your thinking on any topic. write on medium about help legal get the medium app official elasticsearch pricing: elastic cloud, managed elasticsearch | elastic products customers learn company pricing try freelogin products elastic enterprise search workplace, website, and app search elastic observability unified logs, metrics, and apm data elastic security siem, endpoint, and threat hunting elastic (elk) stack elasticsearchstore, search, analyze kibanavisualize, navigate, share deploy nowview integrationsview all products learn blogstutorials, updates, people docsproduct guides eventselasticon and meetups videoswebinars, customers, tutorials communitygroups, forums, code consultingproject acceleration packages customer successsuccess planning guide trainingcritical skill-building and certification featured topics elastic (elk) stack upgradingelastic stack getting startedkibana getting startedapp search getting startedworkplace search getting startedapm overview upgrade the elastic stack. watch video company aboutour story, teams, source code customerscase studies, videos, blogs jobsperuse our opportunities partnersfind or become a partner investor relationsresults, filings, resources awardsrecognizing remarkable work news elastic . releasedwhat’s new in elastic enterprise search . . what's new in elastic observability . . what’s new in elastic security . . what’s new in elastic cloud for . . elastic is a search company. watch video contact login try free elastic pricing the best way to consume elastic is elastic cloud, a public cloud managed service available on major cloud providers. customers who want to manage the software themselves, whether on public, private, or hybrid cloud, can download the elastic stack. standard a great place to start for small projects capabilities: core elastic stack security solution features in elastic enterprise search, observability, and security elastic app search elastic workplace search elastic apm elastic logs elastic metrics elastic uptime elastic siem malware prevention kibana lens, elastic maps, and canvas alerting and in-stack actions core free and open features clustering & high availability powerful search and analysis data visualization and dashboarding support ticket-based support (elastic cloud only) elastic cloud as low as $ /month try free self-managed (basic) includes basic features of elastic cloud on kubernetes to deploy, orchestrate, and operate elastic products on kubernetes. download free gold dedicated support and enhanced features everything in standard plus: reporting third-party alerting actions watcher support business hours support ticket-based support support response time slas elastic cloud as low as $ /month try free self-managed includes key features like third-party alerting and reporting. contact us platinum advanced functionality with / support everything in gold plus: advanced elastic stack security features machine learning cross-cluster replication support / / support ticket-based support enhanced support response time slas elastic cloud as low as $ /month try free self-managed includes advanced features for security, machine learning, and high availability. contact us enterprise data tier support and endpoint protection everything in platinum plus: access to elastic endgame searchable snapshots (beta) support for searchable cold (beta) and frozen tiers (coming soon) support / / support ticket-based support enhanced support response time slas elastic cloud get in touch for more info try free self-managed includes cloud orchestration — elastic cloud enterprise — and enterprise features for elastic cloud on kubernetes. contact us want to get into the pricing details for elastic cloud? use our elastic cloud pricing calculator. additional pricing resources want to learn more about our features, support, and more? see exactly what we have to offer with the elastic stack and our solutions. feature matrix peruse our comprehensive feature list to see availability for each of our subscription tiers for elastic cloud and self-managed. elastic cloud featuresself-managed features supported platforms view platform and software configurations eligible for support. support matrix faq still have queries? check out some of our frequently asked questions on licensing, support, pricing, and more. pricing faqpricing philosophyelastic licenseelastic license v faqbenefits of a managed service customers whose enterprise subscriptions use ece/ece instances as the billing metric must agree to additional terms before they can access these features. please contact us. available for gold tier on self-managed and standard tier on elastic cloud questions? we love a good query. let's hear more about your use cases and the problems you are trying to solve. we'll work together on finding the right solution for you. contact us subscribe to our newsletter follow us products & solutions enterprise search observability security elastic stack elasticsearch kibana logstash beats subscriptions pricing company careers board of directors contact resources documentation what is the elk stack? what is elasticsearch? migrating from splunk compare aws elasticsearch us public sector follow us language deutsch english español français 日本語 한국어 简体中文 português trademarks terms of use privacy brand sitemap © . elasticsearch b.v. all rights reserved elasticsearch is a trademark of elasticsearch b.v., registered in the u.s. and in other countries. apache, apache lucene, apache hadoop, hadoop, hdfs and the yellow elephant logo are trademarks of the apache software foundation in the united states and/or other countries. © . elasticsearch b.v. all rights reserved maps, geocoding, and navigation apis & sdks | mapbox products products maps smooth, fast, real-time maps navigation turn-by-turn routing search turn addresses into coordinates studio design custom maps vision second set of eyes for your car data build with mapbox data platforms web mobile ar atlas solutions industries logistics consumer apps business intelligence all industries → solutions asset tracking real time maps interactive storytelling all solutions → use cases data visualization store locator turn-by-turn navigation all use cases → webinars →how-to videos → documentation company aboutcustomerscareersdiversity & inclusioncommunity pricingblog product update announcing  mobile maps  sdk v mobile maps  sdk v → product update announcing gl js v learn about gljs → new service announcing mapbox tiling service learn about mts → new product understand where and when movement is happening learn about movement data → product update announcing navigation sdk for ios and android learn about navigation sdk → maps and location  for developers precise location data and powerful developer tools to change the way we navigate the world. start mapping for free maps our apis, sdks, and live updating map data give developers tools to build better mapping, navigation, and search experiences across platforms. learn about maps → mapbox studio mapbox studio is like photoshop for maps. we give designers control over everything from colors and fonts, to d features and camera angles, to the pitch of the map as a car enters a turn. design in studio → navigation mapbox provides powerful routing engines, traffic-aware travel times, and intuitive turn-by-turn directions to help you build engaging navigation experiences. explore navigation → search search and geocoding are tied to everything we build — maps, navigation, ar — and underly nearly every app that helps humans explore their world. discover search → case study plan and optimize your route at the snap of a photo learn more → vision the mapbox vision sdk describes every curb, lane, street sign, and road hazard it sees as data. vision's ai-powered semantic segmentation, object detection, and classification deliver precise navigation guidance, display driver assistance alerts, and detect and map road incidents. discover vision → data our data is powered by hundreds of data sources and a distributed global network of more than half a billion monthly active users. learn about data → atlas atlas is truly all of mapbox in a box — including our fast vector maps technology, allowing you to render your data at fps. create custom map styles with mapbox studio and then use them to power your on-premises applications running gl js, our maps sdks for ios and android, and even our maps sdk for unity. discover atlas → showcase discover more from our customers → products maps navigation search studio vision sdk data pricing platforms web mobile ar atlas support documentationpaid support tutorialshelp company about customers careers press community contact solutions industries automotive logistics data visualization asset tracking store locator webinars events blog © mapbox termsprivacysecurity by browsing this website you agree to our cookie policy agree shelley gullikson – usability and user experience in academic libraries. mine mostly. skip to content shelley gullikson usability and user experience in academic libraries. mine mostly. menu about web librarians who do ux: access presentation october , ~ shelley ~ leave a comment this is the text (approximately) of my presentation from the virtual access conference on oct. , , “web librarians who do ux: we are so sad, we are so very very sad.” last year, i was doing interviews with library people who do user experience work and noticed that people who were primarily focused on the web had the most negative comments and fewest positive comments overall. it made me think of the song from scott pilgrim—the comic and the movie—“i am so sad, i am so very very sad.” so there’s the title. and i’m saying “we are so sad” because i am also a web person who does ux work. and a lot of what i heard seemed familiar. i want to say that although the title and the visuals are based around a comic and comic book movie, i’m not trying to be flip. a lot of the people who i talked to were very open about being unhappy. not everyone was unhappy. but, there was a lot in common among the people who said they were struggling and those who were pretty positive. here are some quotes from people who were generally pretty positive : “how much can i do that no one will block me from doing?” “why am i really here then, if i’m just moving things around the page?” [i keep feedback] “for promotion purposes but also not-being-sad purposes.” and from the not-so-positive : “you have all the people who have their own personal opinions… and you’re like “you’re violating every good norm of website development”… they think their opinion is just as good as anyone else’s opinion. … that can definitely demoralize you.” “i bounce back and forth between, for my own sanity’s sake, needing to be apathetic about it, saying ‘i can’t change this therefore i can’t be stressed about it’, and also on the other hand, caring that we have crappy stuff out there and wanting to improve it.” “it is what it is. there’s lots of other things to be disappointed by.” heartbreaking, right? so why is this the case? first a tiny bit of background on the research project. the aim of the project was to look at how ux work is structured and supported in academic libraries and then to examine those supports within the context of the structures. i did hour-long semi-structured interviews with people in academic libraries from countries (canada, the us, the uk, sweden, and norway). these were library workers who do ux, so not necessarily librarians, and not necessarily people in ux positions. the people i’m talking about today focus mostly on the web in their jobs. the frustrations of web folks were particularly striking because i didn’t ask a question about frustrations; i asked what supports were helpful to them and what would be helpful. admittedly, asking “what would be helpful” is going to bring up deficiencies, but i didn’t ask what supports were missing or what they found frustrating in their work. and again, the web folks talked more about frustrations and difficulties than participants who didn’t have a web focus. so let’s dig in a bit. why, specifically, are we so sad? first off, we have a tendency to want to think big! do more! “that’s what motivates me—the opportunity to really sit down, talk, observe, have a conversation with our users, how they approach the website, how they approach the research process, how they approach finding out about our services and how we in turn can better highlight our resources, how we can better highlight our collections, our services.” “if i see people struggling with things, i want to make them better.” “i don’t want ux to be just a website thing. i don’t want people to think of it ‘oh, it’s just a web thing.’ i want it to be in everything.” “i just see lots of potential all the time. i see potential everywhere, the whole library. i see things we could do that would enhance things.” that doesn’t sound sad. there’s energy and excitement in those words! but contrast it with: “why am i really here then, if i’m just moving things around the page? i’m trying to get deeper. i’m trying to get a better understanding. it’s not just a matter of moving things around.” web people who do ux are, i think, well positioned—and perhaps uniquely positioned—to see big picture problems across the library. one participant told me they found that users were confused about the circulation section of the website because there were different policies underlying it; they could rewrite the web content but couldn’t do anything about the underlying spaghetti of policies. another said that users found the floor maps confusing but the maps reflected the language used on the library’s signage; they could put clear language on the website’s floor maps but couldn’t do anything about the signage in the building. so we see these problems and naturally want to solve them. we get excited about the potential to make more things better. and we chafe against having to think smaller and do less. which brings us to: lack of authority. lack of authority often comes up around those larger library issues. one participant put it this way: “the ux work is actually informing something else to happen. whether that’s a space being reorganized or a webpage being redesigned—the ux work is informing this other work. right? so it would be easier for me to do the ux work if i could actually do the work that it’s informing.” another person was even having problems at the research stage: [i’d like to] “have the authority and freedom to actively engage with users.” and someone else, in talking specifically about their web work said: “nobody tries to stop me.” the implication being that people try to stop them when they do other things. but for many participants there was a lack of authority even when dealing with the library website: “the web team doesn’t feel like they can really make changes without consult, consult, consult with everybody even though – even if, and even though – the web team has web expertise.” “just because i’m our internal expert on this stuff doesn’t mean i can persuade everybody.” “there’s too much of a sense that these things have to be decided by consensus” “everyone feels… like they should have the right to declare how databases should work, how links should be configured, things like that.” [each library unit feels] “they have the right to do whatever they want with their content and their presentation. … i’m not their boss and they realize that. i’m happy to draw up more guidelines and stuff like that but if i’m not allowed to enforce that… [it’s] hard to keep things together when you just have to go hat in hand to people and say ‘pretty please, stop breaking the guidelines.’” one participant described how having no authority for the one thing they were responsible for made them feel: “of course that has stymied my initiative, not to mention my disposition. my purpose even.” another frustration that came through was resistance from colleagues. a few comments have already touched on colleagues ignoring expertise but resistance comes through in other ways one participant described how they always approach a particular department: [i’m] “treading very slowly and carefully and choosing my words very carefully” another said: “are they deliberately killing the idea but trying to avoid being disagreeable about it but just letting it die from attrition, or do they really actually mean it when they say they agree with the idea in principle but just don’t want to be bothered to follow through? i don’t know – i can’t tell the difference.” these are things participants were told by their colleagues: a manager said that “staff felt unfairly targeted” by their work in opposing to changes to the website: “we have to keep it this way because we teach it this way” and similarly, “it’s our job to teach people how to use, not our job to make it easier to use.” so, not surprisingly, these kinds of things make us feel isolated. feelings of isolation come through in a few ways. some participants felt they were completely on their own when deciding where to focus their attention. this is one participant talking about being new in their position: “i remember asking for, if there were any focuses they wanted to focus on… they said ‘no, there’s nothing. we don’t have any direction for you to go in.” that lack of direction is often coupled with not having colleagues who do the same work: “it’s really me and up to me to figure out where to focus my attention by myself. so sometimes having someone to bounce ideas off of and talk things through with… would be nice.” and when no one else does what you do: “sometimes that’s a barrier, if i’m the ‘expert’ and other people don’t really know what i’m talking about.” so, isolation, having to think small and do less, resistance from colleagues, and lack of authority. yeah, no wonder we feel a bit sad. what are my take-aways? we need to find our people. ux folks who worked with groups of colleagues were more positive about their work. however, people who tried to do ux work with non-ux committees were even more negative than people who had no group at all. so we can’t just look for any people, they have to be the right people. i wrote an article about the larger project that was published in weave earlier this month and in it, one of my recommendations was to try to move beyond the website. but i want to say here that moving beyond the web is not a panacea. i talked to someone who had great success in ux for the website and other digital projects. they wanted to embed ux throughout the library and they had management support to do it. but after continued resistance from colleagues, they realized they couldn’t make it work, and decided to move to a completely different area of the library. which brings me to my next point. advocacy is important, absolutely, but when we’re not getting buy-in, we need look at next steps: do we need to change our tactics? would it be better to have someone else advocate on our behalf? do we need to wait for a change of leadership? or, as a few participants said, a few retirements? at a certain point, do we give up, or do we get out? because advocacy doesn’t always work. and if it’s not working , we shouldn’t keep banging our heads against the post, right? ultimately , i think we need to be clear about authority. we need to understand how authority works in our own library. not just who can block us and who can help, but are there organizational structures that confer some authority? is it better to chair a committee or a working group? for example. then, we need a clear understanding of what our own authority is within our organization. maybe we underestimate the authority we have. maybe not. but we need to be clear before we get to the next part. which is: we need to clearly understand our own tolerance for doing work that will never be acted on. the report that sits in a drawer. if our tolerance is low, if it’s upsetting to have our work ignored, then we need to stick very closely to our own sphere of authority. we have to dream within that sphere or burn out. “dream small or burn out” is an exceptionally grim note to end on. but these frustrations are largely beyond one person’s control. if you’re feeling so very very sad because of some of these things, it’s not just you. the fact that these issues were common to web folks, regardless of how they seemed to feel about their work, suggests that these positions are prone to these kinds of frustrations. i wish i had some ideas for how to fix it! if you do, please add them to the chat, tweet at me, email me (see contact info). i’ll gather it all in a blog post so it’s all in one spot. thanks. sponsored post sponsored post menu more info learn from the experts: create a successful blog with our brand new coursethe wordpress.com blog october , ~ selena jackson ~ leave a comment wordpress.com is excited to announce our newest offering: a course just for beginning bloggers where you’ll learn everything you need to know about blogging from the most trusted experts in the industry. we have helped millions of blogs get up and running, we know what works, and we want you to to know everything we know. this course provides all the fundamental skills and inspiration you need to get your blog started, an interactive community forum, and content updated annually. register now the tpl debacle: values vs people october , ~ shelley ~ comment i can’t stop thinking about the situation at tpl. the short version is that the library has accepted a room rental from an anti-trans speaker, and despite outcry from trans people and their allies, despite a petition and a boycott by writers, despite their own policy on room rentals not allowing events that promote discrimination, they insist on letting the event proceed. some library associations are supporting them because librarians love being champions of intellectual freedom. many people have made cogent arguments about why tpl’s stance is wrong (see posts by fobazi ettarh, sam popowich, kris joseph). i agree. but there seemed to be more of a reason why the whole thing made me so sad. i’m writing because i think i’ve figured it out. in its two public statements on the matter, tpl has made sure to say that they “are supporters of the lgbtq s+ community.” they “are aware that the upcoming room rental event has caused anger and concern.” but the “community is asking us to censor someone because of the beliefs they hold and to restrict a group’s right to equitably access public space and we cannot do either. doing so would also weaken our ability to protect others’ rights to the same in the future.” fine. but they also said “while tpl encourages public debate and discussion about differing ideas, we also encourage those with opposing or conflicting viewpoints to respectfully challenge each other’s ideas and not the library’s democratic mandate to provide space for both.” that doesn’t sound super supportive. and at the board meeting held on october where the matter was discussed, it was clear they were more concerned with a respectful tone than with actually listening and understanding. reading how the trans women who spoke at that meeting felt about how they were treated was heartbreaking. so @torontolibrary is only letting of us into the room to speak or engage the board. they’ve moved most of our group into an overflow room. they had extra security guards and obviously have a plan for how they want to contain us. — runaway supernova (@gwenbenaway) october , i had to leave after i spoke. being forced to recount all of the transphobic violence that i face in daily life to the tpl board in a room of strangers and watch them stare silently at me as if i was subhuman was one of the worst experiences of my life. — runaway supernova (@gwenbenaway) october , i asked @vbowlestpl directly if she would say that trans women were women and she refused. i asked the entire board if they thought that i should use men’s restrooms and if they thought that would be safe. silence again. — runaway supernova (@gwenbenaway) october , no one on the @torontolibrary should serve this community, especially not @vbowlestpl , because regardless of their transphobic beliefs, they couldn’t even acknowledge my humanity in that moment. — runaway supernova (@gwenbenaway) october , i want to thank everyone that was there. i had to leave after i spoke, it was a lot to sit in front of people that barely seemed to care i was speaking. thank you all for your support and love both in the room and out of it. — niko stratis (@nikostratis) october , the tpl board and staff showed us their true colours and allegiances tonight. gwen was the only person to be called at time and that’s because she was asking them to treat her like a human. tonight was disgusting and dehumanizing. https://t.co/yauxcbk er — niko stratis (@nikostratis) october , the tpl board showed us who they are and who they support today. i feel awful, like i have never felt. that was dehumanizing, to throw my trauma out on a table to a sea of uncaring eyesx waiting to move to the next agenda item. i feel honestly sick to my stomach. https://t.co/lippmln rc — niko stratis (@nikostratis) october , it does not sound like these women were talking to “supporters” of their community. and that is what’s making me extra sad about the whole thing. not only is tpl choosing to value intellectual freedom more than they value trans people in their community, they are choosing to value intellectual freedom instead of valuing trans people in their community. it is not incompatible with upholding intellectual freedom to also acknowledge that it’s doing harm. tpl could reach out to the community and say “we know this event makes trans people feel unsafe. but we’re convinced that not allowing it to go forward will set a precedent for future decisions to shut down other events, possibly those that actively support trans people, and we cannot let that happen. we understand that this event will cause harm and undermine our relationships with lgbtq s+ people and your allies. what can we do to mitigate this harm?” it’s not as good as cancelling the event entirely, but at least it would show that tpl has been listening to its community. it would show that they have thought through the consequences of choosing values over people. it would show that they are not just “aware” of “anger and concern” but they understand the fears, risks, and harm their actions are causing. and of course, the community would have every right to tell them, no, there is nothing you can do to mitigate this harm. but that doesn’t mean tpl shouldn’t try. to not just say “we uphold intellectual freedom,” but to acknowledge exactly what that means in this particular case. i’m reminded of the saying that goes something like “your right to swing your arm ends when your fist meets my face.” tpl is insisting that they have the right to keep swinging. fine. but they have been told that their fist has already met the face of the trans community. the compassionate thing would be to offer first aid. but tpl is not interested. which, sadly, speaks volumes. it makes it crystal clear that they do not care about the trans community. it makes it crystal clear that they believe that the trans community and its allies are dispensable to their operations. the consequences of their decision (or, to be fair, their decision not to make a decision) are acceptable collateral damage; they are happy to make no attempt to mitigate any of it. if they really were supporters of the lgbtq s+ community, they would be supporting the lgbtq s+ community. in a way, it’s not surprising that the trans community is the group that so many librarians are choosing to not care about. being trans is simultaneously visible and invisible. a trans person may be visibly trans in that they do not present in the way that some might expect, but what makes them trans is inside them, not outside. what makes a person trans is in their heart and their mind. they know who they are *inside* in a way that cannot be seen by people who don’t know them (people who do know them can see how much happier they are when their outside gets closer to matching their inside). but to the outside eye, to the dispassionate eye, there is no evidence. and without evidence, their trans-ness can be seen as just a belief. and if it’s just a belief, well then, we can debate it. and we should debate it because, as librarians, we are champions of intellectual freedom. i so wish that we were champions of people instead. library workers and resilience: more than self-care october , ~ shelley ~ leave a comment an article in the globe and mail this spring about resilience was a breath of fresh air—no talk about “grit” or bootstraps or changing your own response to a situation. it was written by michael ungar, the canada research chair in child, family, and community resilience at dalhousie university and leader of the resilience research centre there. the research shows that what’s around us is much more important than what’s inside us when it comes to dealing with stress. the article was adapted from ungar’s book, the now-published change your world: the science of resilience and the true path to success. i know, the title is a little cringey. and honestly, some of the book veers into self-help-style prose even as it decries the self-help industry. but on the whole, there is quite a lot that it interesting here. i was looking at it for an upcoming project on help-seeking, but it keeps coming to mind during discussions about self-care and burnout among library workers. ungar writes of the myth of the “rugged individual” who can persevere through their own determination and strength of character. we get fed a lot of stories about rugged individuals, but ungar has found that when you look closely at them, what you find instead are “resourced individuals”—people who have support from the people and environment around them. “resilience is not a do-it-yourself endeavor. striving for personal transformation will not make us better when our families, workplaces, communities, health care providers, and governments provide us with insufficient care and support.” (p. ) ungar is mostly focused on youth but also writes about workplaces, even though this is not his direct area of research. two passages in particular caught my eye: “every serious look at workplace stress has found that when we try and influence workers’ problems in isolation, little change happens. … most telling, when individual solutions are promoted in workplaces where supervisors do not support their workers… resilience training may actually make matters worse, not better.” (p. ) a now-removed article in school library journal explained how one library worker changed herself to deal with her burnout. the reaction to this article was swift and strong. many of us know that individual stories of triumph over adversity are bullshit, particularly when we have seen those same efforts fail in our own contexts. i have found it validating to find research backs that up. ungar does allow that there are times when changing oneself can work—either a) when stress is manageable and we already have the resources (if you can afford to take two weeks off to go to a meditation retreat, why not), or b) when there is absolutely nothing else you can do to change your environment or circumstances (your job is terrible but you can’t leave it and you’ve tried to do what you can to improve things, so sure take some time to meditate at your desk to get you through your day). but most of us live somewhere between perfectly-resourced and completely hopeless. so what needs to be fixed is our environment, not ourselves. i have noticed resilience has been coming up as a theme in my own university over the last year or so—workshops on becoming more resilient or fostering resilient employees. ungar says “to be resilient is to find a place where we can be ourselves and be appreciated for the contributions that we make.” that’s not something individuals can do by themselves. people in leadership positions would do well to better understand the research behind resilience rather than the self-help inspired, grit-obsessed, bootstraps version. workshops and other initiatives that focus on individuals will not fix anything. at best, they are resources for people who are already doing pretty well. at worst, they add to the burden of people already struggling by making them feel like their struggles are caused by their own insufficiency. anyway, these are just some thoughts based on a single book; i’m nowhere in the realm of knowledgeable on this subject. but i thought it might be helpful to share that there is research that backs up the lived experience of the many library workers who struggle in their organizations, despite their own best efforts. research projects: call for help june , ~ shelley ~ leave a comment i’m on a year-long sabbatical as of july and excited to get started on a few different research projects. for two of the projects, i’m going to need some help from the uxlibs/libux community. in one of them, i want to look at love letters that users have written to academic libraries so i need people to send me love letters their users have written. in the other, i want to look at the different ways ux work is structured and supported in academic libraries so i need people who are willing to participate in an interview that will take around minutes. do you want to know more? read more about the love letters project. or, read more about the ux work project. i am happy to answer any and all questions: shelley.gullikson[at]carleton.ca or @shelley_gee on twitter, or in the comments below. thank you in advance for considering! and endless appreciation if you decide to help! uxlibsv: notes june , june , ~ shelley ~ comment five years of uxlibs – hurrah! let’s dive straight in. barriers to ux design: andy priestner andy kicked off the conference with his address about why he thinks not many of us are moving beyond research reports when it comes to doing ux work in our libraries: we see research as the finish line. ux is about uncovering actionable insights, not about statistical significance we’re terrible at idea generation. we tend to get set on the first “safe” idea we come up with. we pursue perfection. instead, we should evolve services with our users. we’re too cautious. after talking with library directors, andy thinks library staff perceive less agency than we actually have; directors say they want their staff to try new things. we’re not agile enough. not everyone needs to be consulted before we can take action. issues around ownership and politics. there is uncertainty about where ux sits and the scope is misunderstood. ignoring the basics. ux is often perceived as innovation (and institutions love innovation) but ux can also be sorting out the basics. fear of failure. we overreact to negative comments. failure is not modeled; we may hear that it’s okay to fail but we don’t tend to see it. andy then gave some examples of projects where libraries created prototypes out of their ux research, and iterated to improve the design to actually meet user needs. leadership is key—my ux journey: anneli friberg anneli gave a very warm and personal keynote, talking about her experiences growing ux at her library. one of the things that stood out most for me was her explanation of how “the user perspective” is different from “the user’s perspective.” library workers often feel they have “the user perspective” because they spend so much time serving users. but anneli said that this “user perspective” is only ever the best guess of library workers, looking from the inside-out. “the user’s perspective” is outside-in; we walk along with our users to learn what they actually do, say, and feel. it’s not a guess. anneli showed us her version of a ux maturity model (created in swedish and translated into english). she talked about the importance of recognizing what kind of organization you work in and where you are in the maturity model. she spoke about the frustrations she encountered when her library was in the early stages of maturity and how it helped her to have an external network she could rely on for support. to get through the frustration of the early stages of ux maturity, you have to shape the culture of your library. anneli recommended leading this culture change by example. michael west has said “the core of leadership is compassion and kindness” and lays out four aspects of leadership: attending, understanding, empathizing, and helping. he describes “attending” as “listening with fascination,” which i really like as an idea. a few other interesting bits from anneli’s keynote: failure is success in progress do idea generation together with your users take pictures of how students are using the library so you can easily show needs and gaps (e.g. a student hanging their coat on shelved books points to the need for coat hooks!) lead by clearing the path (help remove barriers for others) anneli had some interesting and useful things to say about failure. she believes that having a project fail was an important step in moving her ux vision forward. her team did some research, found a problem, and wanted to try a solution. anneli was pretty sure it wouldn’t work, but didn’t discourage them. they launched the solution and, sure enough, it didn’t work as well as they’d hoped. but having the experience of a failure, they were able to move on and try other things. they saw that failure wasn’t the end of the world, that the important thing was to try something, learn, and move on to try something else. neurodiversity, universal design and secrets of the library: penny andrews penny started her plenary talk by defining what neurodiversity is and is not. she then talked about how neurodiverse people experience the library. and often it’s not good. libraries have a lot of unwritten rules and unspoken social norms, and this is very challenging for neurodiverse students. library staff often don’t want to be the police so we expect users to manage the space themselves. but this usually relies on those unspoken social norms. clarity of the rules and enforcement of those rules would help neurodiverse students. silent study spaces can be difficult because they are never actually silent. it’s easier to hear things like people chewing and keyboards clacking in silent areas. but often, silent areas are where individual study spaces are found. having individual spaces in non-silent areas could be helpful. penny told us that most neurodiverse students do not ask for individual accommodations, or else wait until their situation is completely unbearable. autistic students are most likely to drop out within their first year. but if they continue, they tend to have the highest marks. so, what can libraries do? be upfront with our information (not hide it under “services for disabled students”). library websites have so much information and no good way into it. related, be specific with our communications. don’t just say “we’re here to help!” but make it clear how and why to make a one-on-one appointment. use universal design and consider various people’s needs from the start, not as an add-on. we can’t do one-size-fits-all because of competing needs, but our designs can account for these competing needs. don’t depend on disability services as a liaison. not all students declare their disabilities so disability services won’t know what those students need. recruiting can be difficult. talk to people in the library who look like they’re not having a good time. go to special interest groups that might draw neurodiverse people (penny recommended something geek-related). regular recruiting methods often bring out the outliers who always want to join in and who don’t represent the majority of neurodiverse people. always go in assuming we know nothing. a little bit of knowledge (knowing one neurodiverse person) is worse than knowing nothing. neurodiverse people are a diverse group. after penny’s presentation, someone asked her if there were certain ux research methods that neurodiverse people found difficult. penny responded that ambiguous prompts—particularly things like “draw your research experience” or “build your ideal library”—tend to be difficult, as is anything with group work. definitely good things to keep in mind. tales of the unexpected: hannah fogg and lorraine noel both speakers talked about the experiences of having front-line staff engage in ux work at their libraries. hannah started off with the experience at angela ruskin university (aru). at aru, they didn’t want ux to be just for librarians, so they brought in andy priestner to do ux training for their frontline staff. as part of the training, the staff did mini ux projects using their newfound knowledge of ux research methods. having “mini” projects was meant to not overwhelm some staff who might be scared off by a big project, and at the same time not give free rein to others who would be tempted to be too ambitious. one of the projects hannah highlighted was a mapping exercise that showed users completely avoiding the print journals shelving (they diverged to one side or the other), so a decision was made to move those shelves out of that area of the library entirely. lorraine was up next to talk about the experience at huddersfield. they had seen what aru had done and wanted to replicate it, in hopes of professionalizing their front-line staff and enhancing the user experience. bryony ramsden led the workshops for huddersfield staff. attendance was mandatory and they also had to work in groups on a “modest ux project.” those groups had to include staff from at least two different areas of the library (i love that idea!), and each of the groups had a manager as a “guide on the side.” there were a lot of benefits to the huddersfield experience, but lorraine also mentioned that there was some initial resentment from staff, likely due to the mandatory nature of the project. hannah said that at aru, staff appreciated learning skills in project management that could help with their career progression. also, aru lost their ux expert and staff were happy to feel empowered to carry on the ux work on their own. passionate about floorplans: tim graves (i was excited about this session because floorplans are the bane of my existence. we get a lot of requests to make them fancier or add functionality, but keeping them up to date is a constant struggle. i finally resigned myself to walking through our floors three times a year, making any necessary corrections on printed versions of our maps so i can update the ones on the web. the maps posted in our building get updated by the campus facilities people and at times bear little resemblance to the web versions. argh!) anyway, tim also wanted to improve the floorplans on the website of the university of sussex. the library was receiving a lot of questions about how to find things in the library and tim thought that better floorplans on the website might help people better navigate to what they needed. first, he came up with a version based on printed floorplans, but they were too complex and not responsive on smaller screens. inspired by the london tube map, he created a simplified version, but discovered it was *too* abstracted from reality to be useful. the “just right” solution came after he did a lot of reading in the design literature (especially alberto savoia and jeanne liedtka & tim ogilvie) and started iterating his design with users. tim mentioned the usefulness of “pretotyping” a solution to see if it’s worth getting to the prototyping stage. a pretotype is essentially a very rough, low-fi prototype. it might be a good thing to keep in mind if you work with people who find it difficult to create quick and dirty prototypes. you could say “we don’t need a prototype yet, let’s just pretotype it!” even though *you* know a prototype can just be a rough sketch, they can think it’s a whole different (and new!) thing. you can see tim’s improved floorplans. and he said that he’s happy to share the code that drives them. you can contact tim at t.c.graves[at]sussex.ac.uk. appreciative inquiry workshop: kristin meyer appreciative inquiry is a method that helps people focus on solutions instead of problems, leads groups to action, and does so in a very positive way. i was really excited about this workshop because anything that kristin does always seems excellent. i was not disappointed. the workshop started with an introduction to appreciative inquiry and then kristin led us through a sped-up process of appreciate inquiry as we worked through an issue that’s been raised through ux research at her own library. the steps we took: connect to purpose: look at the big picture and why this problem is important. how could exploring this area benefit users? frame it and flip it: clearly state the problem so that everyone is on the same page. then, think about the desired state instead of the problem and come up with a question to help us explore what we desire for our users. dream of the ideal future: think about words and phrases that describe an ideal solution. how will success look and feel? ideate: we skipped this step in the workshop because it takes a lot of time. kristin mentioned her favourite ideation technique is brainwriting, described in the book gamestorming ( ). prototype internally: our colleagues may have good ideas and asking them for feedback can help build early buy-in. generative questions keep things positive: what do you like about this idea? how can we improve this idea? prototype with users: again, we skipped this step because we had no users to prototype with. i liked step , where we flipped the problem into a desired state. i’m guessing that thinking of “what do we want to happen” instead of “what do we want to stop” could help avoid the “solution” of putting up a sign or trying to curb behaviour with punitive measures. i also really like the idea of connecting to colleagues with generative questions, rather than asking for general feedback. andy may have said that not everyone needs to be consulted, but sometimes it’s important or useful to consult our colleagues. using generative questions would be a way to lessen the chances of hearing “that will never work” or “why don’t you do x instead?” advanced interview techniques: danielle cooper since i’m about to embark on a project that involves a lot of interviewing, i thought it made sense to make sure that i took advantage of any opportunity to improve my skills in this area. danielle has a lot of experience with interviewing users in her job at ithaka s+r. the short version of this workshop is that the best way to get better at interviewing is to keep doing it, so we spent most of the time in groups of taking turns being interviewer, interviewee, and observer. danielle gave us some practical tips as well. to probe for more information, from least obtrusive to most: silence non-verbal affirmation echoing the response affirmative neutral comments repeating or clarifying the interview question summarize and synthesize the answer simply saying “tell me more” if participants are not very forthcoming, you can try a variety of these probes. be willing to cut your losses and end the interview if you’re not getting any useful information. on the other hand, if participants are way too chatty, you can try the following: gentle inattention polite transitions graceful interruptions working in difficult environments: lessons from the world of civic design: suzanne chapman suzanne started her keynote with some examples of behaviour that many of us recognized from our workplaces. she then pointed out that these behaviours were from the simple sabotage field manual from the oss (predecessor to the cia), a document explaining to spies how to sabotage enemy organizations. she gave a quotation from a senior person in one of the organizations she’d worked in: “we are trying to do as much end user testing as possible without actually talking to users.” suzanne noted that ux maturity models, such as the one anneli showed in her keynote, are missing the part where humans are difficult and sabotage-y. she also noted that doing ux in libraries is extremely hard. but this work can be made easier if everyone can agree on specific guiding principles. she shared seven that she uses at the centre for civic design: do the most good for the most people who need it the most (italics mine). this goes beyond the / rule and looks at need rather than just numbers. delivery is the strategy. given the choice between culture change and “getting shit done,” they have chosen to let culture change come second. work lean, iterate quickly. sometimes this means doing the least shitty thing, but it always means that you should only make *new* mistakes. we use design to make things better. design means working your way through the problem in order to reach a solution, not just grabbing a solution. we design with users, not for them. this is similar to anneli’s message to take the “user’s perspective” rather than the “user perspective.” also, research is done with a goal of improvement, not just for learning. hire and empower great people. and there has to be agreement about what it means to be empowered; there should not be responsibility without authority. these principles may not resonate, or even be possible in your library. but going through the process of deciding what your library’s guiding principles are can be your anti-sabotage model. my web committee went through this process, based on guiding principles suzanne wrote while she was still working in libraries. the process was very helpful in making sure we really were on the same page. it’s also a useful document to show people coming on to the committee for the first time. it would definitely be *more* useful if it went beyond just our committee, but it’s something. if you’re interested, we’ve made our guiding principles public. uxvr: the potential of virtual reality to ux research: victor alfson victor spoke about a project he did at the black mountain library in stockholm. he asked users to create a great library for themselves using a vr headset, tilt brush (a d-painting app), and a d model of the existing library. he asked participants to narrate their actions, but also jumped in with questions. it’s a similar task to what you could do with pen and paper, but using vr gave a different angle. to recruit participants, victor asked the (possibly slighty creepy) question, “do you want to come down to the basement to try something cool?” / people that he asked agreed to participate! and once they were there, they stayed—for minutes on average— because the task was novel and engaging. victor found that participants were very candid in what they said, and he wondered if that was due to people feeling like they were in a private space. with the vr headset on, they were alone in the d library space, with victor’s disembodied voice occasionally asking them questions. so what did users draw and talk about? well, it was the usual things: food, noise, finding the right kind of space. but the insights were interesting. a few kids drew a mcdonalds in the library, and went on to say that they just wanted to be able to eat their snack without a librarian bugging them. one kid drew a vortex in the library that would take them directly to their home. victor asked further about this and found out that this kid had to take two buses and the metro to get home from the library. i wondered if this kind of thing would have come out in a pen-and-paper exercise, or if it was the technology that made the kid think about an amazing technological solution to their transportation problem. overall, victor said that it was very fun research for both him and the participants. and his library will be following up on some of the insights they gained, such as creating a new quiet study room for kids working on their homework. previously, these kids tried to find quiet nooks and crannies to work in, so both they and their needs were unseen by library staff. victor’s project brought them out of their quiet corners and gave them a new space of their own. a nice real-world result for this vr project. internships and ethnography: students researching students: claire browne claire spoke about using a student intern to carry out a ux project using a cultural probe to get to know the needs of taught postgraduate students at the university of birmingham. the university’s careers department was looking for meaningful student placements that showcased careers in higher education and gave students experience with project management and data analysis. it was a great fit with the library’s desire to expand their ux work. before the intern was hired, the library had to have the project go through ethics review and recruit participants ( in total). they had ideas for what they wanted in the cultural probe, but the intern, luke, was able to put his stamp on it, finalizing the tasks and adding notes and jokes to the participant diaries to keep their engagement up throughout the weeks of daily tasks. some of the tasks were: answering specific questions, writing a letter with advice to a student starting out, card sorting, a photo study showing their typical day, a love letter/break-up letter, and a cognitive map. all participants did every task, which seems to show that luke did a great job keeping everyone engaged. participants enjoyed the variety of tasks and provided a lot of rich information in the self-reflective tasks. luke gave a presentation to senior staff about his findings and they were very engaged with this year old telling them about the problems in their library. i want to know more about this; were they more engaged because he was an “outsider,” because he was a student, because he was young? related, claire mentioned that one of the benefits of having a student intern on this project was that he was not influenced by restraints or constraints felt by library staff; he saw only the user side. another benefit claire mentioned was that luke was able to engage with the student participants in a natural and informal way that she didn’t think would be possible for librarians. she thought the librarians would have been too formal or crossed the line into “cringey.” if you want to know more, luke wrote a report about the project and the techniques that were used in the cultural probe. love at first sight: consolidating first impressions: debbie phillips debbie also spoke about doing a cultural probe, this time at royal holloway and focused on the experience of new students in their first weeks on campus. the focus was not entirely on the library, as the project was a collaboration among the library, campus life, and internal communications. the campus life team were able to help with recruitment and students agreed to participate, though only actually finished all the tasks. still, since they were hoping for participants, this was a good result. i was struck that, like claire, debbie said they were “hoping for a good mix” of participants. both projects got a reasonable mix but missed out on representation from one or two groups. i think we often do generic recruitment when we want a mix, assuming that we should recruit from a wide group to get a wide range of participants. but if we want, for example, mature students or international students as part of the participant group, we really need to recruit them specifically in order to make sure of it. (i believe claire did make this point as something they would do differently next time.) some of the tasks in the cultural probe at royal holloway: diary questions ( questions from each of the teams plus some general ones), photo tasks, postcard to friends/family (participants could ask for it to be posted but no one did), campus map with emoji stickers to indicate how they felt about specific buildings or areas of campus. the library found they were surprised at how many students came to the library during their first visit to campus. they were also surprised at how few students attended their library induction. so, they’re planning to try to find ways to help students learn more about the library during that first campus visit, rather than waiting for induction. related, they also found that students expressed a preference for learning about campus prior to arrival, so the library will increase their communications ahead of arrivals week, rather than waiting until students are actually on campus. final thoughts i usually do a full post about my thoughts on the conference, but i don’t have a lot more to say. i had an amazing time, as usual, thanks to the wonderful group of people who come to this conference. in my professional life, uxlibs is my very favourite place. i’m about to head off on sabbatical (maybe you can help with some of my projects!), so i’m not going to immediately apply much of what i learned but i am already excited to do that when my leave is over. i realize that i’ve been emphasizing the research part of ux because research is actually part of my job description and, outside of the website, design and prototyping is not. i felt comfortable doing research beyond the scope of the website, but not finding a way to move that research into action. when i get back to work i hope i can figure out how to, as both keynotes exhorted: get shit done. website refresh: first round of iterative testing december , december , ~ shelley ~ leave a comment as i mentioned in my last post, we’re doing a design refresh of our library website, with a goal to make it “beautiful.” as such, we’re not touching much of the organization. but of course we have to pay attention to not just how the information is categorized but also where it appears on the page. we learned that a few years back when we tried adding a “spotlight” feature near our library hours (tl;dr: people stopped being able to see the hours when other content shared the space). so we are firm believers that user testing and iterative design is vital in making sure we don’t make parts of our site invisible by moving elements around. after the results of our user research earlier in the fall, we came up with a design drawn from the sites that our users liked most that also worked within our current site structure. the layout was essentially the same, with three major changes: we pulled “quick links” out of the menu and put it in a box on the front page hours moved from a box on the side to a banner under the search box our help and chat button also moved to this banner we wanted to do user testing to make sure that users could: find today’s hours get to the full set of hours figure out how to access help or chat. we also asked them if there was anything they hated about the draft design. just to flag anything that could cause problems but that we weren’t specifically asking about. since we were doing this testing early in the process, we didn’t have a live site to show. our web developer, the fabulous kevin bowrin, built the mockup in drupal since he’s more comfortable in drupal than in photoshop, but it wasn’t on a public server. so we used a printed screenshot for this round of testing. the first version of the design had a grey banner and small text and it was clear after talking to a few users that visibility was a problem. we only talked to people, but only saw the hours and they were really squinting to make it out. finding when the library is open should be really really easy. we decided to increase the text size and remove the grey background. version this time, even fewer people saw the hours: out . since people didn’t see today’s hours, we couldn’t even get to the part where we tested whether they knew how to access the full set of hours. we decided to see if adding an “all hours →” link would help; perhaps by echoing the convention of the “view more →” links in other parts of the page, it would be clearer that this section was part of the content. nope. version again, quite quickly we saw that this section remained invisible. only person in saw it. one user noticed it later on and said that he’d thought that part of the website was just a heading so he ignored it. clearly, something was making people’s eyes just skip over this part of the website. we needed another approach. kevin and i talked about a few options. we decided to try making the section more visible by having library hours, help and chat, and quick links all there. kevin tweeted at me after i’d left for the day: “just dropped the latest iteration on your desk. i kinda hate it, but we’ll see what the patrons have to say!” i had a look the next morning. i also hated it. no point in even testing that one! a blurry photo of the hated, not-tested version we decided to put hours where the quick links box was, to see if that would be more visible. we moved chat down, trying to mimic the chat call-out button on the mcmaster library website. quick links were removed completely. we have some ideas, but they were never a vital part of the site so we can play with them later. success! most of the people we talked to saw the hours and almost all of them could get from there to the full set of hours. (i did this round of testing without a note-taker, thinking i could keep good enough track. “good enough?” yes. actual numbers? no.) the downside was that most people didn’t notice the help and chat link (not pictured here). however, i think we’ll really need to test that when we can show the site on a screen that people can interact with. the “always visible” nature of that button is hard to replicate with a print-out. i feel like we’re in a good enough place that we can start building this as more than just a mock-up. oh, and no one we talked to hated anything about the design. a low bar perhaps, but i’m happy that we cleared it. version we did all of this in one week, over afternoons. for version , kevin just added text to the screenshot so we could get it in front of people faster. quick iterating and testing is such a great process if you can make it work. next steps: menu interactions and interior pages. user research: beautiful websites? october , ~ shelley ~ comments my university librarian has asked for a refresh of the library website. he is primarily concerned with the visual design; although he thinks the site meets the practical needs of our users, he would like it to be “beautiful” as well. eep! i’m not a visual designer. i was a little unsure how to even begin. i decided to attack this the way we attack other problems: user research! web committee created a set of guiding principles a few years back (based on suzanne chapman’s document). number one in that list is “start with user needs & build in assessment” so even though i was having difficulty wrapping my head around a beautiful website as a user need, it made sense to move forward as if it were. background how does one assess a beautiful website? i looked at a whole bunch of library websites to see which stood out as particularly beautiful and then discern what it was that made them so. let me tell you, “beautiful” is not a word that immediately leaps to mind when i look at library websites. but then i came across one site that made me give a little exclamation of disgust (no, i won’t tell you which one). it was busy, the colours clashed garishly, and it made me want to click away instantly—ugh! well. we might not be able to design a site that people find beautiful but surely we can design something that doesn’t make people feel disgusted. i had an idea then to show users a few different websites and ask them how they felt about the sites. beauty can mean different things to different people, but it does conjure a positive feeling. coming up with feeling words can be difficult for people, so i thought it might be easier for me to come up with a list they could choose from (overwhelming, calm, inspiring, boring, etc.). then i decided that it might be better to have users place the sites on a continuum rather than pick a single word for their feeling: is the page more calming or more stressful? is it more clear or more confusing? i came up with feelings described on a continuum, plus an overall 🙂 to 🙁. i wasn’t completely confident about this and assumed others had done work in this area, so i did some reading on emotions, aesthetics, and web design. (emotion and website design from the encyclopedia of human-computer interaction, nd ed.; aesthetics and preferences of web pages from behaviour & information technology ( ); assessing dimensions of perceived visual aesthetics of web sites from international journal of human-computer studies ( ); and measuring aesthetic emotions: a review of the literature and a new assessment tool from plos one ( ).) turns out my method was in line with the research in this area. and although the wording sometimes differed, the feelings i had come up with were all represented. onward! there had been some talk of the library website perhaps needing to mirror other carleton university websites a little more closely. however, there is not uniformity of design across carleton sites, so i wanted to show users a mix of those sites to get a sense of which designs were most pleasing. i also wanted to show a few different library sites to get a sense of which of those designs were most appealing to our users. i worked with web committee to come up with a list of library sites and carleton sites. there was no way i was going to ask someone to give us feedback on different websites; i decided a selection of was plenty for one person to work through. since i was looking mostly for visceral reactions, i didn’t think we needed a lot of people to see each site. if each site was viewed times (with our own library site as a baseline so we could measure improvement of the new design), we needed participants. that was three times what we often see for a single round of ux research, but still doable. method i planned a -minute process—longer than our usual processes where we test one or two things—and wanted to compensate students for this much of their time. that fell apart at the last minute and all i had was a box of halloween mini-chocolates so revamped the process to remove a few pre- and post- questions and cut the number of continuums from to ( feelings plus the overall positive/negative). that cut the time down to about minutes for most people, and i was comfortable with a -minutes-for-chocolate deal. so in the end, these are the continuums we asked people to use to label the sites: welcoming ↔ off-putting disorganized ↔ organized clear ↔ confusing up-to-date ↔ old-fashioned calming ↔ stressful useful ↔ useless inspiring ↔ discouraging ugly ↔ beautiful 🙂 ↔ 🙁 we set up in the lobby of the library and saw people over four time slots (each was - minutes long). there were participants instead of because the last person came with a friend who also wanted to participate. happily, the only person to have difficulty understanding what to do was one of these very last people we saw. he had such trouble that if he’d been the first person we’d seen, i likely would have reconsidered the whole exercise. but thankfully everyone else was quick to understand what we wanted. most people saw one carleton site, one library site, and then our own carleton library site. because we had more library sites than carleton sites, a few people saw two library sites then the carleton library site. i had planned out in advance which participant would see which sites, making sure that each site would be seen the same number of times and not always in the same order. participants looked at one site at a time on a tablet with a landscape orientation, so the sites looked similar to how they would look on a laptop. they filled out the continuum sheet for one site before looking at the next. they could refer back to the site as they completed the sheet. i had a note-taker on hand to keep track of the sites visited and to record any comments participants made about the sites (most people didn’t say much at all). partway through, i discovered a problem with the “up-to-date / old-fashioned” continuum. i was trying to get at whether the design felt old and stale or contemporary and up-to-date. but many people assumed we were referring to the information on the site being up-to-date. i thought that using “old-fashioned” rather than “outdated” would mitigate this, but no. so this was not a useful data point. usually with these kinds of processes, i have a sense of what we’re learning as we go. but with this one, i had very little idea until i started the analysis. so what did we find? results i had purposely not used a likert-type scale with numbers or labels on any of the mid-points. this was not quantitative research and i didn’t want users to try to put a number on their feelings. so, when it came time for analysis, i didn’t want to turn the continuum ratings into numbers either. i colour-coded the responses, with dark green corresponding to one end of the continuum, red to the other and yellow for the middle. i used light green and orange for less strong feelings that were still clearly on one side or the other. in determining what colour to code a mark, i looked at how the person had responded to all three sites. if all their marks were near the extremes, i used light green/orange for any mark tending toward the middle. if all their marks were clustered around the middle, i looked for their outer ranges and coded those as dark green/red (see examples in the image below). in this way, the coding reflected the relative feelings of each person rather than sticking to strict borders. two marks in the same place on the continuum could be coded differently, depending on how that user had responded overall. the circled mark on the left was coded light green even though it’s quite close to the end. the circled mark on the right was coded red even though it’s not very close to the end. after coding, i looked at the results for the 🙂 ↔ 🙁 continuum to get a sense of the general feeling about each site. i gave them all an overall assessment (bad, ugh, meh, or ok). no site got better than ok because none was rated in the green by everyone who saw it. then i looked at how often each was coded green, yellow, and red across all the continuums. unsurprisingly, those results corresponded to my bad/ugh/meh/ok rating; participants’ 🙂 / 🙁 ratings had been reflective of their overall feelings. our site ended up on the high end of “meh.” however, several participants made sure to say their ratings of our site were likely high because of familiarity, so we are really likely firmly in “meh” territory. now that i’d looked at the overall, i wanted to look at each of the continuums. what was our current site doing really well with? i was happy to see that our current site felt useful and organized to participants. “organized” is good because it means that i feel confident about keeping the structure of the site while we change the visual design. what did we need to improve? participants felt the site was discouraging and ugly. “discouraging” is something i definitely feel motivated to fix! and “ugly?” well, it helps me feel better about this project to make the site beautiful. more beautiful at least. after this, i looked at which sites did well on the aspects we needed to improve. for both the carleton sites and the library sites, the ones felt to be most inspiring and beautiful were the same ones that were rated highly overall. these same sites were most felt to be welcoming, clear, and calming. so these are the aspects that we’ll concentrate on most as we move through our design refresh. next steps now, web committee will take a closer look at the two library sites and two carleton sites that had the best feeling and see what specific aspects of those sites we’d like to borrow from. there’s no big time squeeze, as we’re aiming for a spring launch. lots of time for many design-and-test iterations. i’ll report back as we move forward. access : a ux perspective october , ~ shelley ~ leave a comment i started my access conference experience with a meetup of library people interested in ux. there were only five of us, but we had good conversations about research ethics boards and ux research, about being a ux team of one, and about some of the projects we’ll be working on in the coming year. we also chatted about how we would like to communicate more regularly but how difficult it can be to sustain virtual communities. (canada is big. heck, even ontario is big.) it was nice to start off the conference with ux friends – old and new – and my focus stayed on the ux side of things throughout the conference so that’s what i want to write about here. on day , the first post-keynote presentation was all about ux. eka grguric talked about her experience one year in as ux librarian at mcgill. she gets brought into projects in her library as a ux consultant, and also supports others doing ux and user research in the library. she also offers training on ux research methods for interested library staff. her work is a combination of operational and project-based. she gave a bit of detail about two projects and her monthly operational tests to give us a flavour of the range of methods and processes she uses. next up was ken fujiuchi and joseph riggie from buffalo state college, who talked about extended reality, a combination of virtual reality, augmented reality, and mixed reality technologies. they covered a few different topics (slides here), but what stood out for me was their mention of how user experiences will change as new interfaces become possible and there are new ways for people to interact with materials. they specifically mentioned oral histories moving from audio-only files to users being able to interact with a holographic image of a person who can tell stories but also answer questions. what’s good ux for oral history holograms? a few presentations also focused on what i see as ux for library staff. juan denzer spoke about a project being developed by a pair of students he’s supervising that aims to make it easier to manage exproxy configuration files (which can easily run to thousands of lines). having tried to troubleshoot stanzas in exproxy myself, i can definitely see how this could improve the ux for staff. however, as one of my table mates said, adding an application to manage a text file also adds overhead for whoever has to maintain and update that application. trade-offs! ruby warren from university of manitoba was fantastic in her description of a project that didn’t quite get off the ground in the six months she’d set aside to complete it. ruby had seen that distance students weren’t learning how to use the library in the same way in-person students were (e.g. no in-class visits from a librarian). she wanted to find a way to teach some basic il to these students and thought that an interactive fiction game would be a good thing to try. she had some great lessons learned (including “don’t do everything in the wrong order” and “plan for apathy”). one of my favourite things about ruby’s presentation was that she was upfront about her failures, including – as a ux person – not planning for user testing during development. it’s gutsy to get up in front of your peers and say that you forgot a basic tenet of your discipline because you were too excited about a project. so human but so hard. yay ruby! another key takeaway was not underestimating appeal when planning this kind of project. as someone who has a bard time seeing the appeal of library games, i appreciated hearing this. (i believe it’s possible, but i think it’s extremely difficult.) ruby’s slides are here. back to ux for staff (and users too, to some extent), calvin mah from simon fraser university spoke about his experience archiving their ils when his library moved from millennium to alma. some kinds of information were not migrated at all, but even the records that were migrated were not trusted by cataloguers; they wanted to be able to go back to the old records and compare. with these two situations – missing information plus untrusted information – it was decided to build an archival copy of the old system. i find this interesting. on the one hand, i can absolutely understand wanting to help staff feel comfortable with the new system by letting them know they still have the old information if they need it; the transition can be more gradual. but calvin noted that even though the information is getting stale, staff are still relying on it. so perhaps it’s more of a security blanket, and that’s not good. also, there was a good library nerd laugh when he said that some staff wanted the archival copy to behave like the old system: “respect the nd indicator non-filing characters skip!” something i see as having both staff and user ux implications is having contract work in library systems (probably everywhere, but in systems for sure). bobbi fox from harvard has been on many sides of this situation (as a contractor, as a person hiring the contractor, as a team member, as a person cleaning up after a contractor) and detailed many things to consider before, during, and after contract work in library it. too often, contract work results in projects that are difficult to maintain after the contractor has gone, if they are even completed at all. i really like that she specifically mentioned thinking about who is providing user support for the thing(s) the contractor is building, as separate from who is going to own/maintain the project going forward. and in talking about documentation, specifying what documentation those user support people need in order to be able to support the users. this will almost always be different documentation that what is required for maintenance. good docs are vital for maintenance but if people can’t use the thing, there’s not much point in maintaining it! nearing the end of the first day was a panel: “when the digital divides us: reconciling emerging and emerged technologies in libraries” that looked at disconnects that can happen on both the staff side and the user side when libraries favour emerging (“shiny”) technology. i thought there were some great points made. monica rettig at brock university talked about issues when access services staff are expected to help troubleshoot technology problems; for staff used to a transactional approach to service, with a heavy reliance on policy and procedures, there is a big cultural shift in moving to a troubleshooting approach. rebecca laroque from north bay public library wondered about providing d printers while she still has users asking for classes on how to use email. monica noted the importance of core services to users even though they’re aren’t shiny or new; she asked who will be the champion for bathrooms or printers in the library? krista godfrey from memorial university asked whether library technology should be evaluate and assessed in the same way that library collections are? lots of questions here, but definitely an agreement that a focus on core infrastructure and services may not be exciting but it’s absolutely vital. day was a bit lighter on the ux side. tim ribaric gave a great presentation on ra and the possible implications of it replacing ip authentication for access to electronic resources in libraries. tim is skeptical about ra and believes it is not good news for libraries (one of his theorems about ra : “we are effed”). his take was very compelling, and from a ux perspective, he is not convinced there is a clear way forward for walk-in users of academic libraries (i.e. users not affiliated with the university or college) to access our subscription-based electronic resources if we move from ip authentication to ra . i know some academic libraries explicitly exclude walk-in users, but others are mandated to provide access to the general public so we are used to providing guest access and our users are used to having it. tim has posted his slides if you’re interested in more on this. another interesting ux moment was in autumn mayes’ lightning talk about working in digital scholarship and digital humanities. part of her job had been working in the humanities interdisciplinary collaboration (thinc) lab at the university of guelph. thinc lab is a members-only space aimed at grad students, postdocs, faculty, etc. who are doing interdisciplinary and digital humanities research. however, they also host events and programs that are open to the larger university population. so autumn found herself having to tell non-members that they weren’t allowed to use the space, but at the same time was trying to promote events and programs to both members and non-members. she very succinctly described this as “get out! but come back!” it’s interesting to think about spaces that are alternately exclusionary and open; what is the impact on users when you make a mostly exclusionary space occasionally welcoming? what about when a mostly welcoming space is occasionally exclusionary? bill jones and ben rawlins from suny geneseo spoke about their tool oasis (openly available sources integrated search), aimed at improving the discovery of open educational resources (oer) for faculty at their campus and beyond. the tool allows searching and browsing of a curated collection of oer (currently over , records). it seems like a nice way to increase visibility and improve the ux of finding oer such as open textbooks. again in library staff ux, may yan and mj suhonos from ryerson university talked about how library-specific technologies can be difficult to use and adapt, so they decided to use wordpress as a web platform for a records management project in their library. one thing i found interesting was that the ryerson library had a strategic systems requirements review that explicitly says that unless library-specific technology has a big value-add, the preference should be to go outside library technology for solutions. from a ux point of view, this could mean that staff spend less time fighting with clunky library software, both using it and maintaining it. the last conference presentation of day reported on the results of ux testing of open badges in an institutional repository. christie hurrell from the university of calgary reported that her institution uses quite a number of open badges. for this project, the team wondered whether having an open badge that demonstrated compliance with an open access policy would encourage faculty to deposit their work in the institutional repository. they did a survey, which didn’t show a lot of love for open badges in general. then they did some user testing of their ir (dspace), to find out whether faculty would add an open badge to their work if the option was there. unfortunately, the option to add an open badge was completely lost in the overall process to deposit a work in the ir, which faculty found extremely time-consuming. since faculty were frustrated with the process in general, it is very unlikely that an open badge would provide an incentive to use the ir again. the conference ended with the dave binkley memorial lecture, given this year by monique woroniak. monique spoke about “doing the work: settle libraries and responsibilities in a time of occupation” where the work is what non-indigenous people and organizations need to do before trying to work with indigenous people and organizations. she gave some clear guidelines on, essentially, how to act with empathy and these guidelines can apply to many communities. however, i definitely don’t want to “all lives matter” this. monique was clearly speaking about indigenous people, and specifically about her experiences with indigenous people in winnipeg. when she spoke of the importance of assessing our capacity before undertaking new work, she included the capacity to build respectful relationships with indigenous people. although it can definitely be argued that a capacity to build respectful relationships is useful for ux work, her caution to never over-promise and under-deliver when working with indigenous people is situated in the canadian context of settlers over-promising and under-delivering time and time and time again. sure, we’ll respect this treaty. sure, we’ll take care of your children. of course we’re ready for reconciliation. over-promising and under-delivering is never a great move, but in this context it is particularly toxic. a few other things that stood out for me in monique’s talk: listen to the breadth of opinions in the community. take the time. this is head work and heart work, and, especially, long-haul work. look to shift the centre of power for not just the big decisions, but the small as well. if this interest you, monique’s talk is available to view in its entirety, as are all of the presentations at the conference (they will be split into individual videos for each talk eventually). monique finished with a lovely quotation from katherena vermette‘s poem “new year’s eve ” from her book river woman: truth is a seed planted deep if you want to get it you have to dig ux from a technical services point of view september , ~ shelley ~ comment this the text of a presentation i did last year at the access conference in regina. emma and i had plans to write this up as a paper, but life intervened and that didn’t happen. i wanted to keep some record beyond the video of the presentation, so here it is. this morning i’m going to talk about a user research project i did with my colleague emma cross. we observed the user experience of students doing academic research online and then looked at that ux from the perspective of technical services staff. i’ll start with talking about the research we did with the students, the results from that research that seemed most relevant to technical services staff, and then i’ll talk a bit about the reaction that technical services staff at carleton had to those results. i’m sorry that emma can’t be here. she was the technical services brain behind it all; i was the user research monkey. so, why did we want to do this? mixing technical services and user experience is not done very often. a technical services supervisor at carleton told us she was finding it difficult to prioritize work for her staff, and she wanted some insight into what was likely to have the most impact for our users. fantastic. methodology emma and i designed the research to be student-led; we didn’t have specific questions but we were interested in where students searched, how they searched, and what kinds of things they looked at in their results. in our sessions, we asked students to search for something they needed for a research assignment, and to try as much as possible to do what they would normally do, not what they thought they “should” do. we emphasized that even though we were from the library, they didn’t have to use library tools or resources if they normally wouldn’t for the kinds of searches they were doing. i moderated the sessions, asking the students to think aloud throughout their searches, prompting them with questions if they were quiet. we let them search until they seemed to finish but let them know when we neared minutes. the sessions lasted anywhere from - minutes, but most were - minutes. emma took notes and we also captured the sessions on video, so we were able to go back and fill in gaps when people worked too quickly for emma to capture everything. we did the research in march of and saw undergraduate and graduate students. emma coded the results and found themes that she thought were most relevant to technical services result # : overwhelming use of the single search box summon and/or google scholar were used by most of the students, and the catalogue not much at all. people used various specialized databases and there was also regular google, wikipedia, tumblr, but summon and google scholar were really the most used. there was little difference between grad and undergrad use of tools, except for catalogue use. the people who used the catalogue were undergraduates. kinda weird. but this is a good time to emphasize that this was a qualitative study, not a quantitative one; we’re not going to extrapolate that % of undergrads use the catalogue and grad students do. the numbers don’t matter – it’s that when observing how students search and listening to how they approach looking for information, the library catalogue doesn’t often come up. it’s not part of their process. result # : popularity of the “get it” button emma’s second theme is the logical corollary to the overwhelming use of single search: the popularity of the “get it” button and the link resolver in general. i love the “get it” link – it makes my life much easier. (graduate student) “get it” is really useful (graduate student) “get it” is helpful!” (undergraduate) hey look carleton offers to “get this” in google scholar – hey that is great! (undergraduate) even when students didn’t mention it explicitly, they used it seamlessly. maybe that seems obvious, but i have seen user research results from other university libraries where students had a hard time understanding their get it links. our students got “get it.” result # : metadata looked at: title, date, abstract; metadata searched: keyword, keyword, keyword a pattern we saw repeated over and over in student research was: scanning search results list quickly reviewing title for relevant keywords check the date – majority of student not interested in old material if interested, click on record to read the abstract if title, date, abstract check out then download / print for further reading. students seem to be so used to this pattern and used to seeing abstracts or snippets of content that when they don’t see an abstract, usually when they are looking at a monograph record, they’re confused and then they move on. and although students look at different metadata fields, they rarely search them. aside from a couple of author searches and one really heartbreaking subject search, most of the searches we saw were keyword, keyword, keyword. result # : speed, impatience and ease of access students quickly skimmed results lists and rarely went beyond the first page of results (or with summon’s infinite scroll, the first or so). undergrads tended to look at fewer results than grad students. many students had no qualms saying they were busy and they didn’t want to waste time. there was a general tendency to skip over materials that were harder to access – things on reserve, in storage, or borrowed, documents that take a long time to download. even when they did pursue these harder to access items, they weren’t necessarily happy about it. this is probably emma’s favourite quote: this is useful if i can find it. it is not online so i will have to search the library itself. this makes me cry a little. generally, the students we saw were easily able to find other things that seemed just as good, so skipping over hard-to-access items didn’t seem to create much of a problem. reaction from technical services staff so these were the findings we thought were most relevant to technical services staff. there are no big surprises here, but we wanted to know how our own technical services staff would react to what we’d found. what would they take from our results? in july, we gave a presentation for library technical services staff, followed by a discussion. here are some of the first comments from staff, to give you a flavor: “on the library website, we now have a summon search box instead of a catalogue search box, and maybe that’s why catalogue use was low.” “are users even aware of the catalogue?” “students don’t seem to be aware of subject headings. they should be taught about the catalogue and how to do subject searches.” “maybe all first years could be given a booklet about how to search properly.” so that was sort of the tone at the beginning. then our head of cataloguing said something like “i’m not buying into this discussion that keyword searching is a bad search. remember that keyword searches subject. indexing is the most important part of this.” then the technical services supervisor whose questions started the project said something like “i found the part about summon and the link resolver very interesting. this validates where we need to spend time. we can call out vendors where there is a consistent problem. now i can be pushy to get issues resolved. if that is what students are relying on, then we have to make sure what we have is right.” yes, having a head and a supervisor weigh in like this is bound to change the tone, but things did become much more positive and proactive from here on in, with comments and suggestions like this: “i’m wondering about loading e-book records. sometimes we have good records but they don’t have subjects. perhaps now i can load these records as they have summaries so they would get picked up in a keyword search.” “cataloguers can change the way we work and include table of contents and summaries in monograph records when we find them. perhaps we could make this an official policy and procedure.” “perhaps we can take more time to see how summon pulls information and where that information is pulled from.” conclusion so: a move to better understand how our discovery system handles our records a push to enrich print and ebook records to improve keyword searching a renewed focus on making sure the knowledge base is accurate so the link resolver works i know these aren’t necessarily ground-breaking ideas but less than an hour earlier, this same group suggested giving first year students a booklet on how to search! hearing that students mostly do keyword searches in summon and google scholar was understandably a little threatening to staff who have a very catalogue-centric view of the library (because that’s where they spend most of their time). but very quickly, they moved on and were suggesting new ways of doing things, and new ways of thinking about their work. it was wonderful. technical services and user experience don’t usually cross over, but we saw that it can be a really good fit. our students do their research online. technical services staff make decisions that affect how library resources are found online. so they are perfectly positioned to improve the user experience of our students. i’ll give the last word to one of our staff members, who after seeing our results said what i think we all want: “now i can attack the right problems with purpose.” adding useful friction to library ux: ideas from uxlibs workshop participants june , ~ shelley ~ comment at this years uxlibs conference, i led a workshop on adding useful friction to the library user experience. i’ve already posted my text of the workshop, but i told the participants that i also wanted to share the ideas that they came up with during the course of the workshop. the ideas were generated around three themes: friction for users, with a goal of helping those same users friction for staff, with a goal of helping users friction to improve inclusion in the library what is below is verbatim, as much as i could decipher, from the post-its. there’s seems to be a combination of examples of bad friction and ideas for good friction. if you were a participant and would like to correct or flesh out what’s here, please get in touch! here are all of the responses from both workshops, in no order at all: remove desk or make it a low height only lower the circulation desk or remove it altogether users: appointments (promote other options first, books/resources) giving the article rather than showing to find the resource answer questions instead of showing how to do it wayfinding no. enquiry — looking at it with fresh eyes staff want to put passive aggressive posters everywhere toilet sign / not a key — gender n. have the students suggest furniture in the library a room with a computer and a loudspeaker where the patron can hear what is on the screen clickable text where a loudspeaker symbol shows you that you can hear what is said wayfinding signage / posters — loads of / passive aggressive enforced preview when editing web pages put forms through web group to ensure they’re not excluding when they click around one website when they order on shelf items when they order articles making them having coffee with other departments and teaching staff making them walk across campus to use another office making them use the public spaces one hour a week you haven’t used library search for a while – do you need some tips? get rid of printed self-help so staff have to promote online self-help friction to help people understand the space they’re in helping new users find books (when they want it!) multi-language at entrances and around remove classification systems!! inclusivity check before things are published remove search term suggestions databases remove phone, e-mail, etc. from the info desk (anything that isn’t talking to students) giving more options for reference help – all hours of the day, off campus, offline, etc. change the quiet reading rooms with the group rooms every week have staff meetings at the group study areas/rooms put the check-out machines on the top floor wary of pop ups but what if pop has the answer to slow down scanning of web pages — part scan and leave just prior to achieving answer pop up box: “sign in to get: full text, saved search, e-shelf, etc.” a confirm button when adding manual charges to accounts all a h staff to be fully trained! had thought of reducing page options/text but could friction be added another way? when they order interlibrary loans we added friction to subj guide but — super friction -> no control for subj lib. therefore like the less friction idea presented by shelley pop up on self-issues: your books will auto-renew for “x” weeks but may be recalled are you sure? deleting records from endnote web ems editions: removing assets exhibition gallery: interactive screen event submission feedback forms find a book / study space blindfolded? stop them from using terms and phrases that people don’t understand test all changes to web page on real users, especially extreme users plain language checkers for web content highlighting research consultations over guides + dbs declarative signage: “you are on the (blank) floor, (blank wing)” website style guides push back on academic staff to upload accessible teaching materials to vle making ill request — have you check whether this is in library? (?) encourage use of learning technologies, but also provide analogue alternatives provide alternative signage options (multiple alternatives) when entering study zones -> be aware of conditions expected in space links that take users to arabic page rather than going back to main page allowing males to borrow / use the library during female hours having box to return the books used inside the library having a shared online spreadsheet if they would like to have someone to cover their desk hours rather than emailing did you know? pop ups on library websites ipads out intermittently to draw attention having to meet with a librarian signage (or something else!) that prompts new students to consider using library catalogue before trawling the physical shelves helpdesk would benefit from friction when students make initial enquiries re: learning difference support (e.g. dyslexia) — in my univ lib they are required to ask about this in an “open” queue without any confidentiality! near shelves potential redirect to lib cat on entry to help students choose appropriate working space on entry think about what student intends to achieve during visit replying to email enquiry messages force scroll to beginning to force people to read whole history have you left this in required state? bog poster for open access disabled loos creating new challenges in every day tasks to upskill staff, provide better services to users asking questions (too many!) to get essential services in place / working properly (e.g. hearing loops [or might be learning loops]) forcing users to rub up against us: “this resources has been brought to you by your library” (for colleagues) flag for spamming no. of forwards and emails to lists per day returning books through the book sorter—asking “have you returned all the books you need to” before issuing a receipt students who don’t have a disability but are anxious to be able to have one-to-one library hours, therefore all need to be asked at induction ilr’s (interlibrary requests) asking “is this available locally” items in store requested through the catalogue—”can this be accessed online” before the final request click—stops unnecessary collections from store that are not collected vendor outlinks “you are about to leave the library’s website” time to choose to read something you wouldn’t have thought of yourself time to reflect on impact of a certain behavior time to advertise additional services that might be helpful screens/maps to look at before looking for books → are you going where you want to? set target times to resolve a query. solutions should be quick and easy. library website: design decision library website: content cmas editors: removing assets ill request form when item not available library clear link when no results on discovery layer disable possibility to take a breath from chat stricter policy for adding web pages slow down book drop friction in ordering interlibrary loans which should be purchases how do we offer booking of “resource rooms”? can we make it more difficult to make web pages inaccessible? forced message to remove usb before the pc shuts down/logs you out triage questions? it vs library only hosting video tutorial with embedded subtitles — don’t rely on youtube autotitles = rubbish!! what images are you using to show your library? does it look inclusive on posters / online / in literature? e.g. pic of our staircase reservation collection—self-issue—extra touch screen with due date for hr loans stop them from rushing to the top floor, like signs in the elevator force staff to actually test the accessibility of web sites students, faculty, other ← library vrs → stop before leaving the chat “are you sure you don’t need further help?” how do we address people / users? double-check before making a poster to “solve” a problem! role management: design does not equal project management peer-checking of presentations / teaching sessions for accessibility writing training materials for students with english as a nd language uploading to online system: large files, microsoft format, video and audio (not stream), copyrighted to support distant or part-time students starting projects without: clarity about outcomes, testing, resources required adding resource e.g. reading list not using the system copying over last years materials to this years module better obstacle than fee for interlibrary loans or document delivery remove “scripts” for staff answers on just ask (im) — be more personal? no pictures of pdfs or text on web — screen readers can’t cope with them pop-ups letting students know access is being provided by the library (to online resources) library website qr codes?? symbols instead of english — puts everyone at the same level of wayfinding regardless of language skills diverse reading lists know your staff wiki! regular process to review existing web content before adding more entrance vestibule to silent study spaces promoting self-service portal at library entrance chatline. faqs page to scroll through to get to input page force a catalogue search before submitting an ill request policy that all staff deal with a request for help at point of need and see through logging all enquiries on an ems pick-up shelf: make users check out their reading room loans database records in summon—people going straight to lib search when not everything is listed sign up form for focus groups so we can pick by course, not first come, first served academic workbooks arranged by topic on cms not just straight link to as server online support and workshops more prominently promoted than : s as easier to same [some?] large number i need to approve all external comms and surveys web edits — i have to approve all pages training on [survey?] software linked to approval from me me as final editor for newsletter (brand / accessibility) gender neutral toilets editing text for screen readers — on all channels check catalogue for students who have incorrect info on reserve items complete a short online library quiz as part of first module activate your student card in the library within the last week of term put “please refer to…” messages where rules aren’t clear ill — request articles/books we already have—way to make them search first? search box — choose what format first (they will type anything in a search box without thinking and then think we don’t have an article because they are looking in catalog) ebooks — add to reserves or pop up asking them to look by title student staff tell students we don’t have an item when we do — need to try other ways — have system prompt? expand chat hours so people uncomfortable approaching desk can still ask questions cms — make popup for alt-text but also color contrast, web writing, close-captioning for videos, etc. content manager for website — approve all changes even subject guides better feedback on item request — many are not picked up knowing who your liaison is if on a certain page staff friction: using crm or equivalent to report issues to other teams, i.e. metadata errors: don’t ring team, logon landesk (crm). has advantages collating themes and work. inclusion: feedback form gender user [gateways portals]: to prompt and remind about compliance maybe – copyright / usage — use of data/info. authentication does this also. staff: printing checklist actions before resorting to use of staff printer user: to prompt remind/inform resources purchased on behalf of students by institution it passwords for faculty users using lockers after library closing hours computers on every floor (staff) toilets (improve inclusion) game area (students) lounge area (students) change main structure of website adding too long text to buttons adding too many main category pages put “silence” signs on every door → there have to be noisy places just grab a book (without having a look to the books around) policy: force all staff to use structured text documents so that they are accessible self-return machines (don’t take think books, so we need to “slow” the users know know this) inclusion: programs → languages open access funding program → read criteria before submitting the application adding too long texts into modals designed to be glanced gender in feedback forms requirement for text and audio on video request / reservation: this book is on the shelves in this library. are you sure you want to request it? [checkbox] yes. sign on ground floor: the only toilets and drinking water in this building are on this floor. (most library services are floors up from here) making gender option in forms more inclusive e.g. more option or textbox before making an order/reservation that costs money before making a reservation before deleting your user account before deleting any info permanently get staff out of their offices — send them to find academic who have not been in the library for a long time we have a lib reciprocal programme across unis in s.a. but in our lib we force users to see an info lib before they get a letter to visit another uni library. catalogue research (first finding is seldom the best) remove option to add media on webpages for most staff accessibility checks before publishing a webpage filling out book request form for somebody clearing a list in the catalog printing single vs. double sided staff designate, monitor, and enforce quiet areas building entrance vs. exit reserving lendable technology requesting items from storage information in subject guides giving information to new students about the library’s services ordering interlibrary loans in the discovery systems request print copies of articles promote new physical and online materials in entrance user (student) testing before buying e-books build ux into all projects prayer facilities a note on self service screen to common ?s. really good idea. spending more time with the unfamiliar symbol sign posting meet and greeters at front door pick up cards at library send librarians out to visit people stop “library” work at enquiry point wellbeing attention grabbing display — subject guide to registration online — pick up library card in person commuter room with lockers — charging (away from home help) auto emails for book arrivals triggered by front desk team so that we are certain it is ready on the shelf friction needed to prevent deletion of content subject guides allow use to browse area and discover other books related to study develop electronic check lists for staff to ensure staff complete all necessary steps in a task on time and in order finding tools — before search encourage users to reflect if using the right finding tool reading lists — cap amount of items that can be added → “do you really want to add this item?” self-issue machines — add “do you want to borrow” for very short loans / high charge (had at public) modernise the till and integrate with lms. creates a couple of steps that slows staff and avoids mistakes on the till from “autopilot” “lost” status and “found” status. create pop up explaining what to do and if want to continue to avoid incorrect use. int’l students — don’t assume that library experience of someone else is the same particularly when they have a different international experience / encourage staff to think before assuming person is just not as smart as culture they are accustomed to. filters — putting a [friction?] to alert people that they can expand their search to include content not available at [library] as well friction for staff: prompts to ask particular questions / edit or do something people often forget when searching: “this search result is showing everything. is that what you want?” or “it looks like you might be searching for a journal title. would you like to do that?” different language options — catalogue, website / signage compulsory reflection on implicit biases before finalising a form / policy / procedure / interview / process / etc…. sometimes it’s good to get “lost” and find hidden spaces… have “no wifi” areas to create “switch-off” spaces… noise control — something that encourages slowing of pace / pause on entry furniture might cue quiet study vs. collaboration if staff are including a gender (or other protected characteristic) question on a form, make them type their justification! supporting assistive tech (friction for staff) stop long forms with every piece of info the librarian needs to order an item shibbolth sign in from pub page — get to the right path, choose the best relationship for access group study facilities — varied tech options more tailored handouts for students who have english as nd language or rd etc. dvd borrowing: “don’t forget to unlock your case!” pop-up? multimedia options for dyslexic students — on entry to library chat box help kiosk for students who feel like “imposters” (afraid to admit what they don’t know) single sign on — subject / cs team comms. consistent approach to adding info to app. autonomy and overall framework. quizzes on vces at end of modules furniture — soft for de-stresses commuter students — find out what their priorities are and how this differs from other students to get integrated in the education with the library competence, so every student gets the same education (information literacy) find location in the library gender free web block them from the staff cataloguing opac — only use for hour a day think of the people you put on the website. still mostly young, happy users. teacher making resource lists users: interlibrary loans pop-up help button after kw searches < minute discover layer: where am i searching website friction from adding content — specifically start “headlines” when coming in to the library — to show services offered that are “unknown” stacks — “did you find out the exact location of your book?” making signs — added friction for personnel multilingual captioning sign friction or not? faculty-librarian meeting for new faculty (in-person? why?) more faculty-librarian friction leaving web presence, what about credibility? evaluate results require ait text on img upload when leaving discovery tool to external site management friction default web editor template; to change, require friction consider for more friction at admin side mandatory meeting with librarian for an assignment swipe card to enter the library baby changing tables rainbow lanyards help uniforms / sashes? program friction — new program proposal signage dual monitor search comp. for info desk enquiries stop users from ordering books on shelf warning pop up !danger! universal design on website pause before changing your brand colors etc. to your online library interface. …consider accessibility first. pause before allowing online systems use your personal data …instead, learn what the provider will do with your data pause before composing the perfect, new metadata or information model for the new library service …instead, involve users and designers in the process shh… quiet beyond this point posts navigation older posts archives october october june december october september june june december october june november august june march january december november tags changes conference notes conference presentation information architecture user research user testing ux uxlibs create a free website or blog at wordpress.com. shelley gullikson create a free website or blog at wordpress.com. add your thoughts here... (optional) post to cancel privacy & cookies: this site uses cookies. by continuing to use this website, you agree to their use. to find out more, including how to control cookies, see here: cookie policy dshr's blog dshr's blog i'm david rosenthal, and this is a place to discuss the work i'm doing in digital preservation. talk at berkeley's information access seminar chromebook linux update effort balancing and rate limits isp monopolies the bitcoin "price" two million page views! the new oldweb.today michael nelson's group on archiving twitter stablecoins risc vs. cisc max ungrounding i rest my case storage media update even more on the ad bubble the order flow the long now unbanking the banked a note on blockchains liability in the software supply chain moxie marlinspike on decentralization don't say we didn't warn you amazon is profitable? open source saturation shout-out to gutenberg project lack of anti-trust enforcement history publications | department of history | university of windsor home search browse collections my account about digital commons network™ skip to main content university of windsor my account faq about home scholarship at uwindsor home > fahss > history > historypub history publications follow index jump to: - submissions from link “hermanos del cielo”*. figuras de vuelo en la poesía y la pintura sevillana de fernando de herrera a juan de arguijo (con francisco pacheco de por medio), guy lazure submissions from link fruits of perseverance: the french presence in the detroit river region, - , guillaume teasdale submissions from pdf blood, sweat, and fear: violence at work in the north american auto-industry - , christina ann burr pdf fashioning fame: fernando de herrera's anotaciones as a space of knowledge, guy lazure pdf a fluid frontier: slavery, resistance, and the underground railroad in the detroit river borderland ed. by karolyn smardz frost and veta smith tucker (review), guillaume teasdale pdf conflicted colony: critical episodes in nineteenth-century newfoundland and labrador, miriam wright submissions from pdf silk stockings and socialism: philadelphia's radical hosiery workers from the jazz age to the new deal, christina ann burr pdf the power of beauty: commercial beauty culture, the body, and women's political activism, christina ann burr pdf why the flapper still matters: feminist pedagogy, the modern girl, and the women artists of the beaver hall group, christina ann burr submissions from link réfléchir par l’histoire du livre, leslie howsam link thinking through the history of the book, leslie howsam link "the disgrace of christendom": shipwreck and ransoming in southern morocco ( s- s), mohamed hassan mohamed pdf canadian ocean fisheries, miriam wright pdf death on two fronts: national tragedies and the fate of democracy in newfoundland, – by sean cadigan (review), miriam wright submissions from pdf pratiques intellectuelles et transmission du savoir dans les milieux lettrés sévillans. l'archéologie de deux grandes bibliothèques, xvie-xviie siècles, guy lazure link la grande famine en irlande, frank rynne and adam pole link une amérique française, - : dynamiques du corridor créole, guillaume teasdale and tangi villerbu submissions from pdf stephanie gilmore. groundswell: grassroots feminist activism in postwar america. new york: routledge, ., natalie atkin pdf history in the periodical press online: a revised informal introduction to hippo, leslie howsam link epistolario, guy lazure, pedro vélez de guevara, and bartolomé pozuelo calero link women in the newfoundland fishery, miriam wright submissions from pdf hearts and minds: canadian romance at the dawn of the modern era, – , christina ann burr link the beauty soap of film stars: lux toilet soap, star endorsements, and building a global beauty brand., christina ann burr pdf fanny “bobbie” rosenfeld: a “modern woman” of sport and journalism in twentieth-century canada, christina ann burr and carol a. reader pdf fanny “bobbie” rosenfeld: a “modern woman” of sport and journalism in twentieth-century canada, christina ann burr and carol a. reader pdf new directions for research and pedagogy in book history, leslie howsam pdf rural ireland: the inside story, adam pole link sheriffs in victorian london, adam pole link french and indians in the heart of north america, - , guillaume teasdale and robert englebert submissions from link gendered limitations on women property owners: three women of early modern cairo, shauna huffaker link representations of ahmed urabi: hegemony, imperialism, and the british press, – , shauna huffaker link review: a history of women’s seclusion in the middle east, shauna huffaker link albores de un humanismo vernáculo: el entorno catedralicio y la traducción de libros en la sevilla de principios del siglo xvi, guy lazure link carrera profesional y orígenes socio-económicos de a élite cultural sevillana del siglo xvi y xvii, guy lazure pdf the escorial: art and power in the renaissance, guy lazure link le moment anglais des jésuites en espagne. réflexions autour de la série de rois saints de francisco pacheco (v. ), guy lazure and cécile vincent-cassy pdf africanists and africans of the maghrib ii: casualties of secularity, mohamed hassan mohamed link between caravan and sultan: the bayruk of southern morocco, a study in history and identity, mohamed hassan mohamed link old friends and new foes: french settlers and indians in the detroit river border region, guillaume teasdale submissions from link 'the closest thing to perfect': celebrity and the body politics of jamie lee curtis., christina ann burr pdf building bridges between antwerp and seville : friends and followers of benito arias montano, - , guy lazure link rodrigo caro ( - ) y la corte de felipe iv : itinerario de unas ambiciones frustradas, guy lazure pdf review of lydon, ghislaine, on trans-saharan trails: islamic law, trade networks, and cross-cultural exchange in nineteenth-century western africa, mohamed hassan mohamed pdf fighting like the devil for the sake of god: protestants, catholics and the origins of violence in victorian belfast, by mark doyle, adam pole pdf les débuts de l'Église catholique américaine et le monde atlantique français : le cas de l'ancienne colonie française de détroit, guillaume teasdale pdf review of les réfugiés acadiens en france, - : l'impossible réintégration?, guillaume teasdale pdf clearing the coastline: the nineteenth‐century ecological and cultural transformation of cape cod., miriam wright pdf terranova: the spanish cod fishery on the grand banks of newfoundland in the twentieth century (review), miriam wright submissions from pdf a few good women: america's military women from world war i to the wars in iraq and afghanistan (review), natalie atkin link introducing book networks and cultural capital: space, society and the nation, leslie howsam pdf africanists and africans of the maghrib: casualties of analogy, mohamed hassan mohamed pdf aboriginal gillnet fishers, science, and the state: salmon fisheries management on the nass and skeena rivers, british columbia, - , miriam wright submissions from link posséder le sacré. monarchie et identité dans la collection de reliques de philippe ii à l'escorial, guy lazure link brian dillon, adam pole link john wallace crawford, adam pole pdf the british monarchy and ireland: to the present, adam pole pdf des destinées distinctes : les français de la région de la rivière détroit et leurs voisins amérindiens, - , guillaume teasdale pdf marcel bénéteau et peter w. halford, mots choisis. trois cents ans de francophonie au détroit du lac Érié, ottawa, les presses de l’université d’ottawa, , p., guillaume teasdale pdf review of the upper country: french enterprise in the colonial great lakes, guillaume teasdale pdf landing native fisheries: native reserves and fishing rights in british columbia, – : book review, miriam wright submissions from link the other side: the rhetoric of labour reform in toronto during the s.”, christina ann burr pdf what is the historiography of books? recent studies in authorship, publishing, and reading in modern britain and north america, leslie howsam link the lay of the land: property, gender and household in the development of al-darb al-ahmar district in mamluk cairo, shauna huffaker link ‘un vehemente deseo de comprender la imagen de aquel famoso templo se adueña de mí’. seeing and understanding the temple of solomon according to juan bautista villalpando s.j. ( ), guy lazure pdf ireland, radicalism and the scottish highlands, c. - , adam pole pdf 'building the great lucrative fishing industry': aboriginal gillnet fishers and protests over salmon fishery regulations for the nass and skeena rivers, s- s, miriam wright submissions from link printing in canada in the twentieth century, christina ann burr link nadal au nouveau monde. une traduction poétique des «evangelicae historiae imagines», pérou, c. , guy lazure pdf possessing the sacred: monarchy and identity in philip ii's relic collection at the escorial, guy lazure submissions from link canada's victorian oil town: the transformation of petrolia from a resource town into a victorian community, christina ann burr link mecenazgo y clientelismo en los años sevillanos de benito arias montano: genealogía social e intelectual de un humanista, guy lazure link les systèmes religieux amérindiens et inuit: perspectives historiques et contemporaines, guillaume teasdale and claude gélinas submissions from pdf "oil mania": colonial land policy, land speculation, and settlement in enniskillen township, s- s | histoire sociale / social history, christina ann burr pdf Étude et connaissance des religions autochtones au québec : un bilan historique, claude gélinas and guillaume teasdale pdf david mckitterick. new worlds for learning, - , vol of a history of cambridge university press., leslie howsam pdf imperial publishers and the idea of colonial history, - , leslie howsam link sheriffs' sales during the land war, - , adam pole pdf the irish national league in dingle, county kerry, – , adam pole pdf book review: parham, clair, from great wilderness to seaway towns, miriam wright pdf images of the fisher folk in newfoundland, - s, miriam wright submissions from pdf from margin to mainstream: american peace movements, s - s, natalie atkin pdf at odds: gambling and canadians, - , christina ann burr pdf great canadian political cartoons, to (review), christina ann burr link working in the printing trades, christina ann burr pdf academic discipline or literary genre? the establishment of boundaries in historical writing, leslie howsam pdf the royal irish constabulary. a history and personal memoir, adam pole pdf cod moratorium, miriam wright pdf territorial waters, miriam wright submissions from pdf no turning back: the history of feminism and the future of women/ the world split open: how the modern women's movement changed america (book), natalie atkin pdf lacrosse: a history of the game, christina ann burr pdf some adventures of the boys: enniskillen township's "foreign drillers," imperialism, and colonial discourse, - , christina ann burr link book history unbound: transactions of the written word made public, leslie howsam pdf review: the decline of the big house in ireland: a study of irish landed families, – , adam pole pdf the decline of the big house in ireland: a study of irish landed families, – , adam pole pdf newfoundland and labrador history, - , miriam wright submissions from pdf gender, sexuality, and nationalism in j.w. bengough's verses and political cartoons, christina ann burr pdf manufacturing montreal: the making of an industrial landscape, to ,, christina ann burr link hermanos del cielo: Ícaro, faetón y otras figuras del vuelo en el humanismo sevillano, guy lazure link un catálogo de las obras de isidoro de sevilla conservadas en diversas bibliotecas españolas en el siglo xvi, guy lazure and pérez antonio dávila link early warning signs: changes in the newfoundland inshore fisheries in the 's and s, miriam wright search enter search terms: select context to search: in this series in this repository across all repositories advanced search notify me via email or rss browse collections disciplines disciplines authors electronic theses and dissertations author corner author faq policies how to submit about open access submit research links department of history website digital commons home | about | faq | my account | accessibility statement privacy copyright data science community - service manual - gov.uk skip to main content tell us whether you accept cookies we use cookies to collect information about how you use gov.uk. we use this information to make the website work as well as possible and improve government services. accept all cookies set cookie preferences you’ve accepted all cookies. you can change your cookie settings at any time. hide gov.uk service manual show or hide search search on gov.uk search coronavirus (covid- ) national lockdown: stay at home brexit check what you need to do beta complete our quick -question survey to help us improve our content. service manual communities communities data science community give feedback about this page last updated: over year ago page contents: who the community is for get involved community events training courses areas of special interest community resources the data science community exists to: bring together data scientists and improve data science capability across government develop and share data science best practice help digital and other analytical teams work effectively with data scientists discuss and challenge the way government uses machine learning and artificial intelligence to improve services who the community is for you might be interested in this community if your work involves: making sure decisions are based on evidence presenting insights in an understandable way to both technical and non-technical audiences building credible statistical models from data and using best coding practices to generate reproducible work commissioning data science projects working with policy and operations to understand where data science can improve services for users exploring new data science technology, tools and algorithms deciding how to deploy data science at scale get involved you can use the following online spaces to discuss data science related issues: the cross-government data science slack - you’ll need a government email address to access this channel the uk government data science github - this is where to find repositories for code related to data science in government community events the government data science partnership (gdsp) brings together public servants to share their knowledge about data science. the partnership runs meetups every to months. they’re open to anyone in government with an interest in data science and often include speakers from across government, academia and industry. the meetups are hosted in different places around the country. there’s also an annual cross-government data science conference. the next one is planned for late . to sign up for either of these events, you can: sign up to attend via the mailing list join the #community_of_interest slack channel training courses the data science accelerator is a programme to help analysts and aspiring data scientists from across the public sector develop their data science skills. participants propose and work on a data science project for months under the guidance of a mentor. the gds academy also runs relevant courses including: the introduction to artificial intelligence course the foundation programme and the service owner programme, both of which include introductory analytics training you might also find some of the courses run by the government statistical service or the data science campus useful. areas of special interest if you’re particularly interested in text analytics and natural language processing, you can join the #nlp channel in the cross-government data science slack. you could also join the text analytics mailing list. if you’re interested in reproducible analytical pipelines (rap), which is a method for automating the production of statistical publications, you can: visit the rap website see the government statistical service’s page about rap champions join the #rap-collaboration channel on the cross-government data science slack community resources use these resources to learn more about data science in government: data scientist role requirements - explains the skills you’ll need to be a data scientist in government data in government blog - discusses the tools and techniques that gds uses for data analysis and data science data ethics framework - guide the design of appropriate data use in government and the wider public sector last update: june guidance first published is this page useful? maybe yes this page is useful no this page is not useful thank you for your feedback there is something wrong with this page close help us improve gov.uk don’t include personal or financial information like your national insurance number or credit card details. what were you doing? what went wrong? send close help us improve gov.uk to help us improve gov.uk, we’d like to know more about your visit today. we’ll send you a link to a feedback form. it will take only minutes to fill in. don’t worry we won’t send you spam or share your email address with anyone. email address send me the survey support links help privacy cookies contact accessibility statement terms and conditions rhestr o wasanaethau cymraeg built by the government digital service open government licence all content is available under the open government licence v . , except where otherwise stated © crown copyright gfdrr skip to main content header main navigation who we are how we work where we work knowledge hub what's new dropdown main navigation who we are + funding structure & partnerships meet our team how we work + gfdrr labs challenge fund understanding risk open data for resilience initiative disruptive technologies resilient infrastructure building regulation for resilience global program for safer schools nature-based solutions resilient cities city resilience program hydromet services africa hydromet program climate risk and early warning systems (crews) crews caribbean financial protection sfrarr central asia caribbean regional resilience building facility drf analytics drfi in caribbean octs global risk financing facility social resilience drm-fcv nexus gender resilience to climate change small island states resilience initiative canada-caribbean resilience facility resilient recovery where we work + africa europe and central asia east asia and pacific latin america and caribbean middle east and north africa south asia search grants knowledge hub + publications online tools e-learning what's new + feature stories events blogs news videos search image the latest news and insights news world bank announces winners of techemerge resilience india challenge learn more > publication city resilience program: annual report july – june learn more > news supporting data for resilience at open data day learn more > publication disasters, conflict, and displacement: intersectional risks in south sudan learn more > blog everybody has a role: how stakeholders are contributing to increase urban resilience in cameroon learn more > blog from urban fire risk to effective mitigation – building safer & healthier cities learn more > blog boosting climate resilience and disaster recovery in africa: the africa hydromet program learn more > blog how a hackathon sparked resilience & innovation throughout one of the largest western african cities learn more > who we are the global facility for disaster reduction and recovery (gfdrr) is a global partnership that helps developing countries better understand and reduce their vulnerability to natural hazards and climate change. learn more what we do gfdrr is organized around these eight priority topics. image gfdrr labs gfdrr labs uses cutting-edge science and technology to make high-quality risk information available faster and at lower costs, and develop new tools that allow decision-makers and communities to collect, share, and understand risk information. learn more image resilient infrastructure infrastructure is the critical lifeline of social and economic activity, connecting communities, industry and markets with essential services for the operation of daily life. learn more image resilient cities we support cities in strengthening their ability to better manage stresses and shocks from disaster and climate hazards. learn more image hydromet services and early warning systems we help countries plan and prepare for extreme weather by working with them to strengthen their hydromet monitoring, forecasting and early warning systems. learn more image financial protection we support countries in their efforts to design and implement financial protection strategies and instruments which can help them manage the costs of disaster and climate shocks. learn more image social resilience we work with countries to tackle the gender and social dimensions of disaster risk, while also mobilizing the expertise of citizens and communities for strengthening resilience. learn more image resilience to climate change we work with countries to strengthen their ability to plan, design and implement climate resilient policies and investments. learn more image resilient recovery we support governments prepare for and respond to emergencies to ensure quicker and more resilient post-disaster recovery. learn more image image image image image image image image image knowledge hub explore our repository of knowledge on resilience and disaster risk management. visit the hub image “whenever a disaster, a crisis or a risk happens, the first ones to be affected are women and children … it is for this reason that it is very important to involve women in disaster prevention." fatima ahler, director of niger digital cartography agency, president of femme & tic niger highlights take a deep dive into some of our work at the cutting edge of resilience and disaster risk management. publication lifelines: the resilient infrastructure opportunity the business case for investing in more resilient infrastructure worldwide. learn more publication unbreakable the report warns that human & economic impacts of extreme weather on poverty are more devastating learn more publication building back better achieving resilience through stronger, faster, and more inclusive post-disaster recovery. learn more follow us our key partners featured programs the japan-world bank program for mainstreaming drm acp-eu natural disaster risk reduction program gfdrr members & observers australia austria canada germany india italy japan luxembourg norway serbia sweden switzerland united states africa, caribbean & pacific (acp) secretariat european union united nations office for disaster risk reduction world bank group denmark finland france portugal spain saudi arabia united kingdom global network of civil society organisations for disaster reduction huairou commission international federation of red cross and red crescent societies islamic development bank organisation for economic co-operation and development united nations development programme world meteorological organization global facility for disaster reduction and recovery (gfdrr) world bank headquarters h street nw washington dc (u.s.) footer main navigation who we are funding structure & partnerships meet our team how we work gfdrr labs resilient infrastructure resilient cities hydromet services financial protection social resilience resilience to climate change resilient recovery where we work africa europe and central asia east asia and pacific latin america and caribbean middle east and north africa south asia search grants knowledge hub publications online tools e-learning what's new feature stories events blogs news videos search follow us © global facility for disaster reduction and recovery terms of use privacy policy contact us influx library user experience – just another wordpress site contact partner with influx and create delightful experiences for your patrons a few things we do prefab we've already built a great website for your library. ux evaluations what is your library doing well, and what can you improve? website reviews looking for some expert feedback on your website? get our eyes on it. presentations level up your staff's ux skills. website redesign full service research / design / build and anything in-between. contact hello@weareinflux.com . . be nice and have fun be nice and have fun be nice and have fun × contact records released in response to presidential records act (pra) inquiries, trump administration | national archives top skip to main content the u.s. national archives home menu blogs · bookmark/share · contact us search main menu research our records veterans' service records educator resources visit us america's founding documents freedom of information act (foia) home > freedom of information act (foia) > records released in response to presidential records act (pra) inquiries, trump administration freedom of information act (foia) foia reference guide electronic (foia) reading room nara foia regulations nara foia directives foia annual reports department of justice foia guidance privacy program resources congressional hearings & testimony contacts archival fee schedule other foia resources records released in response to presidential records act (pra) inquiries, trump administration nara officials are often consulted about records management of presidential and federal records. the presidential records act governs access to records after the end of an administration. for more information visit https://www.archives.gov/presidential-libraries/laws/ -act.html. the national archives will receive, preserve, and provide public access to all official trump administration social media content, including deleted posts from @realdonaldtrump and @potus. the white house has been using an archiving tool that captures and preserves all content, in accordance with the presidential records act and in consultation with national archives officials. these records will be turned over to the national archives beginning on january , , and the president's accounts will then be made available online at nara's newly established trumplibrary.gov website. below are documents concerning records management related inquiries about the trump administration's presidential records. document title document date white house letter to the archivist of the united states: trump pra designation february , white house memo to all personnel re pra obligations (attached to oct. compliance reminder email) february , letter from sens. mccaskill and carper to archivist david s. ferriero march , archivist response to sens. mccaskill and carper march , attachments to march , letter: briefing on the pra background on the pra guidance on presidential records (publication) presidential records, u.s.c. chapter proposed rule on presidential records (note: now a final rule) notification procedures approved requests for waiver of incumbent president privilege review approved requests for disposal of incumbent presidential records nsc memo, feb. , on procedure for access to pra records of previous administration archivist's memo to senior agency officials for records management on records management priorities for records express blog on records management of social media and electronic records white house email: compliance reminder; pra october , white house email: monthly legal compliance and ethics training reminder october , american oversight letter: presidential advisory commission on election integrity and potential alienation of records january , nara response letter to american oversight: presidential advisory commission on election integrity and potential alienation of records january , white house letter to the archivist of the united states: updated trump designation of pra representatives january , this page was last reviewed on january , . contact us with questions or comments. archives.gov information for… citizen archivists federal employees genealogists members of congress preservation records managers the press publications federal register prologue magazine subscribe to email newsletters all publications orgs. & offices center for legislative archives federal records center office of the inspector general presidential libraries staff directory about us what is the national archives? doing business with us plans and reports open government our plain language activities i want to… get my military record plan a research visit visit the museum view online exhibits apply for a grant participate attend an event donate to the archives work at the archives volunteer at the archives resources a-z index america's founding docs contact us en español faqs forms shop online visit the national archives store buy reproductions and microfilm connect with us facebook twitter instagram tumblr youtube blogs flickr contact us · accessibility · privacy policy · freedom of information act · no fear act · usa.gov the u.s. national archives and records administration - -nara-nara or - - - top influx library user experience influx library user experience just another wordpress site iosevka iosevka . . -rc. version 📦 download fontscustomizerepositoryspecimen iosevka is an open-source, sans-serif + slab-serif, monospace + quasi‑proportional typeface family, designed for writing code, using in terminals, and preparing technical documents. iosevka monospace weights × slopes × widths iosevka term monospace weights × slopes × widths iosevka fixed monospace, ligation off weights × slopes × widths iosevka slab monospace weights × slopes × widths iosevka term slab monospace weights × slopes × widths iosevka fixed slab monospace, ligation off weights × slopes × widths iosevka aile quasi-proportional weights × slopes × widths iosevka etoile quasi-proportional weights × slopes × widths thin extralight light regular medium semibold bold extrabold heavy upright oblique italic normal extended characters covered glyphs included languages supported character variant features stylistic set features ligation sets spatial efficient the iosevka’s monospace family is provided in a slender outfit by default: glyphs are exactly / em wide. compared to the competitors, you could fit more columns within the same screen width. iosevka provides two widths, normal and extended. if you prefer more breeze between the character, choose extended and enjoy. iosevkaiosevka extended fira codeibm plex # ucd data section b ; degree sign ; so ; ; et ; ; ; ; ; n ; ; ; ; ; b ; plus-minus sign ; sm ; ; et ; ; ; ; ; n ; plus-or-minus sign ; ; ; ; b ; superscript two ; no ; ; en ; ; ; ; ; n ; superscript digit two ; ; ; ; b ; superscript three ; no ; ; en ; ; ; ; ; n ; superscript digit three ; ; ; ; b ; acute accent ; sk ; ; on ; ; ; ; ; n ; spacing acute ; ; ; ; b ; micro sign ; ll ; ; l ; bc ; ; ; ; n ; ; ; c ; ; c b ; pilcrow sign ; po ; ; on ; ; ; ; ; n ; paragraph sign ; ; ; ; b ; middle dot ; po ; ; on ; ; ; ; ; n ; ; ; ; ; b ; cedilla ; sk ; ; on ; ; ; ; ; n ; spacing cedilla ; ; ; ; b ; superscript one ; no ; ; en ; ; ; ; ; n ; superscript digit one ; ; ; ; ba ; masculine ordinal indicator ; lo ; ; l ; f ; ; ; ; n ; ; ; ; ; bb ; right-pointing double angle quotation mark ; pf ; ; on ; ; ; ; ; y ; right pointing guillemet ; ; ; ; bc ; vulgar fraction one quarter ; no ; ; on ; ; ; ; / ; n ; fraction one quarter ; ; ; ; bd ; vulgar fraction one half ; no ; ; on ; ; ; ; / ; n ; fraction one half ; ; ; ; be ; vulgar fraction three quarters ; no ; ; on ; ; ; ; / ; n ; fraction three quarters ; ; ; ; bf ; inverted question mark ; po ; ; on ; ; ; ; ; n ; ; ; ; ; c ; latin capital letter a with grave ; lu ; ; l ; ; ; ; ; n ; latin capital letter a grave ; ; ; e ; c ; latin capital letter a with acute ; lu ; ; l ; ; ; ; ; n ; latin capital letter a acute ; ; ; e ; c ; latin capital letter a with circumflex ; lu ; ; l ; ; ; ; ; n ; latin capital letter a circumflex ; ; ; e ; c ; latin capital letter a with tilde ; lu ; ; l ; ; ; ; ; n ; latin capital letter a tilde ; ; ; e ; c ; latin capital letter a with diaeresis ; lu ; ; l ; ; ; ; ; n ; latin capital letter a diaeresis ; ; ; e ; c ; latin capital letter a with ring above ; lu ; ; l ; a ; ; ; ; n ; latin capital letter a ring ; ; ; e ; c ; latin capital letter ae ; lu ; ; l ; ; ; ; ; n ; latin capital letter a e ; ; ; e ; c ; latin capital letter c with cedilla ; lu ; ; l ; ; ; ; ; n ; latin capital letter c cedilla ; ; ; e ; c ; latin capital letter e with grave ; lu ; ; l ; ; ; ; ; n ; latin capital letter e grave ; ; ; e ; c ; latin capital letter e with acute ; lu ; ; l ; ; ; ; ; n ; latin capital letter e acute ; ; ; e ; code and terminal terminal emulators have a stricter compatibility requirements for fonts. therefore, iosevka and iosevka slab all contain two specialized families, term and fixed, targeting terminal users. in these families, the symbols will be narrower to follow terminals’ ideology of column count. in the fixed families, the ligation will be disabled to ensure better compatibility in certain environments. iosevka ∑ { n ∈ ▲ } 🅇(n) ○-> Ⓨ[n] ▢△◈ iosevka term ∑ { n ∈ ▲ } 🅇(n) ○-> Ⓨ[n] ▢△◈ iosevka fixed ∑ { n ∈ ▲ } 🅇(n) ○-> Ⓨ[n] ▢△◈ variants à la carte iosevka provides are configurable characters and stylistic sets, which gives you incredible ability to customize one variant that fills exactly what you need. you can either select one pre-defined stylistic set, or cherry-pick your flavor. sansslabaileetoile italicoblique extended @real fox.quick(h){ *is_brown && it_jumps_over(dogs.lazy) } abc.def.ghi.jkl.mno.pqrs.tuv.wxyz ß <=`¶^$#%'>= stylistic sets offdefaultss andale mono styless anonymous pro styless consolas styless menlo styless fira mono styless liberation mono styless monaco styless pragmata pro styless source code pro styless envy code r styless x window styless ubuntu mono styless lucida styless jetbrains mono styless ibm plex mono styless pt mono styless recursive mono styless curly style cherry-picking a▼aaaa b▼bbbbbbbbbbbb c▼ccccc d▼dddddd e▼ee f▼ff g▼ggggggggggggggggggggggggggg h▼hh i▼iii j▼jjjjjjjjjj k▼kkkkkkkk l▼ll m▼mmmm n▼nnnn p▼pppp q▼qqqqqqqq r▼rrrrrrrrrrrr s▼sssss t▼tt u▼uuuuuuuuuu v▼vvvv w▼wwwwwwwwww x▼xxxx y▼yyyyyy z▼zzzzzzzzzz a▼aaaaaaaaaaaaaa b▼bbbbbb c▼ccccc d▼dddddddd e▼ee f▼ffffffffffffffffffff g▼gggggggggg h▼hhhh i▼iiiiiiiiiiii j▼jjjjjjjj k▼kkkkkkkkkkkk l▼llllllllllll m▼mmmmmmmmmmmmmmmmmmmm n▼nnnnnnnn p▼pppp q▼qqqqqqqqqqqq r▼rrrrrrrrrrrr s▼sssss t▼ttttttt u▼uuuuuuuu v▼vvvvvv w▼wwwwwwwwwwww x▼xxxx y▼yyyyyyyyyyyy z▼zzzzzzzzzz ß▼ßßß Λ▼ΛΛ α▼αα Γ▼ΓΓ ɛ▼ɛɛɛ ɜ▼ɜɜɜ ι▼ιιιιιι λ▼λλλλ к▼кккккккк У▼УУУУУУ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ~▼~~ *▼****** _▼___ ¶▼¶¶ ^▼^^^ (▼(( {▼{{ #▼#### &▼&&&&&&& @▼@@@ $▼$$$$$$ ¢▼¢¢¢ %▼%%%% |▼|| <=▼<=<= '▼'' `▼``` ?▼??? show usagebake custom build ligations for coding iosevka’s monospace families supports not only one ligation set, but also language-specific ligations. enable the corresponded opentype feature in your editor, and the correct ligation will appear. leveraging advanced opentype techniques, iosevka also supports long ligatures like long arrows or horizontal bars built up with equal signs. sansslab italicoblique extended -<< -< -<- <-- <--- <<- <- -> ->> --> ---> ->- >- >>- =<< =< =<= <== <=== <<= <= => =>> ==> ===> =>= >= >>= <-> <--> <---> <----> <=> <==> <===> <====> --------> <~~ <~ ~> ~~> :: ::: == != /= ~= <> === !== =/= =!= := :- :+ <* <*> *> <| <|> |> <. <.> .> +: -: =: :> (* *) [| |] {| |} ++ +++ \/ /\ |- -|