Search
Search

Why aren't programs affecting this pipe output?

Stuart P. Bentley Published at Dev
14
Stuart P. Bentley

I'm passing the output of a command to sort, but the result I'm getting back isn't sorted:

$ plushu logs foo | sort
2015-05-23T11:07:18.294569666Z Unhandled rejection RqlDriverError: First argument to `run` must be an open connection.
2015-05-23T11:07:18.294627441Z     at new RqlDriverError (/app/node_modules/endex/node_modules/rethinkdb/errors.js:14:13)
2015-05-23T11:07:18.294632871Z     at MakeArray.TermBase.run (/app/node_modules/endex/node_modules/rethinkdb/ast.js:129:29)
2015-05-23T11:07:18.294636744Z     at /app/node_modules/endex/index.js:131:24
2015-05-23T11:07:18.294640994Z     at tryCatcher (/app/node_modules/endex/node_modules/bluebird/js/main/util.js:24:31)
2015-05-23T11:07:18.294644642Z     at Promise._resolveFromResolver (/app/node_modules/endex/node_modules/bluebird/js/main/promise.js:427:31)
2015-05-23T11:07:18.294648772Z     at new Promise (/app/node_modules/endex/node_modules/bluebird/js/main/promise.js:53:37)
2015-05-23T11:07:18.294652389Z     at Object.endexRun [as run] (/app/node_modules/endex/index.js:130:17)
2015-05-23T11:07:18.294656144Z     at /app/index.js:16:8
2015-05-23T11:07:18.294659485Z     at tryCatcher (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/util.js:24:31)
2015-05-23T11:07:18.294662897Z     at Promise._settlePromiseFromHandler (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/promise.js:454:31)
2015-05-23T11:07:18.294666862Z     at Promise._settlePromiseAt (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/promise.js:530:18)
2015-05-23T11:07:18.294670210Z     at Promise._settlePromises (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/promise.js:646:14)
2015-05-23T11:07:18.294673750Z     at Async._drainQueue (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/async.js:177:16)
2015-05-23T11:07:18.294677000Z     at Async._drainQueues (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/async.js:187:10)
2015-05-23T11:07:18.294691659Z     at Immediate.Async.drainQueues [as _onImmediate] (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/async.js:15:14)
2015-05-23T11:07:18.294698794Z     at processImmediate [as _immediateCallback] (timers.js:358:17)
2015-05-23T11:03:47.067095017Z Detected 512 MB available memory, 512 MB limit per process (WEB_MEMORY)
2015-05-23T11:03:47.067302784Z Recommending WEB_CONCURRENCY=1
2015-05-23T11:07:17.513011019Z Detected 512 MB available memory, 512 MB limit per process (WEB_MEMORY)
2015-05-23T11:07:17.513270538Z Recommending WEB_CONCURRENCY=1

I get the same results when running plushu logs foo | grep WEB.

What's wrong with this output that's making it so sort is not working (leaving the results out of order) and grep is printing every line, unmodified, with no filtering?

Stuart P. Bentley

Make sure your pipe source is outputting to STDOUT. STDERR (as that error message is being printed on) doesn't go through the pipe, and will be output to your console unmodified. (In this case, the STDERR output was coming from the logs command in Docker.)

You can check if the output is coming from STDERR by redirecting file descriptor 2:

$ plushu logs foo 2>/tmp/test-errors.log | sort
2015-05-23T11:03:47.067095017Z Detected 512 MB available memory, 512 MB limit per process (WEB_MEMORY)
2015-05-23T11:03:47.067302784Z Recommending WEB_CONCURRENCY=1
2015-05-23T11:07:17.513011019Z Detected 512 MB available memory, 512 MB limit per process (WEB_MEMORY)
2015-05-23T11:07:17.513270538Z Recommending WEB_CONCURRENCY=1
$ cat /tmp/test-errors.log
2015-05-23T11:07:18.294569666Z Unhandled rejection RqlDriverError: First argument to `run` must be an open connection.
2015-05-23T11:07:18.294627441Z     at new RqlDriverError (/app/node_modules/endex/node_modules/rethinkdb/errors.js:14:13)
2015-05-23T11:07:18.294632871Z     at MakeArray.TermBase.run (/app/node_modules/endex/node_modules/rethinkdb/ast.js:129:29)
2015-05-23T11:07:18.294636744Z     at /app/node_modules/endex/index.js:131:24
2015-05-23T11:07:18.294640994Z     at tryCatcher (/app/node_modules/endex/node_modules/bluebird/js/main/util.js:24:31)
2015-05-23T11:07:18.294644642Z     at Promise._resolveFromResolver (/app/node_modules/endex/node_modules/bluebird/js/main/promise.js:427:31)
2015-05-23T11:07:18.294648772Z     at new Promise (/app/node_modules/endex/node_modules/bluebird/js/main/promise.js:53:37)
2015-05-23T11:07:18.294652389Z     at Object.endexRun [as run] (/app/node_modules/endex/index.js:130:17)
2015-05-23T11:07:18.294656144Z     at /app/index.js:16:8
2015-05-23T11:07:18.294659485Z     at tryCatcher (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/util.js:24:31)
2015-05-23T11:07:18.294662897Z     at Promise._settlePromiseFromHandler (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/promise.js:454:31)
2015-05-23T11:07:18.294666862Z     at Promise._settlePromiseAt (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/promise.js:530:18)
2015-05-23T11:07:18.294670210Z     at Promise._settlePromises (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/promise.js:646:14)
2015-05-23T11:07:18.294673750Z     at Async._drainQueue (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/async.js:177:16)
2015-05-23T11:07:18.294677000Z     at Async._drainQueues (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/async.js:187:10)
2015-05-23T11:07:18.294691659Z     at Immediate.Async.drainQueues [as _onImmediate] (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/async.js:15:14)
2015-05-23T11:07:18.294698794Z     at processImmediate [as _immediateCallback] (timers.js:358:17)

(The given example grep output in the original question appears unmodified because the only 4 lines that are going through STDOUT all match the pattern - all the other lines, while they don't match the pattern, are also not going through grep.)

To send STDERR output into your pipe, you can copy the file descriptor for STDOUT (&1) to STDERR (&2) for the command that is generating the STDERR output you want to go through the pipe:

$ plushu logs foo 2>&1 | sort -sk1,1
2015-05-23T11:03:47.067095017Z Detected 512 MB available memory, 512 MB limit per process (WEB_MEMORY)
2015-05-23T11:03:47.067302784Z Recommending WEB_CONCURRENCY=1
2015-05-23T11:07:17.513011019Z Detected 512 MB available memory, 512 MB limit per process (WEB_MEMORY)
2015-05-23T11:07:17.513270538Z Recommending WEB_CONCURRENCY=1
2015-05-23T11:07:18.294569666Z Unhandled rejection RqlDriverError: First argument to `run` must be an open connection.
2015-05-23T11:07:18.294627441Z     at new RqlDriverError (/app/node_modules/endex/node_modules/rethinkdb/errors.js:14:13)
2015-05-23T11:07:18.294632871Z     at MakeArray.TermBase.run (/app/node_modules/endex/node_modules/rethinkdb/ast.js:129:29)
2015-05-23T11:07:18.294636744Z     at /app/node_modules/endex/index.js:131:24
2015-05-23T11:07:18.294640994Z     at tryCatcher (/app/node_modules/endex/node_modules/bluebird/js/main/util.js:24:31)
2015-05-23T11:07:18.294644642Z     at Promise._resolveFromResolver (/app/node_modules/endex/node_modules/bluebird/js/main/promise.js:427:31)
2015-05-23T11:07:18.294648772Z     at new Promise (/app/node_modules/endex/node_modules/bluebird/js/main/promise.js:53:37)
2015-05-23T11:07:18.294652389Z     at Object.endexRun [as run] (/app/node_modules/endex/index.js:130:17)
2015-05-23T11:07:18.294656144Z     at /app/index.js:16:8
2015-05-23T11:07:18.294659485Z     at tryCatcher (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/util.js:24:31)
2015-05-23T11:07:18.294662897Z     at Promise._settlePromiseFromHandler (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/promise.js:454:31)
2015-05-23T11:07:18.294666862Z     at Promise._settlePromiseAt (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/promise.js:530:18)
2015-05-23T11:07:18.294670210Z     at Promise._settlePromises (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/promise.js:646:14)
2015-05-23T11:07:18.294673750Z     at Async._drainQueue (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/async.js:177:16)
2015-05-23T11:07:18.294677000Z     at Async._drainQueues (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/async.js:187:10)
2015-05-23T11:07:18.294691659Z     at Immediate.Async.drainQueues [as _onImmediate] (/app/node_modules/rethinkdb/node_modules/bluebird/js/main/async.js:15:14)
2015-05-23T11:07:18.294698794Z     at processImmediate [as _immediateCallback] (timers.js:358:17)

(Explanation of the -sk1,1 arguments for sorting timestamps in this answer to a separate question.)

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

Why aren't programs written in Assembly more often?

Hover affecting elements that aren't siblings

Why aren't these two sql statements returning same output?

Why aren't my unit tests running in the project output folder?

Letters aren't appearing in the output?

Programs detecting pipe on its output to disable coloration

How to pipe a command output to multiple programs

Why can't I pipe assembler output to stdout?

Why aren't "NOT IN" and "NOT EXISTS" working?

Why aren't arrays expandable?

Why aren't these vectors equal?

Why aren't Enumerations Iterable?

Why aren't these ACLs working?

Why Aren't These Functions Available?

why aren't primitives nullable

Why aren't there enough inputs?

Why aren't these strings equal

Why can't certain programs (like readlink) take input from a pipe?

Why is the output of these 2 programs not same

Python Socket Programming, other programs aren't connecting to the server

Two compiled python programs aren't connecting through pipes properly

Pipe output of two programs running in parallel to another program

Why aren't there < flex > or < grid > elements?

Why aren't the two code blocks equivalent?

Why aren't built-in arrays safe?

Why aren't my text labels appearing?

Why aren't Guava ImmutableCollections interfaces?

Why aren't these 2 expressions equivalent in bash?

Why aren't all applications 'portable'?

TOP Ranking

  1. 1

    Can't pre-populate phone number and message body in SMS link on iPhones when SMS app is not running in the background

  2. 2

    Failed to listen on localhost:8000 (reason: Cannot assign requested address)

  3. 3

    pump.io port in URL

  4. 4

    Loopback Error: connect ECONNREFUSED 127.0.0.1:3306 (MAMP)

  5. 5

    How to import an asset in swift using Bundle.main.path() in a react-native native module

  6. 6

    Spring Boot JPA PostgreSQL Web App - Internal Authentication Error

  7. 7

    3D Touch Peek Swipe Like Mail

  8. 8

    BigQuery - concatenate ignoring NULL

  9. 9

    How to how increase/decrease compared to adjacent cell

  10. 10

    Make a B+ Tree concurrent thread safe

  11. 11

    Emulator wrong screen resolution in Android Studio 1.3

  12. 12

    Can a 32-bit antivirus program protect you from 64-bit threats

  13. 13

    Svchost high CPU from Microsoft.BingWeather app errors

  14. 14

    Double spacing in rmarkdown pdf

  15. 15

    Unable to use switch toggle for dark mode in material-ui

  16. 16

    java.lang.NullPointerException: Cannot read the array length because "<local3>" is null

  17. 17

    Google Chrome Translate Page Does Not Work

  18. 18

    How to fix "pickle_module.load(f, **pickle_load_args) _pickle.UnpicklingError: invalid load key, '<'" using YOLOv3?

  19. 19

    Using Response.Redirect with Friendly URLS in ASP.NET

  20. 20

    Bootstrap 5 Static Modal Still Closes when I Click Outside

  21. 21

    SSIS setting column with data in Script Component

HotTag

Archive