• This topic has 42 replies, 17 voices, and was last updated 5 months ago by poly.
Viewing 40 posts - 1 through 40 (of 43 total)
  • Quick help with a basic linux command
  • Premier Icon Earl
    Free Member

    I need to write a command but I don’t have the ability to test at the moment. Will this work?

    cd //server/Inbound/ && ls -t BBC > //server/Inbound/List_20220214.txt

    From wherever I am, go to “//server/Inbound/” and list all the files that contain ‘BBC’ anywhere in the name. Sequence it in the time I received the file and write that list to a new file called “List_20220214.txt”

    Much thanks.

    Premier Icon Earl
    Free Member

    Do * need to use the * character?

    i.e.
    cd //server/Inbound/ && ls -t *BBC* > //server/Inbound/List_20220214.txt

    Premier Icon Cougar
    Free Member

    Reckon you’ll need wildcards around BBC. ls -t *BBC*

    Also, that’s a UNC pathname, are you attaching to a Samba share?

    Premier Icon Cougar
    Free Member

    Also also,

    Do you need to supply permissions somewhere? (I’ve never used Samba)

    Premier Icon GHill
    Free Member

    You’ll need the wildcards, but looks OK for bash.

    Premier Icon Cougar
    Free Member

    Also also also,

    You’ve CD’ed to the directory then running the command. So I don’t think you’d need the full path, just the filename.

    (I’m assuming of course that CDing to a remote SMB share just like any other directory is even valid, as I said I’ve never used it.)

    Premier Icon Aidy
    Free Member

    It might be weird if you have any directories that match *BBC*

    Premier Icon Earl
    Free Member

    I understand about not needing a full path name – tick.
    Wildcard * – tick.
    UNC / Samba? – err I have heard UNC being mentioned. Need to do some more research.

    So:
    cd //server/Inbound/ && ls -t *BBC* > List_20220214.txt

    Premier Icon Earl
    Free Member

    Christ – my first google attempt bought up Lion King. Muppet!

    Premier Icon footflaps
    Full Member

    You won’t catch any filename starting with ‘.’ using ‘ls’. ‘ls -a’ will show them but doesn’t seem to pick them out when using wildcards. Best I found was ‘ls -a | grep BBC’.

    If it matters to you goodle “globbing” eg https://mywiki.wooledge.org/glob

    Premier Icon Earl
    Free Member

    And how can I add a match by extension as well as the BBC

    i.e. BBC anywhere in the name. Must end with .txt. So it will only list the first 2 files below.

    aaaa_BBC_0000.txt
    aaaa_BBC_1111.txt
    aaaa_BBC_1111.csv
    aaaa_BBC_0000.csv
    aaaa_ITV_1111.txt
    aaaa_ITV_1111.csv

    Premier Icon sofaman
    Full Member

    You won’t catch any filename starting with ‘.’ using ‘ls’

    ls .*BBC* *BBC*

    or use find.

    Premier Icon Cougar
    Free Member

    BBC anywhere in the name. Must end with .txt.

    *BBC*.txt

    * means “anything”. So it’s listing [anything]BBC[anything].txt here.

    Premier Icon poly
    Free Member

    any particular reason to string the two commands together. I know its what most of us do in real life on the command line – but sometimes its easier debugging if you just have one command.

    ls -t //server/Inbound/*BBC* > //server/Inbound/List_20220214.txt

    should work…

    Premier Icon mogrim
    Free Member

    ls -t /server/Inbound/*BBC*.txt > List_20220214.txt

    Edit: beaten to it!

    (although with the slight difference that mine will generate the output list in the current directory)

    Premier Icon Earl
    Free Member

    Sweet. Thanks very much all.

    cd //server/Inbound/ && ls -t *BBC*.txt > List_20220214.txt

    This is good fun actually but it does get crazy powerful/complex very quickly.

    Premier Icon Earl
    Free Member

    ls -t //server/Inbound/*BBC* > //server/Inbound/List_20220214.txt

    This looks better!

    Premier Icon multi21
    Free Member

    Earl
    Free Member

    I understand about not needing a full path name – tick.
    Wildcard * – tick.
    UNC / Samba? – err I have heard UNC being mentioned. Need to do some more research.

    So:
    cd //server/Inbound/ && ls -t *BBC* > List_20220214.txt

    That won’t work, you will need to mount the SMB share to a normal UNIX directory, then do your ls and > to that location.

    Premier Icon Cougar
    Free Member

    If you’ve heard UNC mentioned but not Samba,

    Is the target a Windows machine?

    Premier Icon Cougar
    Free Member

    From wherever I am

    From different machines? Assuming the remote machine is Linux I’d likely SSH to it and run the command directly on the box.

    Why don’t you tell us what you’re actually trying to achieve, then we can tell you how best to go about it? There’s little point in you asking “recommend me a hammer?” if you’re sitting there with a box of screws.

    Premier Icon tonyd
    Free Member

    Sounds like everything is in that one directory, but you do need to make sure it’s mounted first. Something like this should do you (assuming it’s not already mounted):

    mount -t cifs -o username=USER_NAME,password=PASSWORD //server/Inbound /mnt
    cd /mnt ; ls -t *BBC*txt > List_20220214.txt
    cd
    umount /mnt

    Lot’s more elegant ways to do it, but that should get you what you need (assuming permissions etc are all OK)

    Premier Icon footflaps
    Full Member

    ls .*BBC* *BBC*

    That’s what I like about Linux, even the most basic commands have so many nuances.

    Every time I look up something on Stack overflow, the depth of detail in the answers just amazes me, everything has so many options / gotchas / nuances that you never notice 99% of the time.

    Premier Icon kelvin
    Full Member

    Yeah, you have to explicitly include hidden files… because 99.9999% of the time you want them to be ignored/excluded.

    Premier Icon dudeofdoom
    Full Member

    It’s like concocting a spell unix 🙂

    Premier Icon footflaps
    Full Member

    It’s like concocting a spell unix

    My favourite ones are sed when you need to substitute ‘/’ but you have to escape it with ‘\’ so the command just looks like \/\/\/\//\/ but actually does something useful….

    Premier Icon dudeofdoom
    Full Member

    I’ve been doing stuff with docker containers,the Linux stuff is so much more fun than the windoze ones and smaller.
    (Although must admit containers are just Uber cool,either sex)

    Premier Icon sirromj
    Free Member

    Some random stuff I’ll throw in:

    now=$(date ‘+%Y%m%d-%H%M%S’) # timestamp for filename
    out=”List_$now.txt”
    srv=”//server/Inbound” # server mountpoint
    find $srv -maxdepth 1 -type f -iname ‘*BBC*’ > $srv/Inbound/”$out”

    Curious if generated file is processed further? So may not need to have the intermediary file in the first place? You can process output of find within a do/read/while loop – find is a better option for this than ls.

    Premier Icon jca
    Free Member

    My favourite ones are sed when you need to substitute ‘/’ but you have to escape it with ‘\’

    You can avoid that by using a different character in place of ‘/’ to delimt the match/replacement patterns. Just pick something that isn’t in your regex.

    (base) M-000582:~ jabbott $ echo $HOME|sed ‘s/\//:/g’
    :Users:jabbott
    (base) M-000582:~ jabbott $ echo $HOME|sed ‘s|/|:|g’
    :Users:jabbott

    Far more readable!

    Premier Icon Cougar
    Free Member

    That’s what I like about Linux, even the most basic commands have so many nuances.

    Every time I look up something on Stack overflow, the depth of detail in the answers just amazes me, everything has so many options / gotchas / nuances that you never notice 99% of the time.

    One of the things I don’t like about it is that 99% of those answers don’t work on whatever it is you’ve got in front of you. 😁

    Premier Icon tomparkin
    Free Member

    Things like ‘find the files’ are deceptively simple until they’re not because of weird edgecases and filenames that only a maniac would think up 🤷‍♂️

    Premier Icon Earl
    Free Member

    Ok. I don’t know the exact details and I’m not very clued up in this world.

    My program needs to import into a db all the CSV files in a folder that match by wildcard and extensions. Date modified ASC. Hello 1980’s.

    My program will be running on linux with the folder given to me as a UNC path name (???)

    My idea was instead of trying to read the folder a programing language – just run an os command from my program to do the heavy lifting and just read in the names of the CSV files to import – from this new temp txt file.

    I assume if my program needs to run on a Windows machine, I will just need to write the equivalent os command in DOS/powershell?

    Yes I’m out of my depth a bit. 99.99% of any coding I do is read from dB, manipulate, write to dB. Basic stuff.

    Premier Icon tomparkin
    Free Member

    When you say “import into a db” do you mean import the filenames/paths? Or reading the CSV files and generating a db based on their contents?

    The latter is much more of an undertaking of course!

    Other than that I agree, getting the shell to do the heavy lifting of the globbing sounds like a good plan. Might need to be a bit careful about things like line endings if you run on either/or Linux/Windows but that’s not an insurmountable issue by any stretch.

    Premier Icon Earl
    Free Member

    Import the contents of the csv file, a bit of transforming, insert into a DB table. Been done a million times by people down the ages.

    In terms of line endings, I hope the API (I think it’s called fgets() from memory) will take care of all that. Well I’ll find out soon!

    Premier Icon tomparkin
    Free Member

    Aye, I was thinking of the CSV that’s all straight-forward until you reach the weird edgecases and field names dreamed up by a maniac 🤪

    Uniform and consistent CSV fairly easy of course.

    Anyway, good luck with your project 👍

    Premier Icon Bikingcatastrophe
    Free Member

    Confused now. Your script earlier was supposed to be looking for .txt files and now you talk about importing csv files? Not that I am going to be much help 🙂

    Premier Icon Cougar
    Free Member

    I don’t know the exact details

    Then step away from the command line and find out. Otherwise you’re pissing into the wind.

    Where’s the database held, is that the same remote machine? What has access to what? What OSes are involved? Is this a one-off, on-demand or periodically? What are you going to do when you have this list? How are you handling files already imported? What exactly are you responsible for, why is this your problem? Etc, etc, etc. And of course, the $64,000 question, WHY are you even doing this, what’s barfing out CSVs in the first place?

    There’s a a hundred questions and a million variables here. You’ve got to define a problem accurately before you can ever hope to solve it.

    Premier Icon sofaman
    Full Member

    sed when you need to substitute ‘/’

    The delimiter is not fixed in sed, e.g. comma is used here and slash is converted to dot:

    echo “a/b/c” | sed ‘s,/,.,g’

    Premier Icon footflaps
    Full Member

    The delimiter is not fixed in sed,

    Didn’t know that!

    Learn something new every day with Linux.

    Pretty sure I’ve never seen a sed example not using ‘/’ until this thread…..

    Although back to hacking VBA today – finished the last bash project…

    Premier Icon tonyd
    Free Member

    Apologies to all but the pedant in me just had a panic attack.

    *BBC*.txt
    * means “anything”. So it’s listing [anything]BBC[anything].txt here.

    . is also a wildcard. * means any number of [anything] and . means a single instance of [anything] so you may as well do away with it in the above and just use *BBC*txt

    srv=”//server/Inbound” # server mountpoint
    find $srv -maxdepth 1 -type f -iname ‘*BBC*’ > $srv/Inbound/”$out”

    You’ve already included Inbound when you declared the $srv variable, and you shouldn’t need double quotes (depends on shell I guess)

    find is a better option for this than ls.

    Disagree – find is an excellent tool, but ls is much simpler, and if that does what you need why overcomplicate things?

    Pretty sure I’ve never seen a sed example not using ‘/’ until this thread

    Most versions of sed will automatically use the character following the s as a delimiter

    I do agree that more info is required before the problem can be answered properly, but from what the OP has described so far I would say this would do you:

    cd //server/Inbound/ ; ls -t *BBC*txt > List_20220214.txt

    Just make sure the path is correct and mounted (sounds like that part is out of your control).

    Premier Icon footflaps
    Full Member

    One of the things I don’t like about it is that 99% of those answers don’t work on whatever it is you’ve got in front of you.

    A lot of the systems I’m on are embedded and run a bespoke busy box implementation of Linux, so limited command set and the commands you do have will only run a subset of normal options e.g. there’s no find command but you can run “ls -R | grep”.

Viewing 40 posts - 1 through 40 (of 43 total)

You must be logged in to reply to this topic.

Thanks for popping by - why not stay a while?IT'S FREE

Sign up as a Singletrack Member and you can leave comments on stories, use the classified ads, and post in our forums, do quizzes and more.

Join us, join in, it’s free, and fun.