Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Advertise on LowEndTalk.com
Cannot df -h in Proxmox
New on LowEndTalk? Please read our 'Community Rules' by clicking on it in the right menu!

Cannot df -h in Proxmox

Dear all,

I have some dedicated with Proxmox. Some of them installed from Debian OS, and the other is template from provider. The last proxmox installation was about 1 week ago. When I try to "df -h" all of my proxmox, the process need long time, reach some minutes and I need to Ctrl + c to stop the process without any result, even in my most recent installation that is not production yet.
I try to see iotop when doing df -h, no high IO usage detected. Cannot get enough result from google to determine the problem.

Do you all see the same problem with proxmox?
How can I see disk usage beside of using df -h?

Comments

  • racksxracksx Member without signature

    du -sh
    fdisk -l
    maybe you should debug your logs, check your resources and do not oversell.

  • @racksx said:
    du -sh
    fdisk -l
    maybe you should debug your logs, check your resources and do not oversell.

    The host and the VMs are not in high load, maybe nearly idle, and this is happened on all of my proxmox (about 3 host). I use them for private server. I just want to know that the others have same problem or not.

    Do you mean du -sh on /var/lib? Yes this command can show file size in the folder, but I also want to see overall usage like as df -h command. Anyway thanks for this

    I can't see free disk space with fdisk -l.

  • Do you have any non local mount points, e.g. NFS mounts on a remote system that might not be responding. NFS is a common cause of a hanging 'df'.

    Thanked by 1Falzo
  • @cochon said:
    Do you have any non local mount points, e.g. NFS mounts on a remote system that might not be responding. NFS is a common cause of a hanging 'df'.

    Only local storage. The command hang on proxmox with and without any network based storage.

  • If it's proxmox 3, sometimes /etc/pve gets stuck. You can try stopping/restarting the proxmox services (can't remember names, corosync maybe?).

    Or restart the whole machine.

  • OT: not sure if it was mentioned here already, but PVE 4.4 has been released a few days ago...

  • most probably not an proxmox related issue. I agree with @cochon that df getting stuck is most likely related to a (network) mount not responding or something on that mount in D state etc.

    UltraVPS.eu KVM in US/UK/NL/DE: 15% off first 6 month | Netcup VPS/rootDS - 5€ off: 36nc15279180197 (ref)

  • yomero said: If it's proxmox 3, sometimes /etc/pve gets stuck.

    Sorry I forget to mention proxmox version. I use version 4

    vishalpatelz said: Try df -Th

    Thank you for your suggestion, but it still hung

    @Falzo said:
    most probably not an proxmox related issue. I agree with @cochon that df getting stuck is most likely related to a (network) mount not responding or something on that mount in D state etc.

    After I read this, I try to list all mounted volume. I try command mount -l and cat /proc/self/mounts to two of my proxmox. What I realize here is that some NFS storage are still listed, even some I have deleted from the web panel in the past. Lets call this unexpected NFS volume.

    I think this is not expected result. When I try to unmount the unexpected NFS volume, for example with command "umount /mnt/backup", the umount process hung again. I try to reboot one host from terminal. It needs about 15 minutes until about 9 vm booted again. After that, I do "df -h" in host again, and finally it works. I checked mount -l, and the unexpected NFS volume dissappear.

    I try to restart some proxmox service listed in the proxmox forum here without any result.

    Is there any way to fix this without reboot?

  • @akhfa said:

    >

    Is there any way to fix this without reboot?

    I am afraid not, at least I don't know any..

    network mounts which are kind of hard disconnected while being accessed may result in hung processes or unmountable devices which then may raise issues like you described. Try to minimize use of potentially unstable nfs shares, esp. over the public internet with higher latency and such.

    Thanked by 1akhfa

    UltraVPS.eu KVM in US/UK/NL/DE: 15% off first 6 month | Netcup VPS/rootDS - 5€ off: 36nc15279180197 (ref)

  • @Falzo said:

    @akhfa said:

    >

    Is there any way to fix this without reboot?

    I am afraid not, at least I don't know any..

    network mounts which are kind of hard disconnected while being accessed may result in hung processes or unmountable devices which then may raise issues like you described. Try to minimize use of potentially unstable nfs shares, esp. over the public internet with higher latency and such.

    So if I want to remove NFS, I need to unmount all NFS in all of my proxmox? Isn't there any easier way than unmount that NFS one by one? :(

  • MakenaiMakenai Member
    edited December 2016

    Try checking if the NFS mount has any open file handles via lsof. If there are restart the applications which are using them.

    Another way would be to add an alias interface after which you should be able to run umount.
    http://stackoverflow.com/a/64180/4159217

    Also just to make sure that it is actually an NFS mount you can try running df through strace, you should be able to see what it tries to access when it hangs.

    Thanked by 2akhfa yomero
  • As a side comment, If you find you do need (or want) occasional NFS mounts on unreliable links, say for off node backups, try and set them up under automount (autofs) rather than permanent mounts, that way they're only mounted when actually needed. Though that would have to be set up outside the Proxmox control panel I think.

    Thanked by 2akhfa Falzo
  • Dear all,

    Thanks for all of your help tracing and resolving the problems.

    Solved df -h with interface alias. I can df -h again. I hope this thread can help many people having same problem :D

Sign In or Register to comment.