0 votes

Hello,

We meet some problems to save the skipper files in a shared directory.

Our architecture :
- Skipper 3.2.8.1347 with 2 users license
- Skipper runs on a Ubuntu 16.04 session.
- The shared files are hosted on a Linux Debian local server, with CIFS protocol.

Skipper can open its file (Entity.skipper for example, rights 666).
When we try to save the file (even without change), Skipper create a backup file Entity.skipper.bak (rights 644 ; why 644?) and tries to rewrite the Entity.skipper file with rights 700!
Then an error occurs:
Exception diagnostic info:
Throw location unknown (consider using BOOSTTHROWEXCEPTION)
Dynamic exception type: boost::exceptiondetail::cloneimpl<boost::exceptiondetail::errorinfo_injector >
std::exception::what: failed opening file: Access denied

What we are doing wrong?

Many thanks!

in Solved by (140 points)
recategorized by

1 Answer

0 votes

Hi, thanks for your question. Unfortunately, this seems to be some problem in your OS. Because we don't specify any permissions during file creation. We're using standard boost file system library for creating & copying the files.

Mentioned .bak file is to protect your original file in case that save is interrupted by any way. Because of that we first create .bak file and when save is successful we remove .bak file.

For copying file from .skipper to .bak we're usign following routine (so as you can see no permissions are configured/changed):

boost::filesystem::copy_file(pathFromFile, pathToFile);

and for creating new files (.bak) we're using the following routine

//maped file params
boost::iostreams::basic_mapped_file_params<boostpath> mfp;
mfp.path = fileName.GetAsBoostPath();
mfp.flags = boost::iostreams::mapped_file::readwrite;
mfp.offset = 0;
mfp.new_file_size = lBufferSize;

//open and write to file
boost::iostreams::mapped_file_sink mf(mfp);
memory_trait::CopyMemoryBuffer(pBuffer, mf.data(), lBufferSize);

//close file
mf.close();

I believe that all files are created with permissions based on the user who run the application. Until now we don't receive any similar bug report, so I believe it has to be by some configuration on your side.

by Skipper developer (141k points)

Thanks for your quickly answer. We are going to search a solution, and we will keep you informed, if you want ;-)

Hi, I'm interested in your findings. In case it will be possible I can extend permission management directly in the app, but from what I checked there are no such options in boost library.

I've forgotten to give you an detail: in our configuration, Skipper export without problem the Doctrine entities (with rights 644 wich are OK).
If I resume, Skipper can write its backup files and the annotation files too (rights 644 for all, as given by the CIFS sharing system). On the other hand, Skipper just cannot write its main file.
Very funny, isn't it?

Anyway, it seems we have found a workaround : if we launch Skipper with the admin user (command: sudo ./Skipper), all seems working like a charm.
Note that I 've had to re-enter our licence key...

Thanks for the info and additional info.

I think I know what is causing it and I believe we will be able to fix it. The problem is probably caused by writeable memory-mapping to shared volume. Any other operations are performed directly on FS or as common file system operations.

But it's a strange that Skipper is also able to load Skipper file because memory-mapping is used also for any load/import operation. So maybe this issue occurs only during write access.

I will try to prepare you beta version with changed write logic.

Please download and test latest beta http://support.skipper18.com/402/downloads-skipper-beta

We reworked saving system to use std::fstream fsStream; approach instead of memory mapping.

Please let me know if it helps

Hi Ludek
I've tested your beta version 3.2.9.1349.
Unfortunately that version have another problem for us:
* the Skipper saved file has now the good rights (644 in our example) but its size is zero :-(
* there is no backup file generated
* the annotation files remain OK.
Thx for your help
Alain

Hi Alan, this is definitely not possible.

If you have empty file it has to be definitely something wrong on your computers/sharing.

Skipper is now using the most common way how to store files. There are no other way how to create&save&close file than this one. Also we tested it on several Linuxes including mapped samba from Windows server and everything worked fine.

Backup file is probably removed because Save operation was reported as successful (we don't validate the file content, we trust OS if he tell us that save was succesfull).

What happens when you save it to your local? And what if you save it to another mount?

And here is a screenshot from stored project file on winsrv mount https://dl.dropboxusercontent.com/u/11355235/ShareX/2017/01/2017-01-18_09-24-18.png

As you can see everything works ok. So I believe it has to be something in your mounting options.

I know we have a rather specific network architecture but it seems that doesn't annoying some other applications: for example, a text editor can write a file on our Skipper shared directory...

Our Skipper, under the local user, can save its files on the local disk or on some other mount, like NFS (but we cannot use NFS for some other reasons :-()

Perhaps we have a problem because we use different userid between the Skipper computer (uid=1000, gid=1000) and the shared server (uid=0, gid=0). Maybe that explains why Skipper works under the root user (uid=0, gid=0)?

Anyway, thanks for your help Ludek, I appreciate the time you spent trying to solve our problem!

You're welcome.

Only a quick question. As you wrote that text editor works. In case that you run text editor under the same user as Skipper, the text editor is still able to write to shared disk? This is very strange.

I believe that boost library is almost "standard" way how to access to resources and it's a strange that other apps are able to do that and boost library don't

Just a clue: I confirm a text editor (gedit for instance) is able to read/write a shared file as far as the file has the rights 666.
The difference is -maybe- that gedit does not change the rights of the file then that Skipper changes them?

I believe the difference is that Skipper moves original file to .bak (so the 666 remains to .bak), then creates new file and then deletes .bak file. And this probably causing the difference.

Are you able to create new file by gedit when exectuded by same user as Skipper on the shared file?

Yes I am. In addition I've tried to activate the backup save on gedit.
It's interesting: if I open with gedit the file test.txt (rights 666) then change and save it, gedit create the file text.txt~ with the rights 644. After that gedit can re-open the file text.txt~ but only on read-only mode.
This behavior is quite similar to that of Skipper for the annotation files but not for its own files where are the rights are completely changed (666 -> 700) during the saving. Strange.

I have one more idea. I updated save process to keep Skipper file and copy backup instead of moving Skipper file to backup. In this way the permissions could remain untouched.

Can you please try this version: https://www.dropbox.com/s/yw1zwklxcf07ri4/deploy64-all.zip?dl=0 ? (I suppose you're using 64-bit version)

Thx, but I'm not lucky ;-)
Since I use Skipper with an admin user, I cannot open it with my normal user (either with your new version or the current version). It comes:

Starting application
[Skipper for linux 64-bit] version [3.2.9.1350]
Qt libraries: loaded: 5.7.0 (compiled: 5.7.0)
** Unhandled boost exception
** Exception information:
Throw location unknown (consider using BOOSTTHROWEXCEPTION)
Dynamic exception type: boost::exceptiondetail::cloneimpl >
std::exception::what: failed opening file: Permission non accordée [translation: access denied]

Maybe there is some parameters to adjust?

This is probably because you overwrote all user files with sudo (root) permissions and now common user isn't able to access these files.

Check if application directory/files are accessible to common user as same as if Skipper data files are accessible to the user (located on these paths).

Found! There was some Skipper tmp files (in /tmp), with root rights, that stopped the launch of Skipper. I've deleted all of them.

Your version is much better, Ludek: now when I save a Skipper file (with rights 666), it keeps its rights!
That seems Ok (I have to do some other tests, of course).

Congratulations!

In a similar way, could you apply the same kind of change for the annotation files? There is not as important as my last problem, but Skipper changes also the rights of the annotation files (from 666 to 644), what remains weird.

Anyway, many thanks for your effective help!

Perfect. I'm glad it works.

Regarding the annotation files, you mean .php files, right? Can you also try it on .xml or .yml files?

Ok, I believe I know where is the problem. In case that there is export to schema files (no matter if xml, yml, php), there is such routine:

if ( CheckIfFileExists(pathToFile) )
    DeleteFile(pathToFile);

....
    boost::filesystem::copy_file(pathFromFile, pathToFile);

And pathFromFile is the temporary file created during export. I'm not sure why are permissions 644 but it's a definitely answer why permission of new file is changed.

I will try to update routine not to use copy_file but some content-replace. This should guarantee that file permissions will remain same.

I will try some changes and let you know.

It works! With this version (3.2.9.1351) the rights of Skipper files (xxx.skipper) are kept and the rights of the annotations files (xxx.php) as well.
Well done!
I'll now use this Skipper version with a real project and I hope all will be fine.
Many thanks, Ludek

Perfect. It should work smoothly because these are only two changes made in the code.

Damned! the problem remains with the xxx.skipper.bak file :-(
It comes :

Can't copy file [/media/ResidentService/ResidentService.skipper] to [/media/ResidentService/ResidentService.skipper.bak]. Error: boost::filesystem::copy_file: Access denied: "/media/ResidentService/ResidentService.skipper", "/media/ResidentService/ResidentService.skipper.bak"

or, if I create manually the xxx.skipper.bak file before (always with rights 666):

Can't delete file [/media/ResidentService/ResidentService.skipper.bak]. Error: boost::filesystem::remove: Access denied: "/media/ResidentService/ResidentService.skipper.bak"

Moreover, I wonder why sometimes Skipper tries to generate a bak file, sometimes not. Obviously, I haven't understood something :-D
Can we deactivate the bak file generation? In our case, we do not need this function...

Sooorry : it was my fault (I had not given the correct rights to my project folder).

Anyway, I continue to not understand how works the bak files generation. Does it exist some documentation about this subject?

I changed backup logic several times during testing the permission issue. In your current version is backup performed by followign steps:

//backup old files if exists
XPROGRESS_CHANNEL(Atomix::ProgressChannels::status, _TR("Backuping old project files"));
BOOST_FOREACH(TpairFileNameContent &pairFileContent, lstStoredFiles)
{
    if ( axFileHelper::CheckIfFileExists(pairFileContent.first) == true )
        axFileHelper::CopyFile(pairFileContent.first, pairFileContent.first + ".bak");
}

//creating new storage files
XPROGRESS_CHANNEL(Atomix::ProgressChannels::status, _TR("Saving to project files"));
BOOST_FOREACH(TpairFileNameContent &pairFileContent, lstStoredFiles)
{
    XString &strContent = pairFileContent.second;
    XPROGRESS_CHANNEL_FMT(Atomix::ProgressChannels::status, _TR("Saving XML file %ts"), pairFileContent.first.GetAsPCXSTR());
    axDirectoryHelper::CreateDirectoryPath(pairFileContent.first.GetPath());
    axFileHelper::SaveStringToFileUTF8(pairFileContent.first, strContent);
}

//erasing old files
XPROGRESS_CHANNEL(Atomix::ProgressChannels::status, _TR("Removing backup files"));
BOOST_FOREACH(TpairFileNameContent &pairFileContent, lstStoredFiles)
{
    if ( axFileHelper::CheckIfFileExists(pairFileContent.first + ".bak") == true )
        axFileHelper::DeleteFile(pairFileContent.first + ".bak");
}
  • So, as first step all existing skipper files are copied to new location. This cause the permission change for .bak, but enable us to keep original permission on .skipper file. (originally there was "MoveFile").

  • Then content of skipper files is stored inside the existing files (originally, there was creating of new files).

  • And as last step backup files were removed. This is the same like in previous versions.