how to regenerate custom image in paperclip without using reprocess method - ruby-on-rails

We don't have original style. We have our custom styles.
Now whenever we reprocessing our specific style it's causing 2 issues.
1. It's deleting the specific style file.
2. Generating an empty original file(size: 0)
In below case, we're trying to regenerate small style image. But the existing image getting deleted and an empty original image is getting created.
attachment.reprocess!('small')
[paperclip] copying to local file /tmp/anm7p.gif
[AWS S3 404 0.149833 0 retries] get_object(:bucket_name=>"bucket",:key=>"attachments/19/logo_original.gif") AWS::S3::Errors::NoSuchKey No Such Key
No Such Key - cannot copy /attachments/10/logo_original.gif to local file /tmp/anm7p.gif
[AWS S3 404 0.025721 0 retries] head_object(:bucket_name=>"bucket",:key=>"attachments/10/logo_small.gif") AWS::S3::Errors::NoSuchKey No Such Key
Command :: file -b --mime '/tmp/jbguir.gif'
[paperclip] Content Type Spoof: Filename zope-logo.gif (image/gif from Headers, [#<MIME::Type: image/gif>] from Extension), content type discovered from file command: inode/x-empty. See documentation to allow this combination.
[paperclip] saving /attachments/10/logo_original.gif
[AWS S3 200 0.027039 0 retries] put_object(:acl=>:public_read,:bucket_name=>"bucket",:content_length=>0,:content_type=>"image/gif",:data=>Paperclip::AttachmentAdapter: zope-logo.gif,:key=>"attachments/10/logo_original.gif")
Any work around to get this fixed in paperclip ?

Related

CarrierWave validate content type before processing a file

I've tried everything now and I can't seem to figure out how to add content type validations in CarrierWave before performing actual processing on files I am uploading. The reason for this is that I want to allow only images but a user can upload a spoofed content file and rename file.pdf file to file.jpg. Steps I have tried so far:
photo_uploader.rb
def content_type_whitelist
/image\//
end
I have tried validate_integrity in my uploader but no luck as well.
I have also tried overwriting CarrierWave error messages (which seems to me strange to me actually):
en:
errors:
messages:
content_type_whitelist: "You are not allowed to upload %{content_type} file"
but I am getting an error from MiniMagic
"Failed to manipulate with MiniMagick, maybe it is not an image? Original Error: %{e}"
The thing is that I want to display rails validations instead so that when the content type is not one of those I have defined in my model it displays the message like "File should be one of image/jpeg, image/png, image/gif". So I need a way to force rails validations to trigger before image processing, although I think it is not quite possible.
Here they say that CarrierWave::Uploader::MagicMimeWhitelist should be included in order to perform mime types validations so I've tried that but got uninitialzied constant CarrierWave::Uploader::MagicMimeWhitelist (server restarted before)
include CarrierWave::Uploader::MagicMimeWhitelist
def whitelist_mime_type_pattern
/image\//
end
I've tried using carrier-mimetype-fu but after including CarrierWave::MimetypeFu I got unitilzied constant CarrierWave::MimetypeFu
Do you have any experience with it?
Please make sure you have Imagemagick installed on your machine
In Uploader I have included
def content_type_blacklist
%w(image/jpeg image/png image/gif)
end
to filter image files base on their content type. For image files you must have different content types.

How to specify a prefix when uploading to S3 using activestorage's direct upload?

With a standard S3 configuration:
AWS_ACCESS_KEY_ID: [AWS ID]
AWS_BUCKET: [bucket name]
AWS_REGION: [region]
AWS_SECRET_ACCESS_KEY: [secret]
I can upload a file to S3 (using direct upload) with this Rails 5.2 code (only relevant code shown):
form.file_field :my_asset, direct_upload: true
This will effectively put my asset in the root of my S3 bucket, upon submitting the form.
How can I specify a prefix (e.g. "development/", so that I can mimic a folder on S3)?
Sorry, that’s not currently possible. I’d suggest creating a bucket for Active Storage to use exclusively.

Update S3 Paperclip paths when converting data to UUID in Rails

I'm migrating several tables of existing data from sequential IDs to UUIDs, approximately a million records total. Most of them have a Paperclip file attachment on AWS S3, which are currently using the default path:
:class/:attachment/:id_partition/:style/:filename
How do I preserve the attachments during the migration?
My initial thought is to save the current pre-migration URL for each object and then post-migration use the [AWS move_to][1] command to move the file to a new URL structure that uses the UUID, which I will then set as the Paperclip :path. Is this the most efficient way to preserve the data?
Edit: How do I handle the various styles each attachment might have?

Rails/Paperclip “spoofing error” on PDF file

I'm using Paperclip in a Ruby-on-Rails app to manage various files, including uploaded PDFs. I've had no problems with it in the past, including a PDF upload just a week ago; but then a couple days ago I went to upload a new PDF and it's failing: when I try to upload the file the edit form redisplays with a red dashed line around the upload button.
The logfile has this to say:
[paperclip] Content Type Spoof: Filename hwk2.pdf (video/x-flv from Headers, [#<MIME::Type:0x0000000180e548 #content_type="application/pdf", #raw_media_type="application", #raw_sub_type="pdf", #simplified="application/pdf", #media_type="application", #sub_type="pdf", #extensions=["pdf"], #encoding="base64", #system=nil, #registered=true, #url=["IANA", "RFC3778"], #obsolete=nil, #docs=nil>] from Extension), content type discovered from file command: application/pdf. See documentation to allow this combination.
It is in fact a PDF file (which I generated myself using pdflatex), and nearly all the info here agrees on that fact, except that "Headers" seems to think it's a Flash video (video/x-flv) for some reason. What headers is this referring to and how can I fix them? I have found the workaround that simply turns off the spoofing check (by making spoofed? always return false), but I'd prefer to actually correct the error if I can.
Versions of things in case they're relevant: ruby 2.2.3p173, rails (4.0.0), passenger-4.0.59, Apache/2.4.16, paperclip (4.2.4)
Ok, figured it out, and it wasn't (I should have realised this) a RoR problem. For reasons not entirely clear to me, my Firefox installation was identifying video/x-flv as associated with the .pdf extension (in addition to the real mime type). Interestingly, I couldn't edit it within the application's preferences pane; but I could remove it from the mimeTypes.rdf file in my .mozilla/firefox/blahblah.default profile directory, and that fixed the problem.

Using Delayed Paperclip With S3 Direct Upload

I'm using Delayed Paperclip alongside direct uploads to S3. My model is called Photo and its attachment is image.
Images are uploaded to S3 using JavaScript from the Photo form. The file is stored in the location that Paperclip expects the original image to be located, and the file details are saved to hidden fields. When the form is submitted, these attributes are written to the Photo model:
image_file_name
image_file_size
image_content_type
Because writing these attributes alone doesn't seem to be enough to trigger Delayed Paperclip to process the image, after Photo.save I call Photo.image.reprocess! which does get DelayedPaperclip to create a new Sidekiq job which successfully processes the image.
The problem, is that when I call Photo.save in the PhotosController, the file is copied to a temp directory from S3, then back to S3. This happens outside of the job and is blocking:
[paperclip] copying image_assets/grab-original.tiff to local file /var/folders/bv/x495g9g10m7119680c9ssqmr0000gn/T/94943834d26bcb8b471f4eeb2a7f899d20141125-3895-1oqom7l
[AWS S3 200 2.601589 0 retries] get_object(:bucket_name=>"example-com-development",:key=>"image_assets/grab-original.tiff")
[paperclip] saving image_assets/grab-original.tiff
[AWS S3 200 2.47114 0 retries] put_object(:acl=>:public_read,:bucket_name=>"example-com-development",:cache_control=>"max-age=29030400",:content_length=>534472,:content_type=>"image/tiff",:data=>Paperclip::AttachmentAdapter: grab.tiff,:key=>"image_assets/grab-original.tiff")
Why is Paperclip copying the file down and back again?
My approach was wonky.Even if it had worked it wouldn't have added the image_processing attribute to the Photo model.
After digging into the Delayed Paperclip API, the following seems to have done the trick:
Inside PhotosController#create:
# Ensure we are flagged as processing
#media_item.photo.prepare_enqueueing_for(:image)
if #media_item.save
# Add Job
#media_item.photo.enqueue_delayed_processing
end
respond_with(:admin, #galleryable, #media_item)
I have requested a better API here: https://github.com/jrgifford/delayed_paperclip/issues/116

Resources